Neural Galerkin schemes build on the Dirac-Frenkel variational principle to train nonlinear parameterizations sequentially in time to approximation solution fields of time-dependent partial differential equations. Because of the nonlinearity of the parameterizations, few parameters are sufficient to approximate even complex system dynamics. In this work, we show how quantities such as energy, impulse and entropy, which often form the basis of first-principle models, can be conserved in Neural Galerkin schemes. We first reformulate the Galerkin residual equation that is used to evolve the parameters over time so that the quantities are conserved in the limit of continuous time dynamics. Second, we introduce a nonlinear projection that ensures conservation of quantities even after discretizing time and in realistic numerical computations with deep networks beyond idealized settings. Numerical experiments with shallow water and wave equations demonstrate that quantities are preserved up to machine precision.