Nonlinear Kalman Filters (KFs) are powerful and widely-used techniques when trying to estimate the hidden state of a stochastic nonlinear dynamic system. A novel sample-based KF is the Smart Sampling Kalman Filter (S²KF). It is based on deterministic Gaussian samples which are obtained from an offline optimization procedure. Although this sampling technique is quite effective, it does not preserve the point symmetry of the Gaussian distribution. In this paper, we overcome this issue by extending the S²KF with a new point-symmetric Gaussian sampling scheme to improve its estimation quality. Moreover, we also improve the numerical stability of the sample computation. This allows us to accurately approximate thousand-dimensional Gaussian distributions using tens of thousands of optimally placed and equally weighted samples. We evaluate the new symmetric S²KF by computing higher-order moments of standard normal distributions and investigate the estimation quality of the S²KF when dealing with symmetric measurement equations. Additionally, extended object tracking based on many measurements per time step is considered. This high-dimensional estimation problem shows the advantage of the S²KF being able to use an arbitrary number of samples independent of the state dimension, in contrast to other state-of-the-art sample-based Kalman Filters. Finally, other estimators also relying on the S²KF’s Gaussian sampling technique, e.g., the Progressive Gaussian Filter (PGF), will benefit from the new point-symmetric sampling as well.