The stiffness of a rigid body subject to conservative forces and moments is described by a tensor, whose components are best described by a 6×6 Cartesian stiffness matrix. We derive an expression that is independent of the parameterization of the motion of the rigid body using methods of differential geometry. The components of the tensor with respect to a basis of twists are given by evaluating the tensor on a pair of basis twists. We show that this tensor depends on the choice of an affine connection on the Lie group, SE3. In addition, we show that the definition of the Cartesian stiffness matrix used in the literature [1,2] implicitly assumes an asymmetric connection and this results in an asymmetric stiffness matrix in a general loaded configuration. We prove that by choosing a symmetric connection we always obtain a symmetric Cartesian stiffness matrix. Finally, we derive stiffness matrices for different connections and illustrate the calculations using numerical examples.