The neutron-proton mass difference in nuclear matter is calculated within the context of chiral models involving nucleon and meson degrees of freedom. The interplay of chiral symmetry and charge symmetry breaking on neutron and proton self-energies are discussed in a relativistic Hartree–Fock approximation. Exchange terms are crucial to obtain different contributions for the neutron and proton self-energies. Density dependence of meson masses and coupling constants are taken into account. We find that the neutron-proton mass difference in nuclear matter increases as the density increases, contrary to the predictions of several quark models and of QCD sum rules at finite density.