Sm (Co or/and Zn) substituted nanocrystalline barium hexaferrites were synthesized by the sol–gel auto-combustion process, then X-ray diffraction (XRD) and vibrating sample magnetometer (VSM) were used to characterize and discuss the phase composition and the magnetic properties of the as-prepared barium hexaferrites. All results showed that the phase composition and magnetic properties were closely related to the doping elements and x. Owing to hyperfine field, canting spin, magnetic dilution and impurity phases, the saturation magnetization (Ms) of all samples increased firstly, and then decreased. Considering the crystallization, the magnetocrystalline anisotropy and the ions occupancy, the doping elements and x critically affected the coercivity (Hc). Compared the magnetic properties of all the samples, it is concluded that Zn 2+ and Co 2+ influenced each other and Zn 2+ occupied 4f2 sites prior to Co 2+, which led to the increase of Ms and the decrease of Hc.