This paper reports the recoveries of iron, chromium, and nickel from pickling sludge using coal-based smelting reduction. The influences of slag basicity (CaO/SiO2, which is controlled by high phosphorus oolitic hematite iron ores), reduction temperature, reduction time, and the C/O mole ratio on the recoveries of Fe, Cr, and Ni are investigated systematically. The experimental results show that high recoveries of Fe (98.91%), Cr (98.46%), and Ni (99.44%) are produced from pickling sludge with optimized parameters for the smelting reduction process. The optimized parameters are a slag basicity of 1.5; a reduction temperature of 1550 °C, a reduction time of 90 min, and a C/O mole ratio of 2.0. These parameters can be used as technical support for the recycling of pickling sludge with pyrometallurgy.