In AlxCoCrFeNi high entropy alloys (x = 0.3–0.5), the NiAl phase with the B2 structure is precipitated rapidly along the fcc grain boundaries. During recrystallization after conventional cold rolling, the NiAl precipitates effectively suppress the grain growth, which results in the ultrafine-grained microstructure. It should be noted that no severe plastic deformation is necessary to obtain the microstructure. The volume fraction of the NiAl precipitates increases with increasing x. As a result, the average grain size of the fcc matrix (dm) after the recrystallization decreases with increasing x, and therefore, a minimum dm of 0.5 μm can be obtained at x = 0.5. The grain refinement by the NiAl precipitates is consistent with the Zener-Smith model. At x = 0.5, the alloy with dm = 0.5 μm exhibits a yield stress of 1163 MPa and an elongation of 24% at room temperature.