In the paper, a fault feature extraction method for rotor system is proposed based on Hilbert marginal spectrum. Compared with the spectrum analysis method via Fourier transformation, it is more effective for the rotating machinery vibrating signal analysis. Extracting the rotor system fault feature frequency from Hilbert marginal spectrum can not only enhance the frequency resolution, but also remove other unrelated frequency component, so as to make the spectrum peak of the fault feature frequency more obviously, and the analysis diagnosis results more accurately. This method result is applied to the fault feature extraction and diagnosis of the rotor system, and the analysis results of the experiment signal verify the validity of this method.