Analog circuit fault diagnosis technology is widely used in the diagnosis of various electronic devices. The basic strategy is to extract circuit fault characteristics and then to use a clustering algorithm for diagnosis. The discrete Volterra series (DVS) is a common feature extraction method; however, it is difficult to calculate its parameters. To solve the problem of feature extraction in fault diagnosis, we propose an improved hierarchical Levenberg–Marquardt (LM)–DVS algorithm (IDVS). First, the DVS is simplified on the basis of the hierarchical symmetry of the memory parameters, the LM strategy is used to optimize the coefficients, and a Bayesian information criterion based on the symmetry of entropy is introduced for order selection. Finally, we propose a fault diagnosis method by combining the improved DVS algorithm and a condensed nearest neighbor algorithm (CNN) (i.e., the IDVS–CNN method). A simulation experiment was conducted to verify the feature extraction and fault diagnosis ability of the IDVS–CNN. The results show that the proposed method outperforms conventional methods in terms of the macro and micro F1 scores (0.903 and 0.894, respectively), which is conducive to the efficient application of fault diagnosis. In conclusion, the improved method in this study is helpful to simplify the calculation of the DVS parameters of circuit faults in analog electronic systems, and provides new insights for the prospective application of circuit fault diagnosis, system modeling, and pattern recognition.