Adaptive signal processing is used in broad areas. In most practical adaptive systems, there exists substantial nonlinearity that cannot be neglected. In this paper, we analyze the behaviors of an adaptive system in which the output of the adaptive filter has the clipping saturation-type nonlinearity by a statistical-mechanical method. To represent the macroscopic state of the system, we introduce two macroscopic variables. By considering the limit in which the number of taps of the unknown system and adaptive filter is large, we derive the simultaneous differential equations that describe the system behaviors in the deterministic and closed form. Although the derived simultaneous differential equations cannot be analytically solved, we discuss the dynamical behaviors and steady state of the adaptive system by asymptotic analysis, steady-state analysis, and numerical calculation. As a result, it becomes clear that the saturation value S has the critical value SC at which the mean-square stability of the adaptive system is lost. That is, when S > SC, both the mean-square error (MSE) and mean-square deviation (MSD) converge, i.e., the adaptive system is mean-square stable. On the other hand, when S < SC, the MSD diverges although the MSE converges, i.e., the adaptive system is not mean-square stable. In the latter case, the converged value of the MSE is a quadratic function of S and does not depend on the step size. Finally, SC is exactly derived by asymptotic analysis.<br>