This paper uses the simulation technology of computer numerical simulation, combined with the GAMBIT numerical modeling software and ANSYS structure analysis software, we carry out an in-depth research and analysis on the stability of engineering simulation, at the same times establish the mathematical model of ANSYS numerical simulation displacement stability and design the ANSYS command stream program algorithm. Finally, this paper uses the open-pit to underground mining slope stability as an example, to verify the reliability of the model and algorithm. Through the numerical simulation, we get the displacement of slope in the Y direction and Z direction. In the Y direction, the maximum displacement is 0m, the negative maximum displacement is 5.88m and displacement is relatively large; in the Z direction, the maximum displacement is 1.32M, the negative maximum displacement is 1.08M and the stability of slope is different in different position. The numerical simulation provides the theory reference for the safety of mining engineering.