The ill-posedness of nonlinear identification equation in time domain of structural dynamics system is studied and a new calculating method to weaken the influence of ill-posedness is proposed. Damped least squares method is an algorithm of Jacobian matrix positive-definable, which can obtain the solution of ill-posed nonlinear identification equation. But the solution is sensitive to the test noise of response in time domain of the structure. To solve the problem of instability of the solution, a new calculating method is proposed which combines damped least squares method with Tikhonov regularization method. First, the estimate of structural parameters is introduced to Tikhonov regularization function, and a more stable identification equation in time domain can be obtained. Second, the identification equation is solved with damped least squares method, and the iterative result is an approximate solution of the former ill-posed problem. The numerical example shows that the new method in this paper is efficient to solve the ill-posed nonlinear identification equation in time domain.