The coupling between the crankshaft and the camshaft is neglected before in fault diagnosis which may lead to incomplete fault information. In this paper, a new torsional coupling model of a diesel generator transmission system is proposed for fault diagnosis. The natural frequency and forced torsional vibration response of the model are obtained by the system matrix method and Newmark-β method. For the system without considering the lumped mass of camshafts, some key natural frequencies are lost. The vibration dynamics are compared for the transmission system with and without the new coupling model. And important frequency responses are missed in the spectrums of the forced torsional vibration without the new coupling model. Finally, the new coupling model is implemented in fault diagnosis and the cause of an unusual vibration fault is deduced in the simulation, which confirms the feasibility of the proposed model in fault diagnosis.