The defect identification of tunnel lining is a task with a lot of tasks and time-consuming work, and currently, it mainly relies on manual operation. This paper takes the ground-penetrating radar image of the internal defects of the lining as the research object, and chooses the popular VGG16, ResNet34 convolutional neural network (CNN) to build the automatic recognition model for comparative study, and proposes an improved ResNet34 defect-recognition model. In this paper, SGD and Adam training algorithms are used to update network parameters, and the PyTorch depth framework is used to train the network. The test results show that the ResNet34 network has faster convergence speed, higher accuracy rate, and shorter training time than the VGG16 network. The ResNet34 network using the Adam algorithm can achieve 99.08% accuracy. The improved ResNet34 network can achieve an accuracy of 99.25%, and at the same, reduce the parameter amount by 4.22% compared with the ResNet34 network, which can better identify defects in the lining. The research in this paper shows that the deep learning method can provide new ideas for the identification of tunnel lining defects.