It is difficult for traditional image recognition methods to accurately identify ground penetrating radar (GPR) images. This paper proposes a deep-learning based Faster R-CNN algorithm for the automatic classification and recognition of GPR images. Firstly, GPR images with different features were obtained by using gprMax, a professional GPR simulation software. Then, the feature of the target in the image was taken as the recognition object and the data set was made. Finally, Faster R-CNN’s recognition ability of GPR images was analyzed from various accuracy, average accuracy and other indicators. The results showed that Faster R-CNN could successfully identify GPR images and accurately classify them, with an average accuracy rate of 93.9%.