PurposeBook sorting system is one of specific application in smart library scenarios, and it now has been widely used in most libraries based on RFID (radio-frequency identification devices) technology. Book identification processing is one of the core parts of a book sorting system, and the efficiency and accuracy of book identification are extremely critical to all libraries. In this paper, the authors propose a new image recognition method to identify books in libraries based on barcode decoding together with deep learning optical character recognition (OCR) and describe its application in library book identification processing.Design/methodology/approachThe identification process relies on recognition of the images or videos of the book cover moving on a conveyor belt. Barcode is printed on or attached to the surface of each book. Deep learning OCR program is applied to improve the accuracy of recognition, especially when the barcode is blurred or faded. The approach the authors proposed is robust with high accuracy and good performance, even though input pictures are not in high resolution and the book covers are not always vertical.FindingsThe proposed method with deep learning OCR achieves best accuracy in different vertical, skewed and blurred image conditions.Research limitations/implicationsMethods that the authors proposed need to cooperate and practice in different book sorting machine.Social implicationsThe authors collected more than 500 books from a library. These photos display the cover of more than 100 randomly picked books with backgrounds in different colors, each of which has about five different pictures captured from variety angles. The proposed method combines traditional barcode identification algorithm with the authors’ modification to locate and deskew the image. And deep learning OCR is involved to enhance the accuracy when the barcode is blurred or partly faded. Book sorting system design based on this method will also be introduced.Originality/valueExperiment demonstrates that the accuracy of the proposed method is high in real-time test and achieves good accuracy even when the barcode is blurred. Deep learning is very effective in analyzing image content, and a corresponding series of methods have been formed in video content understanding, which can be a greater advantage and play a role in the application scene of intelligent library.