Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting

Many approaches have recently been proposed to detect irregular scene text and achieved promising results. However, their localization results may not well satisfy the following text recognition part mainly because of two reasons: 1) recognizing arbitrary shaped text is still a challenging task, and 2) prevalent non-trainable pipeline strategies between text detection and text recognition will lead to suboptimal performances. To handle this incompatibility problem, in this paper we propose an end-to-end trainable text spotting approach named Text Perceptron. Concretely, Text Perceptron first employs an efficient segmentation-based text detector that learns the latent text reading order and boundary information. Then a novel Shape Transform Module (abbr. STM) is designed to transform the detected feature regions into regular morphologies without extra parameters. It unites text detection and the following recognition part into a whole framework, and helps the whole network achieve global optimization. Experiments show that our method achieves competitive performance on two standard text benchmarks, i.e., ICDAR 2013 and ICDAR 2015, and also obviously outperforms existing methods on irregular text benchmarks SCUT-CTW1500 and Total-Text.

Download Full-text

All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6896 ◽

2020 ◽

Vol 34 (07) ◽

pp. 12160-12167 ◽

Cited By ~ 5

Author(s):

Hao Wang ◽

Pu Lu ◽

Hui Zhang ◽

Mingkun Yang ◽

Xiang Bai ◽

...

Keyword(s):

Computer Vision ◽

State Of The Art ◽

Text Detection ◽

Text Recognition ◽

Scene Text Detection ◽

Scene Text ◽

End To End ◽

Text Spotting ◽

Set Of Points ◽

Instance Segmentation

Recently, end-to-end text spotting that aims to detect and recognize text from cluttered images simultaneously has received particularly growing interest in computer vision. Different from the existing approaches that formulate text detection as bounding box extraction or instance segmentation, we localize a set of points on the boundary of each text instance. With the representation of such boundary points, we establish a simple yet effective scheme for end-to-end text spotting, which can read the text of arbitrary shapes. Experiments on three challenging datasets, including ICDAR2015, TotalText and COCO-Text demonstrate that the proposed method consistently surpasses the state-of-the-art in both scene text detection and end-to-end text recognition tasks.

Download Full-text

Scene text spotting based on end-to-end

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-200903 ◽

2021 ◽

pp. 1-11

Author(s):

Guangcun Wei ◽

Wansheng Rong ◽

Yongquan Liang ◽

Xinguang Xiao ◽

Xiang Liu

Keyword(s):

Semantic Information ◽

State Of The Art ◽

Recognition Task ◽

Text Detection ◽

Text Recognition ◽

Scene Text ◽

End To End ◽

Text Spotting ◽

The One ◽

Text Context

Aiming at the problem that the traditional OCR processing method ignores the inherent connection between the text detection task and the text recognition task, This paper propose a novel end-to-end text spotting framework. The framework includes three parts: shared convolutional feature network, text detector and text recognizer. By sharing convolutional feature network, the text detection network and the text recognition network can be jointly optimized at the same time. On the one hand, it can reduce the computational burden; on the other hand, it can effectively use the inherent connection between text detection and text recognition. This model add the TCM (Text Context Module) on the basis of Mask RCNN, which can effectively solve the negative sample problem in text detection tasks. This paper propose a text recognition model based on the SAM-BiLSTM (spatial attention mechanism with BiLSTM), which can more effectively extract the semantic information between characters. This model significantly surpasses state-of-the-art methods on a number of text detection and text spotting benchmarks, including ICDAR 2015, Total-Text.

Download Full-text

Unconstrained Bilingual Scene Text Reading Using Octave as a Feature Extractor

Applied Sciences ◽

10.3390/app10134474 ◽

2020 ◽

Vol 10 (13) ◽

pp. 4474 ◽

Cited By ~ 1

Author(s):

Direselign Addis Tadesse ◽

Chuan-Ming Liu ◽

Van-Dai Ta

Keyword(s):

Region Of Interest ◽

Input Image ◽

Text Detection ◽

Fine Tuning ◽

Proposed Model ◽

Scene Text ◽

Feature Extractor ◽

Reading Text ◽

Text Reading ◽

End To End

Reading text and unified text detection and recognition from natural images are the most challenging applications in computer vision and document analysis. Previously proposed end-to-end scene text reading methods do not consider the frequency of input images at feature extraction, which slows down the system, requires more memory, and recognizes text inaccurately. In this paper, we proposed an octave convolution (OctConv) feature extractor and a time-restricted attention encoder-decoder module for end-to-end scene text reading. The OctConv can extract features by factorizing the input image based on their frequency. It is a direct replacement of convolutions, orthogonal and complementary, for reducing redundancies and helps to boost the reading text through low memory requirements at a faster speed. In the text reading process, features are first extracted from the input image using Feature Pyramid Network (FPN) with OctConv Residual Network with depth 50 (ResNet50). Then, a Region Proposal Network (RPN) is applied to predict the location of the text area by using extracted features. Finally, a time-restricted attention encoder-decoder module is applied after the Region of Interest (RoI) pooling is performed. A bilingual real and synthetic scene text dataset is prepared for training and testing the proposed model. Additionally, well-known datasets including ICDAR2013, ICDAR2015, and Total Text are used for fine-tuning and evaluating its performance with previously proposed state-of-the-art methods. The proposed model shows promising results on both regular and irregular or curved text detection and reading tasks.

Download Full-text