Scene text spotting based on end-to-end

2021 ◽  
pp. 1-11
Author(s):  
Guangcun Wei ◽  
Wansheng Rong ◽  
Yongquan Liang ◽  
Xinguang Xiao ◽  
Xiang Liu
Keyword(s):  
Semantic Information ◽  
State Of The Art ◽  
Recognition Task ◽  
Text Detection ◽  
Text Recognition ◽  
Scene Text ◽  
End To End ◽  
Text Spotting ◽  
The One ◽  
Text Context

Aiming at the problem that the traditional OCR processing method ignores the inherent connection between the text detection task and the text recognition task, This paper propose a novel end-to-end text spotting framework. The framework includes three parts: shared convolutional feature network, text detector and text recognizer. By sharing convolutional feature network, the text detection network and the text recognition network can be jointly optimized at the same time. On the one hand, it can reduce the computational burden; on the other hand, it can effectively use the inherent connection between text detection and text recognition. This model add the TCM (Text Context Module) on the basis of Mask RCNN, which can effectively solve the negative sample problem in text detection tasks. This paper propose a text recognition model based on the SAM-BiLSTM (spatial attention mechanism with BiLSTM), which can more effectively extract the semantic information between characters. This model significantly surpasses state-of-the-art methods on a number of text detection and text spotting benchmarks, including ICDAR 2015, Total-Text.

scholarly journals All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting

2020 ◽  
Vol 34 (07) ◽  
pp. 12160-12167 ◽  
Author(s):  
Hao Wang ◽  
Pu Lu ◽  
Hui Zhang ◽  
Mingkun Yang ◽  
Xiang Bai ◽  
...  
Keyword(s):  
Computer Vision ◽  
State Of The Art ◽  
Text Detection ◽  
Text Recognition ◽  
Scene Text Detection ◽  
Scene Text ◽  
End To End ◽  
Text Spotting ◽  
Set Of Points ◽  
Instance Segmentation

Recently, end-to-end text spotting that aims to detect and recognize text from cluttered images simultaneously has received particularly growing interest in computer vision. Different from the existing approaches that formulate text detection as bounding box extraction or instance segmentation, we localize a set of points on the boundary of each text instance. With the representation of such boundary points, we establish a simple yet effective scheme for end-to-end text spotting, which can read the text of arbitrary shapes. Experiments on three challenging datasets, including ICDAR2015, TotalText and COCO-Text demonstrate that the proposed method consistently surpasses the state-of-the-art in both scene text detection and end-to-end text recognition tasks.

scholarly journals Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting

2020 ◽  
Vol 34 (07) ◽  
pp. 11899-11907 ◽  
Author(s):  
Liang Qiao ◽  
Sanli Tang ◽  
Zhanzhan Cheng ◽  
Yunlu Xu ◽  
Yi Niu ◽  
...  
Keyword(s):  
Global Optimization ◽  
Text Detection ◽  
Text Recognition ◽  
Scene Text ◽  
Reading Order ◽  
Standard Text ◽  
Text Reading ◽  
Boundary Information ◽  
End To End ◽  
Text Spotting

Many approaches have recently been proposed to detect irregular scene text and achieved promising results. However, their localization results may not well satisfy the following text recognition part mainly because of two reasons: 1) recognizing arbitrary shaped text is still a challenging task, and 2) prevalent non-trainable pipeline strategies between text detection and text recognition will lead to suboptimal performances. To handle this incompatibility problem, in this paper we propose an end-to-end trainable text spotting approach named Text Perceptron. Concretely, Text Perceptron first employs an efficient segmentation-based text detector that learns the latent text reading order and boundary information. Then a novel Shape Transform Module (abbr. STM) is designed to transform the detected feature regions into regular morphologies without extra parameters. It unites text detection and the following recognition part into a whole framework, and helps the whole network achieve global optimization. Experiments show that our method achieves competitive performance on two standard text benchmarks, i.e., ICDAR 2013 and ICDAR 2015, and also obviously outperforms existing methods on irregular text benchmarks SCUT-CTW1500 and Total-Text.

scholarly journals A Scene Text Detector for Text with Arbitrary Shapes

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Weijia Wu ◽  
Jici Xing ◽  
Cheng Yang ◽  
Yuxing Wang ◽  
Hong Zhou
Keyword(s):  
State Of The Art ◽  
Recognition Task ◽  
Text Detection ◽  
Supplementary Information ◽  
Complex Environment ◽  
Irregular Shapes ◽  
Scene Text Detection ◽  
Scene Text ◽  
F Measure ◽  
Instance Segmentation

The performance of text detection is crucial for the subsequent recognition task. Currently, the accuracy of the text detector still needs further improvement, particularly those with irregular shapes in a complex environment. We propose a pixel-wise method based on instance segmentation for scene text detection. Specifically, a text instance is split into five components: a Text Skeleton and four Directional Pixel Regions, then restoring itself based on these elements and receiving supplementary information from other areas when one fails. Besides, a Confidence Scoring Mechanism is designed to filter characters similar to text instances. Experiments on several challenging benchmarks demonstrate that our method achieves state-of-the-art results in scene text detection with an F-measure of 84.6% on Total-Text and 86.3% on CTW1500.

scholarly journals Scene Text Detection with Supervised Pyramid Context Network

2019 ◽  
Vol 33 ◽  
pp. 9038-9045 ◽  
Author(s):  
Enze Xie ◽  
Yuhang Zang ◽  
Shuai Shao ◽  
Gang Yu ◽  
Cong Yao ◽  
...  
Keyword(s):  
Semantic Information ◽  
State Of The Art ◽  
Text Detection ◽  
Detection Methods ◽  
Natural Scenes ◽  
The Past ◽  
Scene Text Detection ◽  
Scene Text ◽  
Previous State ◽  
Feature Pyramid

Scene text detection methods based on deep learning have achieved remarkable results over the past years. However, due to the high diversity and complexity of natural scenes, previous state-of-the-art text detection methods may still produce a considerable amount of false positives, when applied to images captured in real-world environments. To tackle this issue, mainly inspired by Mask R-CNN, we propose in this paper an effective model for scene text detection, which is based on Feature Pyramid Network (FPN) and instance segmentation. We propose a supervised pyramid context network (SPCNET) to precisely locate text regions while suppressing false positives.Benefited from the guidance of semantic information and sharing FPN, SPCNET obtains significantly enhanced performance while introducing marginal extra computation. Experiments on standard datasets demonstrate that our SPCNET clearly outperforms start-of-the-art methods. Specifically, it achieves an F-measure of 92.1% on ICDAR2013, 87.2% on ICDAR2015, 74.1% on ICDAR2017 MLT and 82.9% on

scholarly journals Multi-granularity Deep Local Representations for Irregular Scene Text Recognition

2021 ◽  
Vol 2 (2) ◽  
pp. 1-18
Author(s):  
Hongchao Gao ◽  
Yujia Li ◽  
Jiao Dai ◽  
Xi Wang ◽  
Jizhong Han ◽  
...  
Keyword(s):  
State Of The Art ◽  
Visual Representation ◽  
Text Recognition ◽  
Natural Scene ◽  
Attention Network ◽  
Training Time ◽  
Scene Text ◽  
Benchmark Datasets ◽  
Local Representations ◽  
Scene Text Recognition

Recognizing irregular text from natural scene images is challenging due to the unconstrained appearance of text, such as curvature, orientation, and distortion. Recent recognition networks regard this task as a text sequence labeling problem and most networks capture the sequence only from a single-granularity visual representation, which to some extent limits the performance of recognition. In this article, we propose a hierarchical attention network to capture multi-granularity deep local representations for recognizing irregular scene text. It consists of several hierarchical attention blocks, and each block contains a Local Visual Representation Module (LVRM) and a Decoder Module (DM). Based on the hierarchical attention network, we propose a scene text recognition network. The extensive experiments show that our proposed network achieves the state-of-the-art performance on several benchmark datasets including IIIT-5K, SVT, CUTE, SVT-Perspective, and ICDAR datasets under shorter training time.

scholarly journals DetReco: Object-Text Detection and Recognition Based on Deep Neural Network

2020 ◽  
Vol 2020 ◽  
pp. 1-15
Author(s):  
Fan Zhang ◽  
Jiaxing Luan ◽  
Zhichao Xu ◽  
Wei Chen
Keyword(s):  
Intelligent Transportation Systems ◽  
Research Effort ◽  
Recognition Task ◽  
Autonomous Driving ◽  
Transportation Systems ◽  
Text Detection ◽  
Text Recognition ◽  
Test Time ◽  
Average Precision ◽  
Detection And Recognition

Deep learning-based object detection method has been applied in various fields, such as ITS (intelligent transportation systems) and ADS (autonomous driving systems). Meanwhile, text detection and recognition in different scenes have also attracted much attention and research effort. In this article, we propose a new object-text detection and recognition method termed “DetReco” to detect objects and texts and recognize the text contents. The proposed method is composed of object-text detection network and text recognition network. YOLOv3 is used as the algorithm for the object-text detection task and CRNN is employed to deal with the text recognition task. We combine the datasets of general objects and texts together to train the networks. At test time, the detection network detects various objects in an image. Then, the text images are passed to the text recognition network to derive the text contents. The experiments show that the proposed method achieves 78.3 mAP (mean Average Precision) for general objects and 72.8 AP (Average Precision) for texts in regard to detection performance. Furthermore, the proposed method is able to detect and recognize affine transformed or occluded texts with robustness. In addition, for the texts detected around general objects, the text contents can be used as the identifier to distinguish the object.

Multilingual Scene Text Detection Using Gradient Morphology

2020 ◽  
Vol 10 (3) ◽  
pp. 31-43 ◽  
Author(s):  
Dibyajyoti Dhar ◽  
Neelotpal Chakraborty ◽  
Sayan Choudhury ◽  
Ashis Paul ◽  
Ayatullah Faruk Mollah ◽  
...  
Keyword(s):  
State Of The Art ◽  
High Sensitivity ◽  
Text Detection ◽  
Interesting Problem ◽  
Natural Scene ◽  
The Past ◽  
Scene Text Detection ◽  
Scene Text ◽  
Multiple Languages ◽  
Natural Scene Images

Text detection in natural scene images is an interesting problem in the field of information retrieval. Several methods have been proposed over the past few decades for scene text detection. However, the robustness and efficiency of these methods are downgraded due to high sensitivity towards various complexities of an image. Also, in multi-lingual environment where texts may occur in multiple languages, a method may not be suitable for detecting scene texts in certain languages. To counter these challenges, a gradient morphology-based method is proposed in this paper that proves to be robust against image complexities and efficiently detects scene texts irrespective of their languages. The method is validated using low quality images from standard multi-lingual datasets like MSRA-TD500 and MLe2e. The performance of the method is compared with that of some state-of-the-art methods, and comparably better results are observed.

scholarly journals Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition

2019 ◽  
Vol 33 ◽  
pp. 8610-8617 ◽  
Author(s):  
Hui Li ◽  
Peng Wang ◽  
Chunhua Shen ◽  
Guyu Zhang
Keyword(s):  
State Of The Art ◽  
Text Recognition ◽  
Natural Scene ◽  
Fine Grained ◽  
Word Level ◽  
Scene Text ◽  
Sophisticated Model ◽  
Regular Text ◽  
Algorithm Implementation ◽  
Scene Text Recognition

Recognizing irregular text in natural scene images is challenging due to the large variance in text appearance, such as curvature, orientation and distortion. Most existing approaches rely heavily on sophisticated model designs and/or extra fine-grained annotations, which, to some extent, increase the difficulty in algorithm implementation and data collection. In this work, we propose an easy-to-implement strong baseline for irregular scene text recognition, using offthe-shelf neural network components and only word-level annotations. It is composed of a 31-layer ResNet, an LSTMbased encoder-decoder framework and a 2-dimensional attention module. Despite its simplicity, the proposed method is robust. It achieves state-of-the-art performance on irregular text recognition benchmarks and comparable results on regular text datasets. The code will be released.

Sign in / Sign up

Export Citation Format

Share Document