A Scene Text Detector for Text with Arbitrary Shapes

The performance of text detection is crucial for the subsequent recognition task. Currently, the accuracy of the text detector still needs further improvement, particularly those with irregular shapes in a complex environment. We propose a pixel-wise method based on instance segmentation for scene text detection. Specifically, a text instance is split into five components: a Text Skeleton and four Directional Pixel Regions, then restoring itself based on these elements and receiving supplementary information from other areas when one fails. Besides, a Confidence Scoring Mechanism is designed to filter characters similar to text instances. Experiments on several challenging benchmarks demonstrate that our method achieves state-of-the-art results in scene text detection with an F-measure of 84.6% on Total-Text and 86.3% on CTW1500.

Download Full-text

All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6896 ◽

2020 ◽

Vol 34 (07) ◽

pp. 12160-12167 ◽

Cited By ~ 5

Author(s):

Hao Wang ◽

Pu Lu ◽

Hui Zhang ◽

Mingkun Yang ◽

Xiang Bai ◽

...

Keyword(s):

Computer Vision ◽

State Of The Art ◽

Text Detection ◽

Text Recognition ◽

Scene Text Detection ◽

Scene Text ◽

End To End ◽

Text Spotting ◽

Set Of Points ◽

Instance Segmentation

Recently, end-to-end text spotting that aims to detect and recognize text from cluttered images simultaneously has received particularly growing interest in computer vision. Different from the existing approaches that formulate text detection as bounding box extraction or instance segmentation, we localize a set of points on the boundary of each text instance. With the representation of such boundary points, we establish a simple yet effective scheme for end-to-end text spotting, which can read the text of arbitrary shapes. Experiments on three challenging datasets, including ICDAR2015, TotalText and COCO-Text demonstrate that the proposed method consistently surpasses the state-of-the-art in both scene text detection and end-to-end text recognition tasks.

Download Full-text

Scene Text Detection with Recurrent Instance Segmentation

2018 24th International Conference on Pattern Recognition (ICPR) ◽

10.1109/icpr.2018.8545073 ◽

2018 ◽

Author(s):

Wei Feng ◽

Wen-Hao He ◽

Fei Yin ◽

Cheng-Lin Liu

Keyword(s):

Text Detection ◽

Scene Text Detection ◽

Scene Text ◽

Instance Segmentation

Download Full-text

Multilingual Scene Text Detection Using Gradient Morphology

International Journal of Computer Vision and Image Processing ◽

10.4018/ijcvip.2020070103 ◽

2020 ◽

Vol 10 (3) ◽

pp. 31-43 ◽

Cited By ~ 2

Author(s):

Dibyajyoti Dhar ◽

Neelotpal Chakraborty ◽

Sayan Choudhury ◽

Ashis Paul ◽

Ayatullah Faruk Mollah ◽

...

Keyword(s):

State Of The Art ◽

High Sensitivity ◽

Text Detection ◽

Interesting Problem ◽

Natural Scene ◽

The Past ◽

Scene Text Detection ◽

Scene Text ◽

Multiple Languages ◽

Natural Scene Images

Text detection in natural scene images is an interesting problem in the field of information retrieval. Several methods have been proposed over the past few decades for scene text detection. However, the robustness and efficiency of these methods are downgraded due to high sensitivity towards various complexities of an image. Also, in multi-lingual environment where texts may occur in multiple languages, a method may not be suitable for detecting scene texts in certain languages. To counter these challenges, a gradient morphology-based method is proposed in this paper that proves to be robust against image complexities and efficiently detects scene texts irrespective of their languages. The method is validated using low quality images from standard multi-lingual datasets like MSRA-TD500 and MLe2e. The performance of the method is compared with that of some state-of-the-art methods, and comparably better results are observed.

Download Full-text

Real-time Arabic scene text detection using fully convolutional neural networks

International Journal of Electrical and Computer Engineering (IJECE) ◽

10.11591/ijece.v11i2.pp1634-1640 ◽

2021 ◽

Vol 11 (2) ◽

pp. 1634

Author(s):

Rajae Moumen ◽

Raddouane Chiheb ◽

Rdouan Faizi

Keyword(s):

Real Time ◽

Data Augmentation ◽

State Of The Art ◽

Arabic Language ◽

Text Detection ◽

The State ◽

Convolutional Network ◽

Fully Convolutional Network ◽

Scene Text Detection ◽

Scene Text

The aim of this research is to propose a fully convolutional approach to address the problem of real-time scene text detection for Arabic language. Text detection is performed using a two-steps multi-scale approach. The first step uses light-weighted fully convolutional network: TextBlockDetector FCN, an adaptation of VGG-16 to eliminate non-textual elements, localize wide scale text and give text scale estimation. The second step determines narrow scale range of text using fully convolutional network for maximum performance. To evaluate the system, we confront the results of the framework to the results obtained with single VGG-16 fully deployed for text detection in one-shot; in addition to previous results in the state-of-the-art. For training and testing, we initiate a dataset of 575 images manually processed along with data augmentation to enrich training process. The system scores a precision of 0.651 vs 0.64 in the state-of-the-art and a FPS of 24.3 vs 31.7 for a VGG-16 fully deployed.

Download Full-text

Scene Text Detection with Supervised Pyramid Context Network

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33019038 ◽

2019 ◽

Vol 33 ◽

pp. 9038-9045 ◽

Cited By ~ 30

Author(s):

Enze Xie ◽

Yuhang Zang ◽

Shuai Shao ◽

Gang Yu ◽

Cong Yao ◽

...

Keyword(s):

Semantic Information ◽

State Of The Art ◽

Text Detection ◽

Detection Methods ◽

Natural Scenes ◽

The Past ◽

Scene Text Detection ◽

Scene Text ◽

Previous State ◽

Feature Pyramid

Scene text detection methods based on deep learning have achieved remarkable results over the past years. However, due to the high diversity and complexity of natural scenes, previous state-of-the-art text detection methods may still produce a considerable amount of false positives, when applied to images captured in real-world environments. To tackle this issue, mainly inspired by Mask R-CNN, we propose in this paper an effective model for scene text detection, which is based on Feature Pyramid Network (FPN) and instance segmentation. We propose a supervised pyramid context network (SPCNET) to precisely locate text regions while suppressing false positives.Benefited from the guidance of semantic information and sharing FPN, SPCNET obtains significantly enhanced performance while introducing marginal extra computation. Experiments on standard datasets demonstrate that our SPCNET clearly outperforms start-of-the-art methods. Specifically, it achieves an F-measure of 92.1% on ICDAR2013, 87.2% on ICDAR2015, 74.1% on ICDAR2017 MLT and 82.9% on

Download Full-text

MFECN: Multi-level Feature Enhanced Cumulative Network for Scene Text Detection

ACM Transactions on Multimedia Computing Communications and Applications ◽

10.1145/3440087 ◽

2021 ◽

Vol 17 (3) ◽

pp. 1-22

Author(s):

Zhandong Liu ◽

Wengang Zhou ◽

Houqiang Li

Keyword(s):

Text Detection ◽

Detection Algorithms ◽

Features Fusion ◽

Irregular Shapes ◽

Scene Text Detection ◽

Scene Text ◽

Text Information ◽

Multi Level ◽

Public Datasets ◽

High Level

Recently, many scene text detection algorithms have achieved impressive performance by using convolutional neural networks. However, most of them do not make full use of the context among the hierarchical multi-level features to improve the performance of scene text detection. In this article, we present an efficient multi-level features enhanced cumulative framework based on instance segmentation for scene text detection. At first, we adopt a Multi-Level Features Enhanced Cumulative ( MFEC ) module to capture features of cumulative enhancement of representational ability. Then, a Multi-Level Features Fusion ( MFF ) module is designed to fully integrate both high-level and low-level MFEC features, which can adaptively encode scene text information. To verify the effectiveness of the proposed method, we perform experiments on six public datasets (namely, CTW1500, Total-text, MSRA-TD500, ICDAR2013, ICDAR2015, and MLT2017), and make comparisons with other state-of-the-art methods. Experimental results demonstrate that the proposed Multi-Level Features Enhanced Cumulative Network (MFECN) detector can well handle scene text instances with irregular shapes (i.e., curved, oriented, and horizontal) and achieves better or comparable results.

Download Full-text

Real-Time Scene Text Detection with Differentiable Binarization

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6812 ◽

2020 ◽

Vol 34 (07) ◽

pp. 11474-11481 ◽

Cited By ~ 9

Author(s):

Minghui Liao ◽

Zhaoyi Wan ◽

Cong Yao ◽

Kai Chen ◽

Xiang Bai

Keyword(s):

State Of The Art ◽

Text Detection ◽

Detection Accuracy ◽

Post Processing ◽

Segmentation Method ◽

Performance Improvements ◽

Scene Text Detection ◽

Scene Text ◽

Benchmark Datasets ◽

Bounding Boxes

Recently, segmentation-based methods are quite popular in scene text detection, as the segmentation results can more accurately describe scene text of various shapes such as curve text. However, the post-processing of binarization is essential for segmentation-based detection, which converts probability maps produced by a segmentation method into bounding boxes/regions of text. In this paper, we propose a module named Differentiable Binarization (DB), which can perform the binarization process in a segmentation network. Optimized along with a DB module, a segmentation network can adaptively set the thresholds for binarization, which not only simplifies the post-processing but also enhances the performance of text detection. Based on a simple segmentation network, we validate the performance improvements of DB on five benchmark datasets, which consistently achieves state-of-the-art results, in terms of both detection accuracy and speed. In particular, with a light-weight backbone, the performance improvements by DB are significant so that we can look for an ideal tradeoff between detection accuracy and efficiency. Specifically, with a backbone of ResNet-18, our detector achieves an F-measure of 82.8, running at 62 FPS, on the MSRA-TD500 dataset. Code is available at: https://github.com/MhLiao/DB.

Download Full-text

An Algorithm for Scene Text Detection Using Multibox and Semantic Segmentation

Applied Sciences ◽

10.3390/app9061054 ◽

2019 ◽

Vol 9 (6) ◽

pp. 1054 ◽

Cited By ~ 3

Author(s):

Hongbo Qin ◽

Haodi Zhang ◽

Hai Wang ◽

Yujin Yan ◽

Min Zhang ◽

...

Keyword(s):

Semantic Segmentation ◽

Text Detection ◽

Experimental Results ◽

Natural Scene ◽

Maximum Increase ◽

Correction Process ◽

Bounding Box ◽

Scene Text Detection ◽

Scene Text ◽

F Measure

An outside mutual correction (OMC) algorithm for natural scene text detection using multibox and semantic segmentation was developed. In the OMC algorithm, semantic segmentation and multibox were processed in parallel, and the text detection results were mutually corrected. The mutual correction process was divided into two steps: (1) The semantic segmentation results were employed in the bounding box enhancement module (BEM) to correct the multibox results. (2) The semantic bounding box module (SBM) was used to optimize the adhesion text boundary of the semantic segmentation results. Non-maximum suppression (NMS) was adopted to merge the SBM and BEM results. Our algorithm was evaluated on the ICDAR2013 and SVT datasets. The experimental results show that the developed algorithm had a maximum increase of 13.62% in the F-measure score and the highest F-measure score was 81.38%.

Download Full-text