scholarly journals Cascade Network with Deformable Composite Backbone for Formula Detection in Scanned Document Images

2021 ◽  
Vol 11 (16) ◽  
pp. 7610
Author(s):  
Khurram Azeem Hashmi ◽  
Alain Pagani ◽  
Marcus Liwicki ◽  
Didier Stricker ◽  
Muhammad Zeshan Afzal
Keyword(s):  
Computer Vision ◽  
Relative Error ◽  
State Of The Art ◽  
Error Reduction ◽  
Detection Accuracy ◽  
Document Images ◽  
Backbone Network ◽  
Mathematical Formulas ◽  
Composite Connections ◽  
Areas Of Interest

This paper presents a novel architecture for detecting mathematical formulas in document images, which is an important step for reliable information extraction in several domains. Recently, Cascade Mask R-CNN networks have been introduced to solve object detection in computer vision. In this paper, we suggest a couple of modifications to the existing Cascade Mask R-CNN architecture: First, the proposed network uses deformable convolutions instead of conventional convolutions in the backbone network to spot areas of interest better. Second, it uses a dual backbone of ResNeXt-101, having composite connections at the parallel stages. Finally, our proposed network is end-to-end trainable. We evaluate the proposed approach on the ICDAR-2017 POD and Marmot datasets. The proposed approach demonstrates state-of-the-art performance on ICDAR-2017 POD at a higher IoU threshold with an f1-score of 0.917, reducing the relative error by 7.8%. Moreover, we accomplished correct detection accuracy of 81.3% on embedded formulas on the Marmot dataset, which results in a relative error reduction of 30%.

scholarly journals Cascade Network with Deformable Composite Backbone for Formula Detection in Scanned Document Images

2021 ◽  
Author(s):  
Khurram Azeem Hashmi ◽  
Alain Pagani ◽  
Marcus Liwicki ◽  
Didier Stricker ◽  
Muhammad Zeshan Afzal
Keyword(s):  
Computer Vision ◽  
Relative Error ◽  
State Of The Art ◽  
Error Reduction ◽  
Detection Accuracy ◽  
Document Images ◽  
Backbone Network ◽  
Mathematical Formulas ◽  
Composite Connections ◽  
Areas Of Interest

This paper presents a novel architecture for detecting mathematical formulas in document images, which is an important step for reliable information extraction in several domains. Recently, Cascade Mask R-CNN networks have been introduced to solve object detection in computer vision. In this paper, we suggest a couple of modifications to the existing Cascade Mask R-CNN architecture: First, the proposed network uses deformable convolutions instead of conventional convolutions in the backbone network to spot areas of interest better. Second, it uses a dual backbone of ResNeXt-101, having composite connections at the parallel stages. Finally, our proposed network is end-to-end trainable. We evaluate the proposed approach on the ICDAR-2017 POD and Marmot datasets. The proposed approach demonstrates state-of-the-art performance on ICDAR-2017 POD at a higher IoU threshold with an f1-score of 0.917, reducing the relative error by 7.8%. Moreover, we accomplished correct detection accuracy of 81.3% on embedded formulas on the Marmot dataset, which results in a relative error reduction of 30%.

scholarly journals CasTabDetectoRS: Cascade Network for Table Detection in Document Images with Recursive Feature Pyramid and Switchable Atrous Convolution

2021 ◽  
Author(s):  
Khurram Azeem Hashmi ◽  
Alain Pagani ◽  
Marcus Liwicki ◽  
Didier Stricker ◽  
Muhammad Zeshan Afzal
Keyword(s):  
Relative Error ◽  
State Of The Art ◽  
Error Reduction ◽  
Reliable Information ◽  
Document Images ◽  
Post Processing ◽  
Preliminary Step ◽  
Backbone Networks ◽  
Previous State ◽  
Feature Pyramid

Table detection is a preliminary step in extracting reliable information from tables in scanned document images. We present CasTabDetectoRS, a novel end-to-end trainable table detection framework that operates on Cascade Mask R-CNN, including Recursive Feature Pyramid network and Switchable Atrous Convolution in the existing backbone architecture. By utilizing a comparatively lightweight backbone of ResNet-50, this paper demonstrates that superior results are attainable without relying on pre and post-processing methods, heavier backbone networks (ResNet-101, ResNeXt-152), and memory-intensive deformable convolutions. We evaluate the proposed approach on five different publicly available table detection datasets. Our CasTabDetectoRS outperforms the previous state-of-the-art results on four datasets (ICDAR-19, TableBank, UNLV, and Marmot) and accomplishes comparable results on ICDAR-17 POD. Upon comparing with previous state-of-the-art results, we obtain a significant relative error reduction of 56.36%, 20%, 4.5%, and 3.5% on the datasets of ICDAR-19, TableBank, UNLV, and Marmot, respectively. Furthermore, this paper sets a new benchmark by performing exhaustive cross-datasets evaluations to exhibit the generalization capabilities of the proposed method.

scholarly journals CasTabDetectoRS: Cascade Network for Table Detection in Document Images with Recursive Feature Pyramid and Switchable Atrous Convolution

2021 ◽  
Vol 7 (10) ◽  
pp. 214
Author(s):  
Khurram Hashmi ◽  
Alain Pagani ◽  
Marcus Liwicki ◽  
Didier Stricker ◽  
Muhammad Zeshan Afzal
Keyword(s):  
Relative Error ◽  
State Of The Art ◽  
Error Reduction ◽  
Reliable Information ◽  
Document Images ◽  
Post Processing ◽  
Preliminary Step ◽  
Backbone Networks ◽  
Previous State ◽  
Feature Pyramid

Table detection is a preliminary step in extracting reliable information from tables in scanned document images. We present CasTabDetectoRS, a novel end-to-end trainable table detection framework that operates on Cascade Mask R-CNN, including Recursive Feature Pyramid network and Switchable Atrous Convolution in the existing backbone architecture. By utilizing a comparativelyightweight backbone of ResNet-50, this paper demonstrates that superior results are attainable without relying on pre- and post-processing methods, heavier backbone networks (ResNet-101, ResNeXt-152), and memory-intensive deformable convolutions. We evaluate the proposed approach on five different publicly available table detection datasets. Our CasTabDetectoRS outperforms the previous state-of-the-art results on four datasets (ICDAR-19, TableBank, UNLV, and Marmot) and accomplishes comparable results on ICDAR-17 POD. Upon comparing with previous state-of-the-art results, we obtain a significant relative error reduction of 56.36%, 20%, 4.5%, and 3.5% on the datasets of ICDAR-19, TableBank, UNLV, and Marmot, respectively. Furthermore, this paper sets a new benchmark by performing exhaustive cross-datasets evaluations to exhibit the generalization capabilities of the proposed method.

scholarly journals HybridTabNet: Towards Better Table Detection in Scanned Document Images

2021 ◽  
Author(s):  
Muhammad Zeshan Afzal ◽  
Khurram Hashmi ◽  
Marcus Liwicki ◽  
Didier Stricker ◽  
Danish Nazir ◽  
...  
Keyword(s):  
State Of The Art ◽  
Document Image ◽  
Superior Performance ◽  
Document Images ◽  
Backbone Network ◽  
Unseen Data ◽  
Crucial Information ◽  
Leave One Out ◽  
Processing Steps ◽  
Prior State

Tables in the document image are one of the most important entities since they contain crucial information. Therefore, accurate table detection can significantly improve information extraction from tables. In this work, we present a novel end-to-end trainable pipeline, HybridTabNet, for table detection in scanned document images. Our two-stage table detector uses the ResNeXt-101 backbone for feature extraction and Hybrid Task Cascade (HTC) to localize the tables in scanned document images. Moreover, we replace conventional convolutions with deformable convolutions in the backbone network. This enables our network to detect tables of arbitrary layouts precisely. We evaluate our approach comprehensively on ICDAR-13, ICDAR-17 POD, ICDAR-19, TableBank, Marmot, and UNLV. Apart from the ICDAR-17 POD dataset, our proposed HybridTabNet outperforms earlier state-of-the-art results without depending on pre and post-processing steps. Furthermore, to investigate how the proposed method generalizes unseen data, we conduct an exhaustive leave-one-out-evaluation. In comparison to prior state-of-the-art results, our method reduces the relative error by 27.57% on ICDAR-2019-TrackA-Modern, 42.64% on TableBank (Latex), 41.33% on TableBank (Word), 55.73% on TableBank (Latex + Word), 10% on Marmot, and 9.67% on UNLV dataset. The achieved results reflect the superior performance of the proposed method.

scholarly journals DeHyFoNet: Deformable Hybrid Network for Formula Detection in Scanned Document Images

2022 ◽  
Author(s):  
Muhammad Zeshan Afzal ◽  
Khurram Azeem Hashmi ◽  
Alain Pagani ◽  
Marcus Liwicki ◽  
Didier Stricker
Keyword(s):  
Information Extraction ◽  
Reduction Rate ◽  
Error Reduction ◽  
Hybrid Network ◽  
Document Images ◽  
Mathematical Formulas ◽  
End To End

This work presents an approach for detecting mathematical formulas in scanned document images. The proposed approach is end-to-end trainable. Since many OCR engines cannot reliably work with the formulas, it is essential to isolate them to obtain the clean text for information extraction from the document. Our proposed pipeline comprises a hybrid task cascade network with deformable convolutions and a Resnext101 backbone. Both of these modifications help in better detection. We evaluate the proposed approaches on the ICDAR-2017 POD and Marmot datasets and achieve an overall accuracy of 96% for the ICDAR-2017 POD dataset. We achieve an overall reduction of error of 13%. Furthermore, the results on Marmot datasets are improved for the isolated and embedded formulas. We achieved an accuracy of 98.78% for the isolated formula and 90.21% overall accuracy for embedded formulas. Consequently, it results in an error reduction rate of 43% for isolated and 17.9% for embedded formulas.

scholarly journals YOLOv3-Lite: A Lightweight Crack Detection Network for Aircraft Structure Based on Depthwise Separable Convolutions

2019 ◽  
Vol 9 (18) ◽  
pp. 3781 ◽  
Author(s):  
Yadan Li ◽  
Zhenqi Han ◽  
Haoyu Xu ◽  
Lizhuang Liu ◽  
Xiaoqiang Li ◽  
...  
Keyword(s):  
High Resolution ◽  
Crack Detection ◽  
Detection Method ◽  
State Of The Art ◽  
Aviation Industry ◽  
Detection Accuracy ◽  
Aircraft Structure ◽  
Aircraft Structures ◽  
Backbone Network ◽  
Feature Pyramid

Due to the high proportion of aircraft faults caused by cracks in aircraft structures, crack inspection in aircraft structures has long played an important role in the aviation industry. The existing approaches, however, are time-consuming or have poor accuracy, given the complex background of aircraft structure images. In order to solve these problems, we propose the YOLOv3-Lite method, which combines depthwise separable convolution, feature pyramids, and YOLOv3. Depthwise separable convolution is employed to design the backbone network for reducing parameters and for extracting crack features effectively. Then, the feature pyramid joins together low-resolution, semantically strong features at a high-resolution for obtaining rich semantics. Finally, YOLOv3 is used for the bounding box regression. YOLOv3-Lite is a fast and accurate crack detection method, which can be used on aircraft structure such as fuselage or engine blades. The result shows that, with almost no loss of detection accuracy, the speed of YOLOv3-Lite is 50% more than that of YOLOv3. It can be concluded that YOLOv3-Lite can reach state-of-the-art performance.

scholarly journals CBNet: A Novel Composite Backbone Network Architecture for Object Detection

2020 ◽  
Vol 34 (07) ◽  
pp. 11653-11660 ◽  
Author(s):  
Yudong Liu ◽  
Yongtao Wang ◽  
Siwei Wang ◽  
Tingting Liang ◽  
Qijie Zhao ◽  
...  
Keyword(s):  
Object Detection ◽  
Network Architecture ◽  
State Of The Art ◽  
Feature Maps ◽  
Single Model ◽  
Backbone Network ◽  
Novel Strategy ◽  
Composite Connections ◽  
High Level ◽  
Instance Segmentation

In existing CNN based detectors, the backbone network is a very important component for basic feature1 extraction, and the performance of the detectors highly depends on it. In this paper, we aim to achieve better detection performance by building a more powerful backbone from existing ones like ResNet and ResNeXt. Specifically, we propose a novel strategy for assembling multiple identical backbones by composite connections between the adjacent backbones, to form a more powerful backbone named Composite Backbone Network (CBNet). In this way, CBNet iteratively feeds the output features of the previous backbone, namely high-level features, as part of input features to the succeeding backbone, in a stage-by-stage fashion, and finally the feature maps of the last backbone (named Lead Backbone) are used for object detection. We show that CBNet can be very easily integrated into most state-of-the-art detectors and significantly improve their performances. For example, it boosts the mAP of FPN, Mask R-CNN and Cascade R-CNN on the COCO dataset by about 1.5 to 3.0 points. Moreover, experimental results show that the instance segmentation results can be improved as well. Specifically, by simply integrating the proposed CBNet into the baseline detector Cascade Mask R-CNN, we achieve a new state-of-the-art result on COCO dataset (mAP of 53.3) with a single model, which demonstrates great effectiveness of the proposed CBNet architecture. Code will be made available at https://github.com/PKUbahuangliuhe/CBNet.

scholarly journals Flower End-to-End Detection Based on YOLOv4 Using a Mobile Device

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Zhibin Cheng ◽  
Fuquan Zhang
Keyword(s):  
Computer Vision ◽  
Loss Function ◽  
Mobile Device ◽  
Detection Method ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Detection Accuracy ◽  
Detection Algorithms ◽  
End To End ◽  
Detection Speed

In this paper, a novel flower detection application anchor-based method is proposed, which is combined with an attention mechanism to detect the flowers in a smart garden in AIoT more accurately and fast. While many researchers have paid much attention to the flower classification in existing studies, the issue of flower detection has been largely overlooked. The problem we have outlined deals largely with the study of a new design and application of flower detection. Firstly, a new end-to-end flower detection anchor-based method is inserted into the architecture of the network to make it more precious and fast and the loss function and attention mechanism are introduced into our model to suppress unimportant features. Secondly, our flower detection algorithms can be integrated into the mobile device. It is revealed that our flower detection method is very considerable through a series of investigations carried out. The detection accuracy of our method is similar to that of the state-of-the-art, and the detection speed is faster at the same time. It makes a major contribution to flower detection in computer vision.

scholarly journals A Framework for Instantaneous Driver Drowsiness Detection Based on Improved HOG Features and Naïve Bayesian Classification

2021 ◽  
Vol 11 (2) ◽  
pp. 240
Author(s):  
Samy Bakheet ◽  
Ayoub Al-Hamadi
Keyword(s):  
Computer Vision ◽  
State Of The Art ◽  
Bayesian Classification ◽  
Experimental Results ◽  
Detection Accuracy ◽  
Simple Computation ◽  
Histogram Of Oriented Gradient ◽  
Drowsiness Detection ◽  
Driver Drowsiness ◽  
Strong Contender

Due to their high distinctiveness, robustness to illumination and simple computation, Histogram of Oriented Gradient (HOG) features have attracted much attention and achieved remarkable success in many computer vision tasks. In this paper, an innovative framework for driver drowsiness detection is proposed, where an adaptive descriptor that possesses the virtue of distinctiveness, robustness and compactness is formed from an improved version of HOG features based on binarized histograms of shifted orientations. The final HOG descriptor generated from binarized HOG features is fed to the trained Naïve Bayes (NB) classifier to make the final driver drowsiness determination. Experimental results on the publicly available NTHU-DDD dataset verify that the proposed framework has the potential to be a strong contender for several state-of-the-art baselines, by achieving a competitive detection accuracy of 85.62%, without loss of efficiency or stability.

scholarly journals HybridTabNet: Towards Better Table Detection in Scanned Document Images

2021 ◽  
Vol 11 (18) ◽  
pp. 8396
Author(s):  
Danish Nazir ◽  
Khurram Azeem Hashmi ◽  
Alain Pagani ◽  
Marcus Liwicki ◽  
Didier Stricker ◽  
...  
Keyword(s):  
State Of The Art ◽  
Superior Performance ◽  
Document Images ◽  
Post Processing ◽  
Backbone Network ◽  
Unseen Data ◽  
Crucial Information ◽  
Leave One Out ◽  
Processing Steps ◽  
Prior State

Tables in document images are an important entity since they contain crucial information. Therefore, accurate table detection can significantly improve the information extraction from documents. In this work, we present a novel end-to-end trainable pipeline, HybridTabNet, for table detection in scanned document images. Our two-stage table detector uses the ResNeXt-101 backbone for feature extraction and Hybrid Task Cascade (HTC) to localize the tables in scanned document images. Moreover, we replace conventional convolutions with deformable convolutions in the backbone network. This enables our network to detect tables of arbitrary layouts precisely. We evaluate our approach comprehensively on ICDAR-13, ICDAR-17 POD, ICDAR-19, TableBank, Marmot, and UNLV. Apart from the ICDAR-17 POD dataset, our proposed HybridTabNet outperformed earlier state-of-the-art results without depending on pre- and post-processing steps. Furthermore, to investigate how the proposed method generalizes unseen data, we conduct an exhaustive leave-one-out-evaluation. In comparison to prior state-of-the-art results, our method reduced the relative error by 27.57% on ICDAR-2019-TrackA-Modern, 42.64% on TableBank (Latex), 41.33% on TableBank (Word), 55.73% on TableBank (Latex + Word), 10% on Marmot, and 9.67% on the UNLV dataset. The achieved results reflect the superior performance of the proposed method.

Sign in / Sign up

Export Citation Format

Share Document