Searchable Turkish OCRed historical newspaper collection 1928–1942

2021 ◽  
pp. 016555152110006
Author(s):  
Houssem Menhour ◽  
Hasan Basri Şahin ◽  
Ramazan Nejdet Sarıkaya ◽  
Medine Aktaş ◽  
Rümeysa Sağlam ◽  
...  
Keyword(s):  
Character Recognition ◽  
Optical Character Recognition ◽  
Named Entity Recognition ◽  
Early Modern Period ◽  
Entity Recognition ◽  
Text Search ◽  
Cultural Form ◽  
Named Entity ◽  
Search Mechanism ◽  
Original Website

The newspaper emerged as a distinct cultural form in early 17th-century Europe. It is bound up with the early modern period of history. Historical newspapers are of utmost importance to nations and its people, and researchers from different disciplines rely on these papers to improve our understanding of the past. In pursuit of satisfying this need, Istanbul University Head Office of Library and Documentation provides access to a big database of scanned historical newspapers. To take it another step further and make the documents more accessible, we need to run optical character recognition (OCR) and named entity recognition (NER) tasks on the whole database and index the results to allow for full-text search mechanism. We design and implement a system encompassing the whole pipeline starting from scrapping the dataset from the original website to providing a graphical user interface to run search queries, and it manages to do that successfully. Proposed system provides to search people, culture and security-related keywords and to visualise them.

scholarly journals Combining the NER-OCR methods to improve information retrieval efficiency in the Indonesian posters

2020 ◽  
Vol 8 (4) ◽  
pp. 263-269
Author(s):  
Ahmad Syarif Rosidy ◽  
Tubagus Mohammad Akhriza ◽  
Mochammad Husni
Keyword(s):  
Information Retrieval ◽  
Character Recognition ◽  
Optical Character Recognition ◽  
Named Entity Recognition ◽  
Training Data ◽  
Entity Recognition ◽  
Time Efficiency ◽  
Named Entity ◽  
Retrieval Efficiency ◽  
Information Retrieval Methods

Event organizers in Indonesia often use websites to disseminate information about these events through digital posters. However, manually processing for transferring information from posters to websites is constrained by time efficiency, given the increasing number of posters uploaded. Also, information retrieval methods, such as Named Entity Recognition (NER) for Indonesian posters, are still rarely discussed in the literature. In contrast, the NER method application to Indonesian corpus is challenged by accuracy improvement because Indonesian is a low-resource language that causes a lack of corpus availability as a reference. This study proposes a solution to improve the efficiency of information extraction time from digital posters. The proposed solution is a combination of the NER method with the Optical Character Recognition (OCR) method to recognize text on posters developed with the support of relevant training data corpus to improve accuracy. The experimental results show that the system can increase time efficiency by 94 % with 82-92 % accuracy for several extracted information entities from 50 testing digital posters.

scholarly journals Emerging trends: Deep nets for poets

2021 ◽  
Vol 27 (5) ◽  
pp. 631-645
Author(s):  
Kenneth Ward Church ◽  
Xiaopeng Yuan ◽  
Sheng Guo ◽  
Zewu Wu ◽  
Yehua Yang ◽  
...  
Keyword(s):  
Character Recognition ◽  
Optical Character Recognition ◽  
Question Answering ◽  
Named Entity Recognition ◽  
Fine Tuning ◽  
Entity Recognition ◽  
Training Models ◽  
Named Entity ◽  
Emerging Trends ◽  
Programming Skills

AbstractDeep nets have done well with early adopters, but the future will soon depend on crossing the chasm. The goal of this paper is to make deep nets more accessible to a broader audience including people with little or no programming skills, and people with little interest in training new models. A github is provided with simple implementations of image classification, optical character recognition, sentiment analysis, named entity recognition, question answering (QA/SQuAD), machine translation, speech to text (SST), and speech recognition (STT). The emphasis is on instant gratification. Non-programmers should be able to install these programs and use them in 15 minutes or less (per program). Programs are short (10–100 lines each) and readable by users with modest programming skills. Much of the complexity is hidden behind abstractions such as pipelines and auto classes, and pretrained models and datasets provided by hubs: PaddleHub, PaddleNLP, HuggingFaceHub, and Fairseq. Hubs have different priorities than research. Research is training models from corpora and fine-tuning them for tasks. Users are already overwhelmed with an embarrassment of riches (13k models and 1k datasets). Do they want more? We believe the broader market is more interested in inference (how to run pretrained models on novel inputs) and less interested in training (how to create even more models).

Developing a RadLex-based Named Entity Recognition Tool for Mining Textual Radiology Reports (Preprint)

2020 ◽  
Author(s):  
Shintaro Tsuji ◽  
Andrew Wen ◽  
Naoki Takahashi ◽  
Hongjian Zhang ◽  
Katsuhiko Ogasawara ◽  
...  
Keyword(s):  
Named Entity Recognition ◽  
Noun Phrases ◽  
General Purpose ◽  
Entity Recognition ◽  
Free Text ◽  
Clinical Text ◽  
Named Entity ◽  
Radiology Reports ◽  
Two Measures ◽  
F Measure

BACKGROUND Named entity recognition (NER) plays an important role in extracting the features of descriptions for mining free-text radiology reports. However, the performance of existing NER tools is limited because the number of entities depends on its dictionary lookup. Especially, the recognition of compound terms is very complicated because there are a variety of patterns. OBJECTIVE The objective of the study is to develop and evaluate a NER tool concerned with compound terms using the RadLex for mining free-text radiology reports. METHODS We leveraged the clinical Text Analysis and Knowledge Extraction System (cTAKES) to develop customized pipelines using both RadLex and SentiWordNet (a general-purpose dictionary, GPD). We manually annotated 400 of radiology reports for compound terms (Cts) in noun phrases and used them as the gold standard for the performance evaluation (precision, recall, and F-measure). Additionally, we also created a compound-term-enhanced dictionary (CtED) by analyzing false negatives (FNs) and false positives (FPs), and applied it for another 100 radiology reports for validation. We also evaluated the stem terms of compound terms, through defining two measures: an occurrence ratio (OR) and a matching ratio (MR). RESULTS The F-measure of the cTAKES+RadLex+GPD was 32.2% (Precision 92.1%, Recall 19.6%) and that of combined the CtED was 67.1% (Precision 98.1%, Recall 51.0%). The OR indicated that stem terms of “effusion”, "node", "tube", and "disease" were used frequently, but it still lacks capturing Cts. The MR showed that 71.9% of stem terms matched with that of ontologies and RadLex improved about 22% of the MR from the cTAKES default dictionary. The OR and MR revealed that the characteristics of stem terms would have the potential to help generate synonymous phrases using ontologies. CONCLUSIONS We developed a RadLex-based customized pipeline for parsing radiology reports and demonstrated that CtED and stem term analysis has the potential to improve dictionary-based NER performance toward expanding vocabularies.

Identification of Food Quality Descriptors in Customer Chat Conversations using Named Entity Recognition

2020 ◽  
Author(s):  
Aditya Kiran Brahma ◽  
Prathyush Potluri ◽  
Meghana Kanapaneni ◽  
Sumanth Prabhu ◽  
Sundeep Teki
Keyword(s):  
Food Quality ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity
Sign in / Sign up

Export Citation Format

Share Document