scholarly journals On the Limitations of Visual-Semantic Embedding Networks for Image-to-Text Information Retrieval

2021 ◽  
Vol 7 (8) ◽  
pp. 125
Author(s):  
Yan Gong ◽  
Georgina Cosma ◽  
Hui Fang
Keyword(s):  
Information Retrieval ◽  
Question Answering ◽  
Text Retrieval ◽  
Future Research ◽  
Image Captioning ◽  
Image Objects ◽  
Semantic Embedding ◽  
Average Recall ◽  
Text Information ◽  
Basic Functions

Visual-semantic embedding (VSE) networks create joint image–text representations to map images and texts in a shared embedding space to enable various information retrieval-related tasks, such as image–text retrieval, image captioning, and visual question answering. The most recent state-of-the-art VSE-based networks are: VSE++, SCAN, VSRN, and UNITER. This study evaluates the performance of those VSE networks for the task of image-to-text retrieval and identifies and analyses their strengths and limitations to guide future research on the topic. The experimental results on Flickr30K revealed that the pre-trained network, UNITER, achieved 61.5% on average Recall@5 for the task of retrieving all relevant descriptions. The traditional networks, VSRN, SCAN, and VSE++, achieved 50.3%, 47.1%, and 29.4% on average Recall@5, respectively, for the same task. An additional analysis was performed on image–text pairs from the top 25 worst-performing classes using a subset of the Flickr30K-based dataset to identify the limitations of the performance of the best-performing models, VSRN and UNITER. These limitations are discussed from the perspective of image scenes, image objects, image semantics, and basic functions of neural networks. This paper discusses the strengths and limitations of VSE networks to guide further research into the topic of using VSE networks for cross-modal information retrieval tasks.

scholarly journals A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions

2021 ◽  
Author(s):  
Yunshi Lan ◽  
Gaole He ◽  
Jinhao Jiang ◽  
Jing Jiang ◽  
Wayne Xin Zhao ◽  
...  
Keyword(s):  
Information Retrieval ◽  
Knowledge Base ◽  
Question Answering ◽  
Future Research ◽  
Semantic Parsing ◽  
Complex Knowledge

Knowledge base question answering (KBQA) aims to answer a question over a knowledge base (KB). Recently, a large number of studies focus on semantically or syntactically complicated questions. In this paper, we elaborately summarize the typical challenges and solutions for complex KBQA. We begin with introducing the background about the KBQA task. Next, we present the two mainstream categories of methods for complex KBQA, namely semantic parsing-based (SP-based) methods and information retrieval-based (IR-based) methods. We then review the advanced methods comprehensively from the perspective of the two categories. Specifically, we explicate their solutions to the typical challenges. Finally, we conclude and discuss some promising directions for future research.

Question Answering

2012 ◽  
pp. 304-343 ◽  
Author(s):  
Ivan Habernal ◽  
Miloslav Konopík ◽  
Ondrej Rohlík
Keyword(s):  
Information Retrieval ◽  
Semantic Web ◽  
Information Needs ◽  
World Wide ◽  
Question Answering ◽  
Future Research ◽  
Semantic Relationships ◽  
The World ◽  
Standard Information ◽  
Data Source

Question Answering is an area of information retrieval with the added challenge of applying sophisticated techniques to identify the complex syntactic and semantic relationships present in text in order to provide a more sophisticated and satisfactory response to the user’s information needs. For this reason, the authors see question answering as the next step beyond standard information retrieval. In this chapter state of the art question answering is covered focusing on providing an overview of systems, techniques and approaches that are likely to be employed in the next generations of search engines. Special attention is paid to question answering using the World Wide Web as the data source and to question answering exploiting the possibilities of Semantic Web. Considerations about the current issues and prospects for promising future research are also provided.

scholarly journals Report on the 11th bibliometric-enhanced information retrieval workshop (BIR 2021)

2021 ◽  
Vol 55 (1) ◽  
pp. 1-9
Author(s):  
Ingo Frommholz ◽  
Guillaume Cabanac ◽  
Philipp Mayr ◽  
Suzan Verberne
Keyword(s):  
Information Retrieval ◽  
Lessons Learned ◽  
Future Research ◽  
Research Questions ◽  
Important Branch

The 11th Bibliometric-enhanced Information Retrieval Workshop (BIR 2021) was held online on April 1st, 2021, at ECIR 2021 as a virtual event. The interdisciplinary BIR workshop series aims to bring together researchers from different communities, especially Scientometrics/Bibliometrics and Information Retrieval. We report on the 11th BIR, its invited talks and accepted papers. Lessons learned from BIR 2021 are discussed and potential future research questions identified that position Bibliometric-enhanced IR as an exciting special yet important branch of IR research.

A dummy-based user privacy protection approach for text information retrieval

2020 ◽  
Vol 195 ◽  
pp. 105679
Author(s):  
Zongda Wu ◽  
Shigen Shen ◽  
Xinze Lian ◽  
Xinning Su ◽  
Enhong Chen
Keyword(s):  
Information Retrieval ◽  
Privacy Protection ◽  
User Privacy ◽  
Text Information ◽  
Text Information Retrieval

Experiments with Language-based Aids in Information Retrieval Systems

1988 ◽  
Vol 11 (1-2) ◽  
pp. 33-46 ◽  
Author(s):  
Tove Fjeldvig ◽  
Anne Golden
Keyword(s):  
Information Retrieval ◽  
Text Retrieval ◽  
Considerable Improvement ◽  
Controlled Experiments ◽  
Compound Words ◽  
Search Results ◽  
Retrieval Systems ◽  
Complete Search ◽  
Information Retrieval Systems ◽  
Search Quality

The fact that a lexeme can appear in various forms causes problems in information retrieval. As a solution to this problem, we have developed methods for automatic root lemmatization, automatic truncation and automatic splitting of compound words. All the methods have as their basis a set of rules which contain information regarding inflected and derived forms of words – and not a dictionary. The methods have been tested on several collections of texts, and have produced very good results. By controlled experiments in text retrieval, we have studied the effects on search results. These results show that both the method of automatic root lemmatization and the method of automatic truncation make a considerable improvement on search quality. The experiments with splitting of compound words did not give quite the same improvement, however, but all the same this experiment showed that such a method could contribute to a richer and more complete search request.

Semantic Information Retrieval on Medical Texts

2022 ◽  
Vol 54 (7) ◽  
pp. 1-38
Author(s):  
Lynda Tamine ◽  
Lorraine Goeuriot
Keyword(s):  
Information Retrieval ◽  
Health Informatics ◽  
Medical Information ◽  
State Of The Art ◽  
Lessons Learned ◽  
Semantic Search ◽  
Future Research ◽  
Cross Model ◽  
Wide Range ◽  
Search Systems

The explosive growth and widespread accessibility of medical information on the Internet have led to a surge of research activity in a wide range of scientific communities including health informatics and information retrieval (IR). One of the common concerns of this research, across these disciplines, is how to design either clinical decision support systems or medical search engines capable of providing adequate support for both novices (e.g., patients and their next-of-kin) and experts (e.g., physicians, clinicians) tackling complex tasks (e.g., search for diagnosis, search for a treatment). However, despite the significant multi-disciplinary research advances, current medical search systems exhibit low levels of performance. This survey provides an overview of the state of the art in the disciplines of IR and health informatics, and bridging these disciplines shows how semantic search techniques can facilitate medical IR. First,we will give a broad picture of semantic search and medical IR and then highlight the major scientific challenges. Second, focusing on the semantic gap challenge, we will discuss representative state-of-the-art work related to feature-based as well as semantic-based representation and matching models that support medical search systems. In addition to seminal works, we will present recent works that rely on research advancements in deep learning. Third, we make a thorough cross-model analysis and provide some findings and lessons learned. Finally, we discuss some open issues and possible promising directions for future research trends.

scholarly journals Measuring Machine Intelligence Through Visual Question Answering

AI Magazine ◽  
2016 ◽  
Vol 37 (1) ◽  
pp. 63-72 ◽  
Author(s):  
C. Lawrence Zitnick ◽  
Aishwarya Agrawal ◽  
Stanislaw Antol ◽  
Margaret Mitchell ◽  
Dhruv Batra ◽  
...  
Keyword(s):  
Question Answering ◽  
Machine Intelligence ◽  
Image Captioning ◽  
Visual Question Answering ◽  
Language And Vision ◽  
Measuring Machine

As machines have become more intelligent, there has been a renewed interest in methods for measuring their intelligence. A common approach is to propose tasks for which a human excels, but one which machines find difficult. However, an ideal task should also be easy to evaluate and not be easily gameable. We begin with a case study exploring the recently popular task of image captioning and its limitations as a task for measuring machine intelligence. An alternative and more promising task is Visual Question Answering that tests a machine’s ability to reason about language and vision. We describe a dataset unprecedented in size created for the task that contains over 760,000 human generated questions about images. Using around 10 million human generated answers, machines may be easily evaluated.

Sign in / Sign up

Export Citation Format

Share Document