A Machine Learning-Based Model for Content-Based Image Retrieval

A multimedia index makes it possible to group data according to similarity criteria. Traditional index structures are based on trees and use the k-Nearest Neighbors (k-NN) approach to retrieve databases. Due to some disadvantages of a such approach, the use of neighborhood graphs was proposed. This approach is interesting but it has some disadvantages consisting, mainly, in its complexity. This chapter presents a step in a long process of analyzing, structuring, and retrieving multimedia databases. Indeed, we propose an effective method for locally updating neighborhood graphs which constitute our multimedia index. Then, we exploit this structure in order to make easy and effective the retrieval process using queries in an image form in one hand. In another hand, we use the indexing structure to annotate images in order to describe their semantics. The proposed approach is based on an intelligent manner for locating points in a multidimensional space. Promising results are obtained after experimentations on various databases. Future issues of the proposed approach are very relevant in this domain.

Semantic-Based Video Scene Retrieval Using Evolutionary Computing

A new emotion-based video scene retrieval method is proposed in this chapter. Five video features extracted from a video are represented in a genetic chromosome and target videos that user has in mind are retrieved by the interactive genetic algorithm through the feedback iteration. After the proposed algorithm selects the videos that contain the corresponding emotion from the initial population of videos, the feature vectors from them are regarded as chromosomes, and a genetic crossover is applied to those feature vectors. Next, new chromosomes after crossover and feature vectors in the database videos are compared based on a similarity function to obtain the most similar videos as solutions of the next generation. By iterating this process, a new population of videos that a user has in mind are retrieved. In order to show the validity of the proposed method, six example categories of “action,” “excitement,” “suspense,” “quietness,” “relaxation,” and “happiness” are used as emotions for experiments. This method of retrieval shows 70% of effectiveness on the average over 300 commercial videos.

Neural Networks for Content-Based Image Retrieval

This chapter introduces neural networks for Content-Based Image Retrieval (CBIR) systems. It presents a critical literature review of both the traditional and neural network based techniques that are used in retrieving the images based on their content. It shows how neural networks and fuzzy logic can be used in interpretation of queries, feature extraction and classification of features by describing a detailed research methodology. It investigates a neural network based technique in conjunction with fuzzy logic to improve the overall performance of the CBIR systems. The results of the investigation on a benchmark database with a comparative analysis are presented in this chapter. The methodologies and results presented in this chapter will allow researchers to improve and compare their methods and it will also allow system developers to understand and implement the neural network and fuzzy logic based techniques for content based image retrieval.

Automatic and Semi-Automatic Techniques for Image Annotation

When retrieving images, users may find it easier to express the desired semantic content with keywords than visual features. Accurate keyword retrieval can only occur when images are completely and accurately described. This can be achieved either through laborious manual effort or automated approaches. Current methods for automatically extracting semantic information from images can be classified into (a) text-based methods, which use metadata such as ontological descriptions and/or text associated with images, to assign and/or refine annotations, and (b) image-based methods, which focus on extracting semantic information directly from image content. The focus of this chapter is to create an awareness and understanding of research and advances in this field, by introducing them to basic concepts and theories and then by classifying, summarizing, and describing works from the published literature. It will also identify unsolved problems and offer suggestions for future research directions.

Semantic Multimedia Information Anaylsis for Retrieval Applications

Most of the research in multimedia retrieval applications has focused on retrieval by content or retrieval by example. Since the classical review by Smeulders (2000) a new interest has grown immensely in the multimedia information retrieval community: retrieval by semantics. This exciting new research area arises as a combination of multimedia understanding, information extraction, information retrieval and digital libraries. This chapter presents a comprehensive review of analysis algorithms to extract semantic information from multimedia content. We discuss statistical approaches to analyse images and video content and conclude with a discussion regarding the described methods.

Managing Uncertainties in Image Databases

In this chapter, we focus on those functionalities of multimedia databases that are not present in traditional databases, but are needed when dealing with multimedia information. Multimedia data are inherently subjective: for example, the association of a meaning and the corresponding content description to an image as well as the evaluation of the difference between two images or two pieces of music usually depend on the user who is involved in the evaluation process. For retrieval, such subjective information needs to be combined with objective information, such as image color histograms or sound frequencies, obtained through (generally imprecise) data analysis processes. Therefore, the inherently fuzzy nature of multimedia data, both at subjective and at objective levels, may lead to multiple, possibly inconsistent, interpretations of data. Here, we present the FNF2 data model, a Non First Normal Form extension of the relational model, which takes into account subjectivity and fuzziness, while being intuitive and enabling user friendly information access and manipulation mechanisms.

An Ontology-Based Frameowrk for Semantic Image Analysis and Retrieval

To overcome the limitations of keyword- and content-based visual information access, an ontology-driven framework is developed. Under the proposed framework, an appropriately defined ontology infrastructure is used to drive the generation of manual and automatic image annotations and to enable semantic retrieval by exploiting the formal semantics of ontologies. In this way, the descriptions considered in the tedious task of manual annotation are constrained to named entities (e.g. location names, person names, etc.), since the ontology-driven analysis module automatically generates annotations concerning common domain objects of interest (e.g. sunset, trees, sea, etc.). Experiments in the domain of outdoor images show that such an ontology-based scheme realizes efficient visual information access with respect to its semantics.

Shape-Based Image Retrieval By Alignment

Among the existing Content Based Image Retrieval (CBIR) techniques for still images based on different perceptual features (e. g., colour, texture, etc.), shape-based methods are particularly challenging due to the intrinsic difficulties in dealing with shape localization and recognition problems. Nevertheless, there is no doubt that shape is one of the most important perceptual feature and successful shape-based techniques would significantly improve the spreading of general-purpose image retrieval systems. In this chapter we present a shape-based image retrieval approach able to efficiently deal with domain independent images with possible cluttered backgrounds and partially occluded objects. It is based on an alignment approach proved to be robust in rigid object recognition which we have modified in order to deal with inexact matching between the stylized shape input by the user as query and the real shapes represented in the system’s database. Results with a database composed of difficult real-life images are shown.

The Imapct of Low-Level Features in Semantic-Based Image

Image Retrieval (IR) is generally known as a collection of techniques for retrieving images on the basis of features, either low-level (Content-based IR) or high-level (Semantic-based IR). Since Semantic-based features rely on low-level ones, in this chapter the reader is initially familiarized with the most widely used low-level features. An efficient way to present these features is by means of a statistical tool capable of bearing concrete information, such as the histogram. For use in IR, the histograms extracted from the previously mentioned features need to be compared by means of a metric. The most popular methods and distances are, thus, apposed. Finally, a number of IR systems using histograms are presented in a thorough manner and their experimental results are discussed. The steps in order to develop a custom IR system, along with modern techniques in image feature extraction are also presented.

A Novel Framework for Image Categorization and Automatic Annotation

Image classification and automatic annotation could be treated as effective solutions to enable keyword-based semantic image retrieval. Traditionally, they are investigated in different models separately. In this chapter, we propose a novel framework uniting image classification and automatic annotation by learning semantic concepts of image categories. To choose representative features, feature selection strategy is proposed and visual keywords are constructed, including discrete method and continuous method. Based on the selected features, the Integrated Patch (IP) model is proposed to describe the image category. As a generative model, the IP model describes the appearance of the combination of the visual keywords, considering the diversity of the object. The parameters are estimated by EM algorithm. The experimental results on Corel image dataset and Getty Image Archive demonstrate that the proposed feature selection and image description model are effective in image categorization and automatic image annotation, respectively.