Similarity Learning for Motion Estimation

Motion estimation necessitates an appropriate choice of similarity function. Because generic similarity functions derived from simple assumptions are insufficient to model complex yet structured appearance variations in motion estimation, the authors propose to learn a discriminative similarity function to match images under varying appearances by casting image matching into a binary classification problem. They use the LogitBoost algorithm to learn the classifier based on an annotated database that exemplifies the structured appearance variations: An image pair in correspondence is positive and an image pair out of correspondence is negative. To leverage the additional distance structure of negatives, they present a location-sensitive cascade training procedure that bootstraps negatives for later stages of the cascade from the regions closer to the positives, which enables viewing a large number of negatives and steering the training process to yield lower training and test errors. The authors apply the learned similarity function to estimating the motion for the endocardial wall of left ventricle in echocardiography and to performing visual tracking. They obtain improved performances when comparing the learned similarity function with conventional ones.

Face Recognition and Semantic Features

The authors present a face recognition scheme based on semantic features’ extraction from faces and tensor subspace analysis. These semantic features consist of eyes and mouth, plus the region outlined by three weight centres of the edges of these features. The extracted features are compared over images in tensor subspace domain. Singular value decomposition is used to solve the eigenvalue problem and to project the geometrical properties to the face manifold. They compare the performance of the proposed scheme with that of other established techniques, where the results demonstrate the superiority of the proposed method.

Multimedia Data Indexing

The rapid advances in multimedia capture, storage and communication technologies and capabilities have ushered an era of unprecedented growth of digital media content, in audio, visual, and synthetic forms, and both individually and commercially produced. How to manage these data to make them more accessible and searchable to users is a key challenge in current multimedia computing research. In this chapter, the authors discuss the problems and challenges in multimedia data management, and review the state of the art in data structures and algorithms for multimedia indexing, media feature space management and organization, and applications of these techniques in multimedia data management.

Applications of Semantic Mining on Biological Process Engineering

Semantic mining is an essential part in knowledgebase and decision support systems where it enables the extraction of useful knowledge form available databases with the ultimate goal of supporting the decision making process. In process systems engineering, decisions are made throughout plant / process / product life cycles. The provision of smart semantic mining techniques will improve the decision making process for all life cycle activities. In particular, safety and environmental related decisions are highly dependent on process internal and external conditions and dynamics with respect to equipment geometry and plant layout. This chapter discusses practical methods for semantic mining using systematic knowledge representation as integrated with process modeling and domain knowledge. POOM or plant/process object oriented modeling methodology is explained and used as a basis to implement semantic mining as applied on process systems engineering. Case studies are illustrated for biological process engineering, in particular MoFlo systems focusing on process safety and operation design support.

Image/Video Semantic Analysis by Semi-Supervised Learning

The insufficiency of labeled training samples is a major obstacle in automatic semantic analysis of large scale image/video database. Semi-supervised learning, which attempts to learn from both labeled and unlabeled data, is a promising approach to tackle this problem. As a major family of semi-supervised learning, graph-based methods have attracted more and more recent research. In this chapter, a brief introduction is given on popular semi-supervised learning methods, especially the graph-based methods, as well as their applications in the area of image annotation, video annotation, and image retrieval. It is well known that the pair-wise similarity is an essential factor in graph propagation based semisupervised learning methods. A novel graph-based semi-supervised learning method, named Structure- Sensitive Anisotropic Manifold Ranking (SSAniMR), is derived from a PDE based anisotropic diffusion framework. Instead of using Euclidean distance only, SSAniMR further takes local structural difference into account to more accurately measure pair-wise similarity. Finally some future directions of using semi-supervised learning to analyze the multimedia content are discussed.

Association-Based Image Retrieval

With advances in computer technology and the World Wide Web there has been an explosion in the amount and complexity of multimedia data that are generated, stored, transmitted, analyzed, and accessed. In order to extract useful information from this huge amount of data, many content-based image retrieval (CBIR) systems have been developed in the last decade. A Typical CBIR system captures image features that represent image properties such as color, texture, or shape of objects in the query image and try to retrieve images from the database with similar features. Recent advances in CBIR systems include relevance feedback based interactive systems. The main advantage of CBIR systems with relevance feedback is that these systems take into account the gap between the high-level concepts and low-level features and subjectivity of human perception of visual content. CBIR systems with relevance feedback are more efficient than conventional CBIR systems; however, these systems depend on human interaction. In this chapter, we describe a new approach for image storage and retrieval called association-based image retrieval (ABIR). The authors try to mimic human memory. The human brain stores and retrieves images by association. They use a generalized bi-directional associative memory (GBAM) to store associations between feature vectors that represent images stored in the database. Section I introduces the reader to the CBIR system. In Section II, they present architecture for the ABIR system, Section III deals with preprocessing and feature extraction techniques, and Section IV presents various models of GBAM. In Section V, they present case studies.

Compressed-Domain Image Retrieval Based on Colour Visual Patterns

Image retrieval and image compression have been typically pursued separately. Only little research has been done on a synthesis of the two by allowing image retrieval to be performed directly in the compressed domain of images without the need to uncompress them first. In this chapter the authors show that such compressed domain image retrieval can indeed be done and lead to effective and efficient retrieval performance. They introduce a novel compression algorithm – colour visual pattern image coding (CVPIC) – and present several retrieval algorithms that operate directly on compressed CVPIC data. Their experiments demonstrate that it is not only possible to realise such midstream content access, but also that the presented techniques outperform standard retrieval techniques such as colour histograms and colour correlograms.

Content-Based Video Semantic Analysis

With the explosive growth in the amount of video data and rapid advance in computing power, extensive research efforts have been devoted to content-based video analysis. In this chapter, the authors will give a broad discussion on this research area by covering different topics such as video structure analysis, object detection and tracking, event detection, visual attention analysis, and so forth. In the meantime, different video representation and indexing models are also presented.

Semantic Classification and Annotation of Images

With the rapid growth of image collections, content-based image retrieval (CBIR) has been an active area of research with notable recent progress. However, automatic image retrieval by semantics still remains a challenging problem. In this chapter, the authors will describe two promising techniques towards semantic image retrieval—semantic image classification and automatic image annotation. For each technique, four aspects are presented: task definition, image representation, computational models, and evaluation. Finally, they will give a brief discussion of their application in image retrieval.

Annotating Images by Mining Image Search

Although it has been studied for years by computer vision and machine learning communities, image annotation is still far from practical. In this chapter, the authors propose a novel attempt of modeless image annotation, which investigates how effective a data-driven approach can be, and suggest annotating an uncaptioned image by mining its search results. The authors collected 2.4 million images with their surrounding texts from a few photo forum Web sites as our database to support this data-driven approach. The entire process contains three steps: (1) the search process to discover visually and semantically similar search results; (2) the mining process to discover salient terms from textual descriptions of the search results; and (3) the annotation rejection process to filter noisy terms yielded by step 2. To ensure real time annotation, two key techniques are leveraged – one is to map the high dimensional image visual features into hash codes, the other is to implement it as a distributed system, of which the search and mining processes are provided as Web services. As a typical result, the entire process finishes in less than 1 second. Since no training dataset is required, our proposed approach enables annotating with unlimited vocabulary, and is highly scalable and robust to outliers. Experimental results on real Web images show the effectiveness and efficiency of the proposed algorithm.