Perceptual Image Representations for Support Vector Machine Image Coding

Support vector machine (SVM) image coding relies on the ability of SVMs for function approximation. The size and the profile of the e-insensitivity zone of the support vector regression (SVR) at some specific image representation determines (a) the amount of selected support vectors (the compression ratio), and (b) the nature of the introduced error (the compression distortion). However, the selection of an appropriate image representation is a key issue for a meaningful design of the e-insensitivity profile. For example, in image coding applications, taking human perception into account is of paramount relevance to obtain a good rate-distortion performance. However, depending on the accuracy of the considered perception model, certain image representations are not suitable for SVR training. In this chapter, we analyze the general procedure to take human vision models into account in SVR-based image coding. Specifically, we derive the condition for image representation selection and the associated e-insensitivity profiles.

Discrete Time Signal Processing Framework with Support Vector Machines

Digital signal processing (DSP) of time series using SVM has been addressed in the literature with a straightforward application of the SVM kernel regression, but the assumption of independently distributed samples in regression models is not fulfilled by a time-series problem. Therefore, a new branch of SVM algorithms has to be developed for the advantageous application of SVM concepts when we process data with underlying time-series structure. In this chapter, we summarize our past, present, and future proposal for the SVM-DSP frame-work, which consists of several principles for creating linear and nonlinear SVM algorithms devoted to DSP problems. First, the statement of linear signal models in the primal problem (primal signal models) allows us to obtain robust estimators of the model coefficients in classical DSP problems. Next, nonlinear SVM-DSP algorithms can be addressed from two different approaches: (a) reproducing kernel Hilbert spaces (RKHS) signal models, which state the signal model equation in the feature space, and (b) dual signal models, which are based on the nonlinear regression of the time instants with appropriate Mercer’s kernels. This way, concepts like filtering, time interpolation, and convolution are considered and analyzed, and they open the field for future development on signal processing algorithms following this SVM-DSP framework.

Probabilistic Kernel PCA and its Application to Statistical Modeling and Inference

We present a method of density estimation that is based on an extension of kernel PCA to a probabilistic framework. Given a set of sample data, we assume that this data forms a Gaussian distribution, not in the input space but upon a nonlinear mapping to an appropriate feature space. As with most kernel methods, this mapping can be carried out implicitly. Due to the strong nonlinearity, the corresponding density estimate in the input space is highly non-Gaussian. Numerical applications on 2-D data sets indicate that it is capable of approximating essentially arbitrary distributions. Beyond demonstrating applications on 2-D data sets, we apply our method to high-dimensional data given by various silhouettes of a 3-D object. The shape density estimated by our method is subsequently applied as a statistical shape prior to variational image segmentation. Experiments demonstrate that the resulting segmentation process can incorporate highly accurate knowledge on a large variety of complex real-world shapes. It makes the segmentation process robust to misleading information due to noise, clutter and occlusion.

Building Sequence Kernels for Speaker Verification and Word Recognition

This chapter describes the adaptation and application of kernel methods for speech processing. It is divided into two sections dealing with speaker verification and isolated-word speech recognition applications. Significant advances in kernel methods have been realised in the field of speaker verification, particularly relating to the direct scoring of variable-length speech utterances by sequence kernel SVMs. The improvements are so substantial that most state-of-the-art speaker recognition systems now incorporate SVMs. We describe the architecture of some of these sequence kernels. Speech recognition presents additional challenges to kernel methods and their application in this area is not as straightforward as for speaker verification. We describe a sequence kernel that uses dynamic time warping to capture temporal information within the kernel directly. The formulation also extends the standard dynamic time-warping algorithm by enabling the dynamic alignment to be computed in a high-dimensional space induced by a kernel function. This kernel is shown to work well in an application for recognising low-intelligibility speech of severely dysarthric individuals.

A Complex Support Vector Machine Approach to OFDM Coherent Demodulation

Most of the approaches to digital communication applications using support vector machines (SVMs) rely on the conventional classification and regression SVM algorithms. However, the introduction of complex algebra in the SVM formulation can provide us with a more flexible and natural framework when dealing with complex constellations and symbols. In this chapter, an SVM algorithm for coherent robust demodulation in orthogonal frequency division multiplexing (OFDM) systems is studied. We present a complex regression SVM formulation specifically adapted to a pilot-based OFDM signal, which provides us with a simpler scheme than an SVM multiclassification method. The feasibility of this approach is substantiated by computer simulation results obtained for Institute of Electrical and Electronic Engineers (IEEE) 802.16 broadband fixed wireless channel models. These experiments allow us to scrutinize the performance of the OFDM-SVM system and the suitability of the e-Huber cost function in the presence of non-Gaussian impulse noise interfering with OFDM pilot symbols.

On the Pre-Image Problem in Kernel Methods

In this chapter, we are concerned with the problem of reconstructing patterns from their representation in feature space, known as the pre-image problem. We review existing algorithms and propose a learning-based approach. All algorithms are discussed regarding their usability and complexity, and evaluated on an image denoising application.

Hyperspectral Image Classification with Kernels

The information contained in hyperspectral images allows the characterization, identification, and classification of land covers with improved accuracy and robustness. However, several critical problems should be considered in the classification of hyperspectral images, among which are (a) the high number of spectral channels, (b) the spatial variability of the spectral signature, (c) the high cost of true sample labeling, and (d) the quality of data. Recently, kernel methods have offered excellent results in this context. This chapter reviews the state-of-the-art hyperspectral image classifiers, presents two recently proposed kernel-based approaches, and systematically discusses the specific needs and demands of this field.

Image Classification and Retrieval with Kernel Methods

In this chapter we review some kernel methods useful for image classification and retrieval applications. Starting from the problem of constructing appropriate image representations, we describe in depth and comment on the main properties of various kernel engineering approaches that have been recently proposed in the computer vision and machine learning literature for solving a number of image classification problems. We distinguish between kernel functions applied to images as a whole and kernel functions looking at image features. We conclude by presenting some current work and discussing open issues.

Applications of Kernel Theory to Speech Recognition

Automated speech recognition is traditionally defined as the process of converting an audio signal into a sequence of words. Over the past 30 years, simplistic techniques based on the design of smart feature-extraction algorithms and physiological models have given way to powerful statistical methods based on generative models. Such approaches suffer from three basic problems: discrimination, generalization, and sparsity. In the last decade, the field of machine learning has grown tremendously, generating many promising new approaches to this problem based on principles of discrimination. These techniques, though powerful when given vast amounts of training data, often suffer from poor generalization. In this chapter, we present a unified framework in which both generative and discriminative models are motivated from an information theoretic perspective. We introduce the modern statistical approach to speech recognition and discuss how kernel-based methods are used to model knowledge at each level of the problem. Specific methods discussed include kernel PCA for feature extraction and support vector machines for discriminative modeling. We conclude with some emerging research on the use of kernels in language modeling.

Comparison of Kernel Methods Applied to Smart Antenna Array Processing

Support vector machines (SVMs) are a good candidate for the solution of antenna array processing problems such as beamforming, detection of the angle of arrival, or sidelobe suppression, due to the fact that these algorithms exhibit superior performance in generalization ability and reduction of computational burden. Here, we introduce three new approaches for antenna array beamforming based on SVMs. The first one relies on the use of a linear support vector regressor to construct a linear beamformer. This algorithm outperforms the minimum variance distortionless method (MVDM) when the sample set used for training is small. It is also an advantageous approach when there is non-Gaussian noise present in the data. The second algorithm uses a nonlinear multiregressor to find the parameters of a linear beamformer. A multiregressor is trained off line to find the optimal parameters using a set of array snapshots. During the beamforming operation, the regressor works in the test mode, thus finding a set of parameters by interpolating among the solutions provided in the training phase. The motivation of this second algorithm is that the number of floating point operations needed is smaller than the number of operations needed by the MVDM since there is no need for matrix inversions. Only a vector-matrix product is needed to find the solution. Also, knowledge of the direction of arrival of the desired signal is not required during the beamforming operation, which leads to simpler and more flexible beamforming realizations. The third one is an implementation of a nonlinear beamformer using a non-linear SVM regressor. The operation of such a regressor is a generalization of the linear SVM one, and it yields better performance in terms of bit error rate (BER) than its linear counterparts. Simulations and comparisons with conventional beamforming strategies are provided, demonstrating the advantages of the SVM approach over the least-squares-based approach.