State-of-the Art Motion Estimation in the Context of 3D TV

Progress in image sensors and computation power has fueled studies to improve acquisition, processing, and analysis of 3D streams along with 3D scenes/objects reconstruction. The role of motion compensation/motion estimation (MCME) in 3D TV from end-to-end user is investigated in this chapter. Motion vectors (MVs) are closely related to the concept of disparities, and they can help improving dynamic scene acquisition, content creation, 2D to 3D conversion, compression coding, decompression/decoding, scene rendering, error concealment, virtual/augmented reality handling, intelligent content retrieval, and displaying. Although there are different 3D shape extraction methods, this chapter focuses mostly on shape-from-motion (SfM) techniques due to their relevance to 3D TV. SfM extraction can restore 3D shape information from a single camera data.

Peer-to-Peer Video Streaming

The Internet as a video distribution medium has seen a tremendous growth in recent years. Currently, the transmission of major live events and TV channels over the Internet can easily reach hundreds or millions of users trying to receive the same content using very distinct receiver terminals, placing both scalability and heterogeneity challenges to content and network providers. In private and well-managed Internet Protocol (IP) networks these types of distributions are supported by specially designed architectures, complemented with IP Multicast protocols and Quality of Service (QoS) solutions. However, the Best-Effort and Unicast nature of the Internet requires the introduction of a new set of protocols and related architectures to support the distribution of these contents. In the field of file and non-real time content distributions this has led to the creation and development of several Peer-to-Peer protocols that have experienced great success in recent years. This chapter presents the current research and developments in Peer-to-Peer video streaming over the Internet. A special focus is made on peer protocols, associated architectures and video coding techniques. The authors also review and describe current Peer-to-Peer streaming solutions.

Medical Quality of Service (m-QoS) and Quality of Experience (m-QoE) for 4G-Health Systems

It is well known that the evolution of 4G-based mobile multimedia network systems will contribute significantly to future m-health applications that require high bandwidth, high data rates, and more critically better Quality of service and quality of experience. The key to the successful implementation of these emerging applications is the compatibility of emerging broadband wireless networks such as mobile WiMAX, HSUPA, and LTE networks with future m-health systems. Most recently, the concept of 4G-health is introduced. This is defined as the evolution of m-health towards targeted personalized medical systems with adaptable functionalities and compatibility with future 4G communications and network technologies. This new concept represents the evolution of m-health toward 4G mobility. It will have new challenges especially from the next generation of mobile communications and networks perspective and in particular from relevant quality of service and quality of experience issues. This chapter presents some of these challenges and illustrates the importance of the new concepts of medical Quality of Service (m-QoS) and medical Quality of Experience (m-QoE) for 4G-health systems. The chapter also presents a validation scenario of these concepts for medical video streaming application as a typical 4G-health scenario.

Free-Viewpoint 3DTV

Free-ViewPoint (FVP) interpolation allows creating a new view between the existing reference views. Applied to 3D multi-view video sequences, it leads to two important applications: (1) FVP service provided to the user, which enables the possibility to interactively select the viewing point of the scene; (2) improved compression of multi-view video sequences by using view prediction for inter-view coding. In this chapter, the authors provide an overview of the essential steps for 3D free-view video communication, which consists of the free-viewpoint interpolation techniques, a concept for free-view coding and a scalable free-view video streaming architecture. For facilitating free-view to the user, the chapter introduces the free-viewpoint interpolation techniques and the concept of warping. The authors assume that 3D video is represented by texture and depth images available for each view. Therefore it is possible to apply Depth Image Based Rendering (DIBR), which uses the depth signal as a important cue for geometry information and 3D reconstruction. Authors analyze the involved interpolation problems, such as cracks, ghost contours and disocclusions, which arise from an FVP interpolation and propose several solutions to improve the image quality of the synthesized view. Afterwards, they present a standard approach to FVP rendering used currently by the research community and our FVP interpolation. Additionally, authors show the use of FVP rendering for the multi-view coding and streaming and discuss the gains and trade-offs of it. At the end of the chapter are the state-of-the-art achievements and challenges of FVP rendering and a vision concerning the development of free-viewpoint services.

Advances in Region-of-Interest Video and Image Processing

The advent of cheaper and more powerful devices with the ability to play, create, and transmit video content has led to a dramatic increase in the multimedia content distribution on both wireline and wireless networks. Also, the reduction of cost of digital video cameras along with the development of user-generated video sites (e.g., iTunes™, YouTube™) stimulated a new user-generated video content sector and made unprecedented demands for high-quality and low-delay video communication. The Region-of-Interest (ROI) is a desirable feature in many future scalable video coding applications, such as mobile device applications, which have to be adapted to be displayed on a relatively small screen; thus, a mobile device user may wish to extract and track only a predefined ROI within the displayed video. At the same time, other users having a larger mobile device screen may wish to extract other ROIs to receive higher video stream resolution. Therefore, to fulfill these requirements, it would be beneficial to simultaneously transmit or store a video stream in a variety of ROIs, as well to enable efficiently tracking of the predefined Region-of-Interest. This chapter presents recent advances in Region-of-Interest video and image processing techniques for multimedia applications, while making a special emphasis on a scalable extension of the H.264/AVC standard. The detailed observations and conclusions, which are presented in this chapter, are supported by authors’ personal experience in this field, thereby presenting a variety of experimental results.

Compression Artifacts in Modern Video Coding and State-of-the-Art Means of Compensation

This chapter describes and explains common as well as less common distortions in modern video coding, ranging from artifacts appearing in MPEG-2 Video, MPEG-4 Part 2, H.264, and VC-1 to scalable and multi-view video coding based distortions, including the proposals for next generation video coding (NVC). In addition to a discussion about avoiding these artifacts through encoder-side measures, a state-of-the-art overview of their compensation at the decoder side is given. Finally, artifacts emerging from new sophisticated coding tools in current and upcoming video coding standards are discussed.

Improved Subject Identification in Surveillance Video Using Super-Resolution

The time consuming and labour intensive task of identifying individuals in surveillance video is often challenged by poor resolution and the sheer volume of stored video. Faces or identifying marks such as tattoos are often too coarse for direct matching by machine or human vision. Object tracking and super-resolution can then be combined to facilitate the automated detection and enhancement of areas of interest. The object tracking process enables the automatic detection of people of interest, greatly reducing the amount of data for super-resolution. Smaller regions such as faces can also be tracked. A number of instances of such regions can then be utilized to obtain a super-resolved version for matching. Performance improvement from super-resolution is demonstrated using a face verification task. It is shown that there is a consistent improvement of approximately 7% in verification accuracy, using both Eigenface and Elastic Bunch Graph Matching approaches for automatic face verification, starting from faces with an eye to eye distance of 14 pixels. Visual improvement in image fidelity from super-resolved images over low-resolution and interpolated images is demonstrated on a small database. Current research and future directions in this area are also summarized.

Resilient Video Coding via Improved Motion Compensated Prediction

Motion compensated prediction (MCP) is at the heart of modern video compression standard because of its ability to remove temporal redundancies. However, MCP is responsible for temporal error propagation, which can result in severe quality degradation in lossy environments. In this chapter, the authors present two innovative methods of improving MCP to be more resilient to packet losses. In the first method, the motion trajectory is used to develop a novel distortion weighting technique, and the second method exploits the presence of Intra macroblocks in previously coded frames to develop increase robustness.

Source Coding Methods for Robust Wireless Video Streaming

As real-time video streaming moves to the mobile Internet, there is a greater need to protect fragile compressed bit-streams from the impact of lossy wireless channels. Though forward error correction (FEC) has a role, if it is applied without adaptation, it may introduce excessive communication overhead. Alternatively, error resilience methods provide additional protection at the application layer of the protocol stack, without replication of any protection already provided at the data-like layer. In this chapter, a case study shows that these resilience methods can be applied adaptively through stream switching according to channel conditions. Error resilience can work hand-in-hand with error concealment, again applied through source coding. There are many error resilience and concealment methods, which this chapter surveys at a tutorial level. The chapter also includes an overview of video streaming for those unfamiliar with the topic. Though error concealment is a non-normative feature of the H.264/AVC (Advanced Video Coding) codec standard, there is a range of new techniques that have been included within the Standard such as flexible macroblock ordering and stream switching frames. The chapter additionally reviews error concealment provision, including spatial, temporal, and hybrid methods. Results show that there are tradeoffs between the level of protection and the level of overhead, according to the severity of the wireless channel impairment.

Technical Challenges of 3D Video Coding

Three-dimensional video (3DV) is expected to be the next multimedia technology that provides depth impression of observed scenery with multi-view videos. In fact, studies on 3D video have a long history, heading back two hundred years; but recently, it has risen as the hottest issue due to rapid progresses of IT technologies. Particularly, 3D video systems are the most promising technology in multimedia area. An extension of typical stereoscopic imaging, realistic and natural 3D video technologies are currently under development. In this chapter, the authors describe overall technologies of 3D video systems from capturing to display, including coding standards. Mainly, the chapter focuses on the recent standardization activities by MPEG (moving picture experts group) associated with 3D video coding.