Research on the Application of Environmental Art Design Based on the Combination of VR and Panoramic Video Technology

The panoramic video technology is introduced to collect multiangle data of design objects, draw a 3D spatial model with the collected data, solve the first-order differential equation for the 3D spatial model, obtain the spatial positioning extremes of the object scales, and realize the alignment and fusion of panoramic video images according to the positioning extremes above and below the scale space. Then, the panoramic video is generated and displayed by computer processing so that the tourist can watch the scene with virtual information added to the panoramic video by wearing the display device elsewhere. It solves the technical difficulties of the high complexity of the algorithm in the system of panoramic video stitching and the existence of stitching cracks and the “GHOST” phenomenon in the stitched video, as well as the technical difficulties that the 3D registration is easily affected by the time-consuming environment and target tracking detection algorithm. The simulation results show that the panoramic video stitching method performs well in real time and effectively suppresses stitching cracks and the “GHOST” phenomenon, and the augmented reality 3D registration method performs well for the local enhancement of the panoramic video.

Region-Based Static Video Stitching for Reduction of Parallax Distortion

In this paper, we propose a semantic segmentation-based static video stitching method to reduce parallax and misalignment distortion for sports stadium scenes with dynamic foreground objects. First, video frame pairs for stitching are divided into segments of different classes through semantic segmentation. Region-based stitching is performed on matched segment pairs, assuming that segments of the same semantic class are on the same plane. Second, to prevent degradation of the stitching quality of plain or noisy videos, the homography for each matched segment pair is estimated using the temporally consistent feature points. Finally, the stitched video frame is synthesized by stacking the stitched matched segment pairs and the foreground segments to the reference frame plane by descending order of the area. The performance of the proposed method is evaluated by comparing the subjective quality, geometric distortion, and pixel distortion of video sequences stitched using the proposed and conventional methods. The proposed method is shown to reduce parallax and misalignment distortion in segments with plain texture or large parallax, and significantly improve geometric distortion and pixel distortion compared to conventional methods.

REAL TIME VIDEO STITCHING IMPLEMENTATION ON A ZYNQ FPGA SOC

This project details the design and implementation of an image processing pipeline that targets real time video-stitching for semi-panoramic video synthesis. The scope of the project includes the analysis of possible approaches, selection of processing algorithms and procedures, design of experimental hardware set-up (including the schematic capture design of a custom catadioptric panoramic imaging system) and firmware/software development of the vision processing system components. The goal of the project is to develop a frame-stitching IP module as well as an efficient video registration algorithm capable for synthesis of a semi-panoramic video-stream at 30 frames-per-second (fps) rate with minimal FPGA resource utilization. The developed components have been validated in hardware. Finally, a number of hybrid architectures that make use of the synergy between the CPU and FPGA section of the ZYNQ SoC have been investigated and prototyped as alternatives to a complete hardware solution. Keyword: Video stitching, Panoramic vision, FPGA, SoC, vision system, registration

REAL TIME VIDEO STITCHING IMPLEMENTATION ON A ZYNQ FPGA SOC

This project details the design and implementation of an image processing pipeline that targets real time video-stitching for semi-panoramic video synthesis. The scope of the project includes the analysis of possible approaches, selection of processing algorithms and procedures, design of experimental hardware set-up (including the schematic capture design of a custom catadioptric panoramic imaging system) and firmware/software development of the vision processing system components. The goal of the project is to develop a frame-stitching IP module as well as an efficient video registration algorithm capable for synthesis of a semi-panoramic video-stream at 30 frames-per-second (fps) rate with minimal FPGA resource utilization. The developed components have been validated in hardware. Finally, a number of hybrid architectures that make use of the synergy between the CPU and FPGA section of the ZYNQ SoC have been investigated and prototyped as alternatives to a complete hardware solution. Keyword: Video stitching, Panoramic vision, FPGA, SoC, vision system, registration

Video Stitching method for a Surveillance System Deployed at a Fully Mechanized Mining Face

Using video stitching technology, video images with overlapping parts can be stitched into a complete image, with characteristics such as intuitiveness, visualization, and measurable analysis. This technology could be applied in the operation of coal mines for a remote monitoring and control of coal production. However, when the technology is used in coal mines, there are several challenges such as non-uniform illumination, missing scenes, and oblique panorama. In this paper, methods were purposed to solve the above problems: (1) To overcome the non-uniform illumination on a mining face, we applied the wide dynamic range technology to the images from a single camera and histogram matching algorithm on multiple images to reduce the color difference between the images; (2) To overcome the missing scene problem due to the narrow field of view (FOV) of a single camera, the SURF matching and template recognition methods are combined to achieve a stable stitching; (3) To overcome the oblique panorama issue, we applied the vertical correction technology exploiting the posture information of the camera, and then the adjacent images are concatenated. The results of practical experiments show that the proposed methods are suitable for solving the above problems in a fully mechanized mining face. The research provides a new approach for displaying extended scenes of stope faces in the intelligent collieries.

Multicamera Video Stitching for Multiple Human Tracking

In this paper, a novel approach is proposed to track humans for video surveillance using multiple cameras and video stitching techniques. SIFT key points are extracted from all camera inputs. Using k-d tree algorithm, all the key points are matched and random sample consensus (RANSAC) is used to identify the match correspondence among all the matched points. Homography matrix is calculated using four matched robust feature correspondences, the images are warped with respect to the other images, and the human tracking is performed on the stitched image. To identify the human in the stitched video, background modeling is performed using fuzzy inference system and perform foreground extraction. After foreground extraction, the blobs are constructed around each detected human and centroid point is calculated for each blob. Finally, tracking of multiple humans is done by Kalman filter (KF) with Hungarian algorithm.