Multi-Scale Memetic Image Registration

Many technological applications of our time rely on images captured by multiple cameras. Such applications include the detection and recognition of objects in captured images, the tracking of objects and analysis of their motion, and the detection of changes in appearance. The alignment of images captured at different times and/or from different angles is a key processing step in these applications. One of the most challenging tasks is to develop fast algorithms to accurately align images perturbed by various types of transformations. The paper reports a new method used to register images in the case of geometric perturbations that include rotations, translations, and non-uniform scaling. The input images can be monochrome or colored, and they are preprocessed by a noise-insensitive edge detector to obtain binarized versions. Isotropic scaling transformations are used to compute multi-scale representations of the binarized inputs. The algorithm is of memetic type and exploits the fact that the computation carried out in reduced representations usually produces promising initial solutions very fast. The proposed method combines bio-inspired and evolutionary computation techniques with clustered search and implements a procedure specially tailored to address the premature convergence issue in various scaled representations. A long series of tests on perturbed images were performed, evidencing the efficiency of our memetic multi-scale approach. In addition, a comparative analysis has proved that the proposed algorithm outperforms some well-known registration procedures both in terms of accuracy and runtime.

A deep neural network model for multi-view human activity recognition

Multiple cameras are used to resolve occlusion problem that often occur in single-view human activity recognition. Based on the success of learning representation with deep neural networks (DNNs), recent works have proposed DNNs models to estimate human activity from multi-view inputs. However, currently available datasets are inadequate in training DNNs model to obtain high accuracy rate. Against such an issue, this study presents a DNNs model, trained by employing transfer learning and shared-weight techniques, to classify human activity from multiple cameras. The model comprised pre-trained convolutional neural networks (CNNs), attention layers, long short-term memory networks with residual learning (LSTMRes), and Softmax layers. The experimental results suggested that the proposed model could achieve a promising performance on challenging MVHAR datasets: IXMAS (97.27%) and i3DPost (96.87%). A competitive recognition rate was also observed in online classification.

Real-Time Web Map Construction Based on Multiple Cameras and GIS

Previous VideoGIS integration methods mostly used geographic homography mapping. However, the related processing techniques were mainly for independent cameras and the software architecture was C/S, resulting in large deviations in geographic video mapping for small scenes, a lack of multi-camera video fusion, and difficulty in accessing real-time information with WebGIS. Therefore, we propose real-time web map construction based on the object height and camera posture (RTWM-HP for short). We first consider the constraint of having a similar height for each object by constructing an auxiliary plane and establishing a high-precision homography matrix (HP-HM) between the plane and the map; thus, the accuracy of geographic video mapping can be improved. Then, we map the objects in the multi-camera video with overlapping areas to geographic space and perform the object selection with the multi-camera (OS-CDD) algorithm, which includes the confidence of the object, the distance, and the angle between the objects and the center of the cameras. Further, we use the WebSocket technology to design a hybrid C/S and B/S software framework that is suitable for WebGIS integration. Experiments were carried out based on multi-camera videos and high-precision geospatial data in an office and a parking lot. The case study’s results show the following: (1) The HP-HM method can achieve the high-precision geographic mapping of objects (such as human heads and cars) with multiple cameras; (2) the OS-CDD algorithm can optimize and adjust the positions of the objects in the overlapping area and achieve a better map visualization effect; (3) RTWM-HP can publish real-time maps of objects with multiple cameras, which can be browsed in real time through point layers and hot-spot layers through WebGIS. The methods can be applied to some fields, such as person or car supervision and the flow analysis of customers or traffic passengers.

Person Re-Identification Microservice over Artificial Intelligence Internet of Things Edge Computing Gateway

With the increase in the number of surveillance cameras being deployed globally, an important topic is person re-identification (Re-ID), which identifies the same person from multiple different angles and different directions across multiple cameras. However, because of the privacy issues involved in the identification of individuals, Re-ID systems cannot send the image data to cloud, and these data must be processed on edge servers. However, there has been a significant increase in computing resources owing to the processing of artificial intelligence (AI) algorithms through edge computing (EC). Consequently, the traditional AI internet of things (AIoT) architecture is no longer sufficient. In this study, we designed a Re-ID system at the AIoT EC gateway, which utilizes a microservice to perform Re-ID calculations on EC and balances efficiency with privacy protection. Experimental results indicate that this architecture can provide sufficient Re-ID computing resources to allow the system to scale up or down flexibly to support different scenarios and demand loads.

MonoMR: Synthesizing Pseudo-2.5D Mixed Reality Content from Monocular Videos

MonoMR is a system that synthesizes pseudo-2.5D content from monocular videos for mixed reality (MR) head-mounted displays (HMDs). Unlike conventional systems that require multiple cameras, the MonoMR system can be used by casual end-users to generate MR content from a single camera only. In order to synthesize the content, the system detects people in the video sequence via a deep neural network, and then the detected person’s pseudo-3D position is estimated by our proposed novel algorithm through a homography matrix. Finally, the person’s texture is extracted using a background subtraction algorithm and is placed on an estimated 3D position. The synthesized content can be played in MR HMD, and users can freely change their viewpoint and the content’s position. In order to evaluate the efficiency and interactive potential of MonoMR, we conducted performance evaluations and a user study with 12 participants. Moreover, we demonstrated the feasibility and usability of the MonoMR system to generate pseudo-2.5D content using three example application scenarios.

Understanding the Lateral Dimension of Traffic: Measuring and Modeling Lane Discipline

There is growing interest in understanding the lateral dimension of traffic. This trend has been motivated by the detection of phenomena unexplained by traditional models and the emergence of new technologies. Previous attempts to address this dimension have focused on lane-changing and non-lane-based traffic. The literature on vehicles keeping their lanes has generally been limited to simple statistics on vehicle position while models assume vehicles stay perfectly centered. Previously the author developed a two-dimensional traffic model aiming to capture such behavior qualitatively. Still pending is a deeper, more accurate comprehension and modeling of the relationships between variables in both axes. The present paper is based on the Next Generation SIMulation (NGSIM) datasets. It was found that lateral position is highly dependent on the longitudinal position, a phenomenon consistent with data capture from multiple cameras. A methodology is proposed to alleviate this problem. It was also discovered that the standard deviation of lateral velocity grows with longitudinal velocity and that the average lateral position varies with longitudinal velocity by up to 8 cm, possibly reflecting greater caution in overtaking. Random walk models were proposed and calibrated to reproduce some of the characteristics measured. It was determined that drivers’ response is much more sensitive to the lateral velocity than to position. These results provide a basis for further advances in understanding the lateral dimension. It is hoped that such comprehension will facilitate the design of autonomous vehicle algorithms that are friendlier to both passengers and the occupants of surrounding vehicles.