Single Depth View Based Real-Time Reconstruction of Hand-Object Interactions

2021 ◽  
Vol 40 (3) ◽  
pp. 1-12
Author(s):  
Hao Zhang ◽  
Yuxiao Zhou ◽  
Yifei Tian ◽  
Jun-Hai Yong ◽  
Feng Xu
Keyword(s):  
Real Time ◽  
Synthetic Data ◽  
Real Data ◽  
Depth Image ◽  
Real Time System ◽  
The Real ◽  
Time Performance ◽  
Contact Constraint ◽  
Object Shapes ◽  
Object Interactions

Reconstructing hand-object interactions is a challenging task due to strong occlusions and complex motions. This article proposes a real-time system that uses a single depth stream to simultaneously reconstruct hand poses, object shape, and rigid/non-rigid motions. To achieve this, we first train a joint learning network to segment the hand and object in a depth image, and to predict the 3D keypoints of the hand. With most layers shared by the two tasks, computation cost is saved for the real-time performance. A hybrid dataset is constructed here to train the network with real data (to learn real-world distributions) and synthetic data (to cover variations of objects, motions, and viewpoints). Next, the depth of the two targets and the keypoints are used in a uniform optimization to reconstruct the interacting motions. Benefitting from a novel tangential contact constraint, the system not only solves the remaining ambiguities but also keeps the real-time performance. Experiments show that our system handles different hand and object shapes, various interactive motions, and moving cameras.

Hardware-in-the-loop simulation for real-time software verification of an autonomous underwater robot

2016 ◽  
Vol 4 (3) ◽  
pp. 163-181
Author(s):  
Pouria Sarhadi ◽  
Reza Nad Ali Niachari ◽  
Morteza Pouyan Rad ◽  
Javad Enayati
Keyword(s):  
Software Engineering ◽  
Real Time ◽  
Autonomous Systems ◽  
Systems Design ◽  
Development Stage ◽  
Underwater Robot ◽  
Real Time System ◽  
Content Type ◽  
The Real ◽  
Time Performance

Purpose The purpose of this paper is to propose a software engineering procedure for real-time software development and verification of an autonomous underwater robotic system. High performance and robust software are one of the requirements of autonomous systems design. A simple error in the software can easily lead to a catastrophic failure in a complex system. Then, a systematic procedure is presented for this purpose. Design/methodology/approach This paper utilizes software engineering tools and hardware-inthe-loop (HIL) simulations for real-time system design of an autonomous underwater robot. Findings In this paper, the architecture of the system is extracted. Then, using software engineering techniques a suitable structure for control software is presented. Considering the desirable targets of the robot, suitable algorithms and functions are developed. After the development stage, proving the real-time performance of the software is disclosed. Originality/value A suitable approach for analyzing the real-time performance is presented. This approach is implemented using HIL simulations. The developed structure is applicable to other autonomous systems.

scholarly journals Image Noise Preprocessing of Interactive Projection System Based on Switching Filtering Scheme

Complexity ◽  
2018 ◽  
Vol 2018 ◽  
pp. 1-10 ◽  
Author(s):  
Lei Yu
Keyword(s):  
Real Time ◽  
Image Filtering ◽  
Depth Image ◽  
Projection System ◽  
The Real ◽  
Time Performance ◽  
Arbitrary Switching ◽  
System Method ◽  
Good Target ◽  
Switching Filter

The interactive projection systems based on deep images are usually disturbed by the mixed noise. Generally, several filtering methods are used in combination to resolve this problem. Although the hybrid filter can guarantee the accuracy of the image, but the algorithm is complex and time-consuming, which affects the real-time performance of the interactive projection system. In this paper, the switching system method is introduced into the filter for the first time, and an arbitrary switching filter algorithm is proposed and applied to the depth image filtering system based on Kinect sensor. The experimental results demonstrate and validate that the proposed switching filter algorithm not only effectively removes the noise but also ensures the real-time performance of tracking and achieves good target tracking performance, which makes it applicable in various image filtering processing systems.

scholarly journals Real-Time Multiobject Tracking Based on Multiway Concurrency

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 685
Author(s):  
Xuan Gong ◽  
Zichun Le ◽  
Yukun Wu ◽  
Hui Wang
Keyword(s):  
Real Time ◽  
System Level ◽  
Depth Image ◽  
Tracking Accuracy ◽  
Video Streams ◽  
The Real ◽  
Practical Applications ◽  
Time Performance ◽  
Tracking By Detection ◽  
Multiobject Tracking

This paper explored a pragmatic approach to research the real-time performance of a multiway concurrent multiobject tracking (MOT) system. At present, most research has focused on the tracking of single-image sequences, but in practical applications, multiway video streams need to be processed in parallel by MOT systems. There have been few studies on the real-time performance of multiway concurrent MOT systems. In this paper, we proposed a new MOT framework to solve multiway concurrency scenario based on a tracking-by-detection (TBD) model. The new framework mainly focuses on concurrency and real-time based on limited computing and storage resources, while considering the algorithm performance. For the former, three aspects were studied: (1) Expanded width and depth of tracking-by-detection model. In terms of width, the MOT system can support the process of multiway video sequence at the same time; in terms of depth, image collectors and bounding box collectors were introduced to support batch processing. (2) Considering the real-time performance and multiway concurrency ability, we proposed one kind of real-time MOT algorithm based on directly driven detection. (3) Optimization of system level—we also utilized the inference optimization features of NVIDIA TensorRT to accelerate the deep neural network (DNN) in the tracking algorithm. To trade off the performance of the algorithm, a negative sample (false detection sample) filter was designed to ensure tracking accuracy. Meanwhile, the factors that affect the system real-time performance and concurrency were studied. The experiment results showed that our method has a good performance in processing multiple concurrent real-time video streams.

Real-Time System for Sending Data from Three Clients to One Server with MATLAB

2015 ◽  
Vol 2 (1) ◽  
pp. 35-41
Author(s):  
Rivan Risdaryanto ◽  
Houtman P. Siregar ◽  
Dedy Loebis
Keyword(s):  
Information System ◽  
Real Time ◽  
Time System ◽  
Real Time System ◽  
The Real ◽  
Performance Time ◽  
Data Process ◽  
Effectiveness And Efficiency

The real-time system is now used on many fields, such as telecommunication, military, information system, evenmedical to get information quickly, on time and accurate. Needless to say, a real-time system will always considerthe performance time. In our application, we define the time target/deadline, so that the system should execute thewhole tasks under predefined deadline. However, if the system failed to finish the tasks, it will lead to fatal failure.In other words, if the system cannot be executed on time, it will affect the subsequent tasks. In this paper, wepropose a real-time system for sending data to find effectiveness and efficiency. Sending data process will beconstructed in MATLAB and sending data process has a time target as when data will send.

Single Chip Microcomputer Cluster Management

2014 ◽  
Vol 933 ◽  
pp. 584-589
Author(s):  
Zhi Chun Zhang ◽  
Song Wei Li ◽  
Wei Ren Wang ◽  
Wei Zhang ◽  
Li Jun Qi
Keyword(s):  
Real Time ◽  
Management System ◽  
Large Scale ◽  
Fault Tolerant ◽  
Single Chip ◽  
Single Chip Microcomputer ◽  
Cluster Management ◽  
The Real ◽  
Time Performance ◽  
Operational Phase

This paper presents a system in which the cluster devices are controlled by single-chip microcomputers, with emphasis on the cluster management techniques of single-chip microcomputers. Each device in a cluster is controlled by a single-chip microcomputer collecting sample data sent to and driving the device by driving data received from the same cluster management computer through COMs. The cluster management system running on the cluster management computer carries out such control as initial SCM identification, run time slice management, communication resource utilization, fault tolerance and error corrections on single-chip microcomputers. Initial SCM identification is achieved by signal responses between the single-chip microcomputers and the cluster management computer. By using the port priority and the parallelization of serial communications, the systems real-time performance is maximized. The real-time performance can be adjusted and improved by increasing or decreasing COMs and the ports linked to each COM, and the real-time performance can also be raised by configuring more cluster management computers. Fault-tolerant control occurs in the initialization phase and the operational phase. In the initialization phase, the cluster management system incorporates unidentified single-chip microcomputers into the system based on the history information recorded on external storage media. In the operational phase, if an operation error of reading and writing on a single-chip microcomputer reaches a predetermined threshold, the single-chip microcomputer is regarded as serious fault or not existing. The cluster management system maintains accuracy maintenance database on external storage medium to solve nonlinear control of specific devices and accuracy maintenance due to wear. The cluster management system uses object-oriented method to design a unified driving framework in order to enable the implementation of the cluster management system simplified, standardized and easy to transplant. The system has been applied in a large-scale simulation system of 230 single-chip microcomputers, which proves that the system is reliable, real-time and easy to maintain.

Switching filtering method for interactive systems

2019 ◽  
Vol 233 (10) ◽  
pp. 1340-1350
Author(s):  
Junyi Hou ◽  
Lei Yu ◽  
Yifan Fang ◽  
Shumin Fei
Keyword(s):  
Real Time ◽  
Subjective Evaluation ◽  
Objective Evaluation ◽  
Interactive Systems ◽  
Interactive System ◽  
The Real ◽  
Noise Interference ◽  
Filtering Method ◽  
Time Performance ◽  
Filtering Effect

Aiming at the problem that the mixed noise interference caused by the mixed projection noise system is not accurate and the real-time performance is poor, this article proposes an adaptive system switching filtering method based on Bayesian estimation switching rules. The method chooses joint bilateral filtering and improved adaptive median filtering as the filtering subsystems and selects the sub-filtering system suitable for the noise by switching rules to achieve the purpose of effectively removing noise. The simulation experiment was carried out by the self-developed human–computer interactive projection image system platform. Through the subjective evaluation, objective evaluation, and running time comparison analysis, a better filtering effect was achieved, and the balance between the filtering precision and the real-time performance of the interactive system was well obtained. Therefore, the proposed method can be widely applied to various human–computer interactive image filtering systems.

Large‐scale three‐dimensional seismic models and their interpretive significance

Geophysics ◽  
1990 ◽  
Vol 55 (9) ◽  
pp. 1166-1182 ◽  
Author(s):  
Irshad R. Mufti
Keyword(s):  
Finite Difference ◽  
Wave Field ◽  
Large Scale ◽  
Three Dimensional ◽  
Synthetic Data ◽  
Real Data ◽  
The Real ◽  
Need To Evaluate ◽  
Zero Offset ◽  
Seismic Models

Finite‐difference seismic models are commonly set up in 2-D space. Such models must be excited by a line source which leads to different amplitudes than those in the real data commonly generated from a point source. Moreover, there is no provision for any out‐of‐plane events. These problems can be eliminated by using 3-D finite‐difference models. The fundamental strategy in designing efficient 3-D models is to minimize computational work without sacrificing accuracy. This was accomplished by using a (4,2) differencing operator which ensures the accuracy of much larger operators but requires many fewer numerical operations as well as significantly reduced manipulation of data in the computer memory. Such a choice also simplifies the problem of evaluating the wave field near the subsurface boundaries of the model where large operators cannot be used. We also exploited the fact that, unlike the real data, the synthetic data are free from ambient noise; consequently, one can retain sufficient resolution in the results by optimizing the frequency content of the source signal. Further computational efficiency was achieved by using the concept of the exploding reflector which yields zero‐offset seismic sections without the need to evaluate the wave field for individual shot locations. These considerations opened up the possibility of carrying out a complete synthetic 3-D survey on a supercomputer to investigate the seismic response of a large‐scale structure located in Oklahoma. The analysis of results done on a geophysical workstation provides new insight regarding the role of interference and diffraction in the interpretation of seismic data.

Sign in / Sign up

Export Citation Format

Share Document