Virtual Reality Aided High-Quality 3D Reconstruction by Remote Drones

Artificial intelligence including deep learning and 3D reconstruction methods is changing the daily life of people. Now, an unmanned aerial vehicle that can move freely in the air and avoid harsh ground conditions has been commonly adopted as a suitable tool for 3D reconstruction. The traditional 3D reconstruction mission based on drones usually consists of two steps: image collection and offline post-processing. But there are two problems: one is the uncertainty of whether all parts of the target object are covered, and another is the tedious post-processing time. Inspired by modern deep learning methods, we build a telexistence drone system with an onboard deep learning computation module and a wireless data transmission module that perform incremental real-time dense reconstruction of urban cities by itself. Two technical contributions are proposed to solve the preceding issues. First, based on the popular depth fusion surface reconstruction framework, we combine it with a visual-inertial odometry estimator that integrates the inertial measurement unit and allows for robust camera tracking as well as high-accuracy online 3D scan. Second, the capability of real-time 3D reconstruction enables a new rendering technique that can visualize the reconstructed geometry of the target as navigation guidance in the HMD. Therefore, it turns the traditional path-planning-based modeling process into an interactive one, leading to a higher level of scan completeness. The experiments in the simulation system and our real prototype demonstrate an improved quality of the 3D model using our artificial intelligence leveraged drone system.

Semi-Direct Monocular Visual-Inertial Odometry Using Point and Line Features for IoV

The precise measuring of vehicle location has been a critical task in enhancing the autonomous driving in terms of intelligent decision making and safe transportation. Internet of Vehicles ( IoV ) is an important infrastructure in support of autonomous driving, allowing real-time road information exchanging and sharing for localizing vehicles. Global positioning System ( GPS ) is widely used in the traditional IoV system. GPS is unable to meet the key application requirements of autonomous driving due to meter level error and signal deterioration. In this article, we propose a novel solution, named Semi-Direct Monocular Visual-Inertial Odometry using Point and Line Features ( SDMPL-VIO ) for precise vehicle localization. Our SDMPL-VIO model takes advantage of a low-cost Inertial Measurement Unit ( IMU ) and monocular camera, using them as the sensor to acquire the surrounding environmental information. Visual-Inertial Odometry ( VIO ), taking into account both point and line features, is proposed, which is able to deal with both weak texture and dynamic environment. We use a semi-direct method to deal with keyframes and non-keyframes, respectively. Dual sliding window mechanisms can effectively fuse point-line and IMU information. To evaluate our SDMPL-VIO system model, we conduct extensive experiments on both an indoor dataset (i.e., EuRoC) and an outdoor dataset (i.e., KITTI) from the real-world applications, respectively. The experimental results show that the accuracy of SDMPL-VIO proposed by us is better than the mainstream VIO system at present. Especially in the weak texture of the datasets, fast-moving datasets, and other challenging datasets, SDMPL-VIO has a relatively high robustness.

Muscle force estimation from lower limb EMG signals using novel optimised machine learning techniques

The main objective of this work is to establish a framework for processing and evaluating the lower limb electromyography (EMG) signals ready to be fed to a rehabilitation robot. We design and build a knee rehabilitation robot that works with surface EMG (sEMG) signals. In our device, the muscle forces are estimated from sEMG signals using several machine learning techniques, i.e. support vector machine (SVM), support vector regression (SVR) and random forest (RF). In order to improve the estimation accuracy, we devise genetic algorithm (GA) for parameter optimisation and feature extraction within the proposed methods. At the same time, a load cell and a wearable inertial measurement unit (IMU) are mounted on the robot to measure the muscle force and knee joint angle, respectively. Various performance measures have been employed to assess the performance of the proposed system. Our extensive experiments and comparison with related works revealed a high estimation accuracy of 98.67% for lower limb muscles. The main advantage of the proposed techniques is high estimation accuracy leading to improved performance of the therapy while muscle models become especially sensitive to the tendon stiffness and the slack length.

The Validity of Wireless Earbud-Type Wearable Sensors for Head Angle Estimation and the Relationships of Head with Trunk, Pelvis, Hip, and Knee during Workouts

The present study was performed to investigate the validity of a wireless earbud-type inertial measurement unit (Ear-IMU) sensor used to estimate head angle during four workouts. In addition, relationships between head angle obtained from the Ear-IMU sensor and the angles of other joints determined with a 3D motion analysis system were investigated. The study population consisted of 20 active volunteers. The Ear-IMU sensor measured the head angle, while a 3D motion analysis system simultaneously measured the angles of the head, trunk, pelvis, hips, and knees during workouts. Comparison with the head angle measured using the 3D motion analysis system indicated that the validity of the Ear-IMU sensor was very strong or moderate in the sagittal and frontal planes. In addition, the trunk angle in the frontal plane showed a fair correlation with the head angle determined with the Ear-IMU sensor during a single-leg squat, reverse lunge, and standing hip abduction; the correlation was poor in the sagittal plane. Our results indicated that the Ear-IMU sensor can be used to directly estimate head motion and indirectly estimate trunk motion.

Graph-optimisation-based self-calibration method for IMU/odometer using preintegration theory

Determination of calibration parameters is essential for the fusion performance of an inertial measurement unit (IMU) and odometer integrated navigation system. Traditional calibration methods are commonly based on the filter frame, which limits the improvement of the calibration accuracy. This paper proposes a graph-optimisation-based self-calibration method for the IMU/odometer using preintegration theory. Different from existing preintegrations, the complete IMU/odometer preintegration model is derived, which takes into consideration the effects of the scale factor of the odometer, and misalignments in the attitude and position between the IMU and odometer. Then the calibration is implemented by the graph-optimisation method. The KITTI dataset and field experimental tests are carried out to evaluate the effectiveness of the proposed method. The results illustrate that the proposed method outperforms the filter-based calibration method. Meanwhile, the performance of the proposed IMU/odometer preintegration model is optimal compared with the traditional preintegration models.

Enhanced Attitude and Altitude Estimation for Indoor Autonomous UAVs

In recent years the use of Unmanned Aerial Vehicles (UAVs) has considerably grown in the civil sectors, due to their high flexibility of use. Currently, two important key points are making them more and more successful in the civil field, namely the decrease of production costs and the increase in navigation accuracy. In this paper, we propose a Kalman filtering-based sensor fusion algorithm, using a low cost navigation platform that contains an inertial measurement unit (IMU), five ultrasonic ranging sensors and an optical flow camera. The aim is to improve navigation in indoor or GPS-denied environments. A multi-rate version of the Extended Kalman Filter is considered to deal with the use of heterogeneous sensors with different sampling rates, and the presence of non-linearities in the model. The effectiveness of the proposed sensor platform is evaluated by means of numerical tests on the dynamic flight simulator of a quadrotor. Results show high precision and robustness of the attitude estimation algorithm, with a reduced computational cost, being ready to be implemented on low-cost platforms.

Estimating the Gait Speed of Older Adults in Smart Home Environments

Mobility is one of the key performance indicators of the health condition of older adults. One important parameter is the gait speed. The mobility is usually assessed under the supervision of a professional by standardised geriatric assessments. Using sensors in smart home environments for continuous monitoring of the gait speed enables physicians to detect early stages of functional decline and to initiate appropriate interventions. This in combination with a floor plan smart home sensors were used to calculate the distance that a person walked in the apartment and the inertial measurement unit data for estimating the actual walking time. A Gaussian kernel density estimator was applied to the computed values and the maximum of the kernel density estimator was considered as the gait speed. The proposed method was evaluated on a real-world dataset and the estimations of the gait speed had a deviation smaller than $$0.10 \, \frac{\mathrm{m}}{\mathrm{s}}$$ 0.10 m s , which is smaller than the minimal clinically important difference, compared to a baseline from a standardised geriatrics assessment.

MEMS Accelerometer Noises Analysis Based on Triple Estimation Fractional Order Algorithm

This paper is devoted to identifying parameters of fractional order noises with application to noises obtained from MEMS accelerometer. The analysis and parameters estimation will be based on the Triple Estimation algorithm, which can simultaneously estimate state, fractional order, and parameter estimates. The capability of the Triple Estimation algorithm to fractional noises estimation will be confirmed by the sets of numerical analyses for fractional constant and variable order systems with Gaussian noise input signal. For experimental data analysis, the MEMS sensor SparkFun MPU9250 Inertial Measurement Unit (IMU) was used with data obtained from the accelerometer in x, y and z-axes. The experimental results clearly show the existence of fractional noise in this MEMS’ noise, which can be essential information in the design of filtering algorithms, for example, in inertial navigation.

Implementation of Snow and Ice Sports Health and Sports Information Collection System Based on Internet of Things

The promotion of ice and snow sports not only provides professional athletes for the Winter Olympics but also acts as appreciative mass bases for ice and snow sports. The appearance of ice and snow sports will bring a new consumption pattern and develop a new ice and snow industry. In this paper, an Internet of Things (IoT)-based sports information collection system which is specifically designed and developed for the healthcare domain specifically in the snow and ice sports is proposed. The physiological parameters such as body temperature, ECG, blood pressure, blood sugar, and blood oxygen saturation are captured through various monitoring devices. These physiological parameters are transmitted to the mobile device by the wireless module and mobile device that receives and displays these physiological parameters. A complete hardware design of the whole ice and snow sports health and sports information acquisition system, which is based on the Internet of Things, is given, and then, there is the overall design scheme of the system, such as adopted modular design for the system, attitude measurement unit, UWB positioning unit, data storage, and communication unit, respectively. The measurement results of the professional medical equipment are compared with those of acquisition equipment in real environment of ice and sports. These results have verified accuracy of data collected by acquisition equipment and meet the design requirements of the proposed system.

Predicting Knee Joint Kinematics from Wearable Sensor Data in People with Knee Osteoarthritis and Clinical Considerations for Future Machine Learning Models

Deep learning models developed to predict knee joint kinematics are usually trained on inertial measurement unit (IMU) data from healthy people and only for the activity of walking. Yet, people with knee osteoarthritis have difficulties with other activities and there are a lack of studies using IMU training data from this population. Our objective was to conduct a proof-of-concept study to determine the feasibility of using IMU training data from people with knee osteoarthritis performing multiple clinically important activities to predict knee joint sagittal plane kinematics using a deep learning approach. We trained a bidirectional long short-term memory model on IMU data from 17 participants with knee osteoarthritis to estimate knee joint flexion kinematics for phases of walking, transitioning to and from a chair, and negotiating stairs. We tested two models, a double-leg model (four IMUs) and a single-leg model (two IMUs). The single-leg model demonstrated less prediction error compared to the double-leg model. Across the different activity phases, RMSE (SD) ranged from 7.04° (2.6) to 11.78° (6.04), MAE (SD) from 5.99° (2.34) to 10.37° (5.44), and Pearson’s R from 0.85 to 0.99 using leave-one-subject-out cross-validation. This study demonstrates the feasibility of using IMU training data from people who have knee osteoarthritis for the prediction of kinematics for multiple clinically relevant activities.