Environment Classification for Robotic Leg Prostheses and Exoskeletons using Deep Convolutional Neural Networks

Robotic leg prostheses and exoskeletons can provide powered locomotor assistance to older adults and/or persons with physical disabilities. However, the current locomotion mode recognition systems being developed for intelligent high-level control and decision-making use mechanical, inertial, and/or neuromuscular data, which inherently have limited prediction horizons (i.e., analogous to walking blindfolded). Inspired by the human vision-locomotor control system, we designed and evaluated an advanced environment classification system that uses computer vision and deep learning to forward predict the oncoming walking environments prior to physical interaction, therein allowing for more accurate and robust locomotion mode transitions. In this study, we first reviewed the development of the ExoNet database – the largest and most diverse open-source dataset of wearable camera images of indoor and outdoor real-world walking environments, which were annotated using a hierarchical labelling architecture. We then trained and tested over a dozen state-of-the-art deep convolutional neural networks (CNNs) on the ExoNet database for large-scale image classification of the walking environments, including: EfficientNetB0, InceptionV3, MobileNet, MobileNetV2, VGG16, VGG19, Xception, ResNet50, ResNet101, ResNet152, DenseNet121, DenseNet169, and DenseNet201. Lastly, we quantitatively compared the benchmarked CNN architectures and their environment classification predictions using an operational metric called NetScore, which balances the image classification accuracy with the computational and memory storage requirements (i.e., important for onboard real-time inference). Although we designed this environment classification system to support the development of next-generation environment-adaptive locomotor control systems for robotic prostheses and exoskeletons, applications could extend to humanoids, autonomous legged robots, powered wheelchairs, and assistive devices for persons with visual impairments.

Download Full-text

CNN-based Classification of Degraded Images

Electronic Imaging ◽

10.2352/issn.2470-1173.2020.10.ipas-028 ◽

2020 ◽

Vol 2020 (10) ◽

pp. 28-1-28-7 ◽

Cited By ~ 1

Author(s):

Kazuki Endo ◽

Masayuki Tanaka ◽

Masatoshi Okutomi

Keyword(s):

Neural Networks ◽

Image Restoration ◽

Image Classification ◽

Convolutional Neural Networks ◽

Deep Convolutional Neural Networks ◽

Alternative Approach ◽

Degraded Image ◽

Degraded Images ◽

Straightforward Approach

Classification of degraded images is very important in practice because images are usually degraded by compression, noise, blurring, etc. Nevertheless, most of the research in image classification only focuses on clean images without any degradation. Some papers have already proposed deep convolutional neural networks composed of an image restoration network and a classification network to classify degraded images. This paper proposes an alternative approach in which we use a degraded image and an additional degradation parameter for classification. The proposed classification network has two inputs which are the degraded image and the degradation parameter. The estimation network of degradation parameters is also incorporated if degradation parameters of degraded images are unknown. The experimental results showed that the proposed method outperforms a straightforward approach where the classification network is trained with degraded images only.

Download Full-text

Semi-supervised training of deep convolutional neural networks with heterogeneous data and few local annotations: an experiment on prostate histopathology image classification

Medical Image Analysis ◽

10.1016/j.media.2021.102165 ◽

2021 ◽

pp. 102165

Author(s):

Niccolò Marini ◽

Sebastian Otálora ◽

Henning Müller ◽

Manfredo Atzori

Keyword(s):

Neural Networks ◽

Image Classification ◽

Convolutional Neural Networks ◽

Heterogeneous Data ◽

Deep Convolutional Neural Networks ◽

Supervised Training

Download Full-text

Document Image Classification with Intra-Domain Transfer Learning and Stacked Generalization of Deep Convolutional Neural Networks

2018 24th International Conference on Pattern Recognition (ICPR) ◽

10.1109/icpr.2018.8545630 ◽

2018 ◽

Cited By ~ 9

Author(s):

Arindam Das ◽

Saikat Roy ◽

Ujjwal Bhattacharya ◽

Swapan K. Parui

Keyword(s):

Neural Networks ◽

Image Classification ◽

Transfer Learning ◽

Convolutional Neural Networks ◽

Document Image ◽

Deep Convolutional Neural Networks ◽

Stacked Generalization ◽

Domain Transfer ◽

Domain Transfer Learning

Download Full-text

Medical Image Classification with Weighted Latent Semantic Tensors and Deep Convolutional Neural Networks

Lecture Notes in Computer Science - Experimental IR Meets Multilinguality, Multimodality, and Interaction ◽

10.1007/978-3-319-98932-7_8 ◽

2018 ◽

pp. 89-100 ◽

Cited By ~ 1

Author(s):

Spyridon Stathopoulos ◽

Theodore Kalamboukis

Keyword(s):

Neural Networks ◽

Image Classification ◽

Convolutional Neural Networks ◽

Medical Image ◽

Deep Convolutional Neural Networks ◽

Medical Image Classification

Download Full-text

Facial skin image classification system using Convolutional Neural Networks deep learning algorithm

2018 9th International Conference on Awareness Science and Technology (iCAST) ◽

10.1109/icawst.2018.8517246 ◽

2018 ◽

Author(s):

Chiun-Li Chin ◽

Ming-Chieh Chin ◽

Ting-Yu Tsai ◽

Wei-En Chen

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Image Classification ◽

Convolutional Neural Networks ◽

Classification System ◽

Learning Algorithm ◽

Facial Skin ◽

Deep Learning Algorithm

Download Full-text

A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

10.26686/wgtn.13158299.v1 ◽

2020 ◽

Author(s):

B Wang ◽

Y Sun ◽

Bing Xue ◽

Mengjie Zhang

Keyword(s):

Neural Networks ◽

Image Classification ◽

Convolutional Neural Networks ◽

Network Architecture ◽

Learning Task ◽

Fixed Number ◽

Learning Rate ◽

Current Layer ◽

Training Process ◽

Deep Convolutional Neural Networks

© 2019, Springer Nature Switzerland AG. Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the problem. In recent years, instead of the traditional way of only connecting the current layer with its next layer, shortcut connections have been proposed to connect the current layer with its forward layers apart from its next layer, which has been proved to be able to facilitate the training process of deep CNNs. However, there are various ways to build the shortcut connections, it is hard to manually design the best shortcut connections when solving a particular problem, especially given the design of the network architecture is already very challenging. In this paper, a hybrid evolutionary computation (EC) method is proposed to automatically evolve both the architecture of deep CNNs and the shortcut connections. Three major contributions of this work are: Firstly, a new encoding strategy is proposed to encode a CNN, where the architecture and the shortcut connections are encoded separately; Secondly, a hybrid two-level EC method, which combines particle swarm optimisation and genetic algorithms, is developed to search for the optimal CNNs; Lastly, an adjustable learning rate is introduced for the fitness evaluations, which provides a better learning rate for the training process given a fixed number of epochs. The proposed algorithm is evaluated on three widely used benchmark datasets of image classification and compared with 12 peer Non-EC based competitors and one EC based competitor. The experimental results demonstrate that the proposed method outperforms all of the peer competitors in terms of classification accuracy.

Download Full-text