scholarly journals The PAIR-R24M Dataset for Multi-animal 3D Pose Estimation

2021 ◽  
Author(s):  
Jesse D Marshall ◽  
Ugne Klibaite ◽  
Amanda J Gellis ◽  
Diego E Aldarondo ◽  
Bence P Olveczky ◽  
...  
Keyword(s):  
Pose Estimation ◽  
Challenging Behaviors ◽  
Social Behaviors ◽  
Ground Truth ◽  
Laboratory Animals ◽  
Multi Agent Systems ◽  
Neural Basis ◽  
3D Pose Estimation ◽  
Collective Behaviors ◽  
Benchmark Datasets

Understanding the biological basis of social and collective behaviors in animals is a key goal of the life sciences, and may yield important insights for engineering intelligent multi-agent systems. A critical step in interrogating the mechanisms underlying social behaviors is a precise readout of the 3D pose of interacting animals. While approaches for multi-animal pose estimation are beginning to emerge, they remain challenging to compare due to the lack of standardized training and benchmark datasets. Here we introduce the PAIR-R24M (Paired Acquisition of Interacting oRganisms - Rat) dataset for multi-animal 3D pose estimation, which contains 24.3 million frames of RGB video and 3D ground-truth motion capture of dyadic interactions in laboratory rats. PAIR-R24M contains data from 18 distinct pairs of rats and 24 different viewpoints. We annotated the data with 11 behavioral labels and 3 interaction categories to facilitate benchmarking in rare but challenging behaviors. To establish a baseline for markerless multi-animal 3D pose estimation, we developed a multi-animal extension of DANNCE, a recently published network for 3D pose estimation in freely behaving laboratory animals. As the first large multi-animal 3D pose estimation dataset, PAIR-R24M will help advance 3D animal tracking approaches and aid in elucidating the neural basis of social behaviors.

scholarly journals The Mouse Action Recognition System (MARS): a software pipeline for automated analysis of social behaviors in mice

2020 ◽  
Author(s):  
Cristina Segalin ◽  
Jalani Williams ◽  
Tomomi Karigo ◽  
May Hui ◽  
Moriel Zelikowsky ◽  
...  
Keyword(s):  
Social Behavior ◽  
Action Recognition ◽  
Pose Estimation ◽  
Data Extraction ◽  
Social Behaviors ◽  
Automated Analysis ◽  
Recognition System ◽  
Laboratory Animals ◽  
Freely Behaving ◽  
And Behavior

AbstractThe study of social behavior requires scoring the animals’ interactions. This is generally done by hand— a time consuming, subjective, and expensive process. Recent advances in computer vision enable tracking the pose (posture) of freely-behaving laboratory animals automatically. However, classifying complex social behaviors such as mounting and attack remains technically challenging. Furthermore, the extent to which expert annotators, possibly from different labs, agree on the definitions of these behaviors varies. There is a shortage in the neuroscience community of benchmark datasets that can be used to evaluate the performance and reliability of both pose estimation tools and manual and automated behavior scoring.We introduce the Mouse Action Recognition System (MARS), an automated pipeline for pose estimation and behavior quantification in pairs of freely behaving mice. We compare MARS’s annotations to human annotations and find that MARS’s pose estimation and behavior classification achieve human-level performance. As a by-product we characterize the inter-expert variability in behavior scoring. The two novel datasets used to train MARS were collected from ongoing experiments in social behavior, and identify the main sources of disagreement between annotators. They comprise 30,000 frames of manual annotated mouse poses and over 14 hours of manually annotated behavioral recordings in a variety of experimental preparations. We are releasing this dataset alongside MARS to serve as community benchmarks for pose and behavior systems. Finally, we introduce the Behavior Ensemble and Neural Trajectory Observatory (Bento), a graphical interface that allows users to quickly browse, annotate, and analyze datasets including behavior videos, pose estimates, behavior annotations, audio, and neural recording data. We demonstrate the utility of MARS and Bento in two use cases: a high-throughput behavioral phenotyping study, and exploration of a novel imaging dataset. Together, MARS and Bento provide an end-to-end pipeline for behavior data extraction and analysis, in a package that is user-friendly and easily modifiable.

scholarly journals LiftPose3D, a deep learning-based approach for transforming 2D to 3D pose in laboratory animals

2020 ◽  
Author(s):  
Adam Gosztolai ◽  
Semih Günel ◽  
Marco Pietro Abrate ◽  
Daniel Morales ◽  
Victor Lobato Ríos ◽  
...  
Keyword(s):  
Pose Estimation ◽  
Laboratory Animals ◽  
3D Pose Estimation ◽  
Deep Network ◽  
Experimental Systems ◽  
Camera View ◽  
Kinematic Studies ◽  
Freely Behaving ◽  
Indispensable Tool ◽  
Camera Arrays

AbstractMarkerless 3D pose estimation has become an indispensable tool for kinematic studies of laboratory animals. Most current methods recover 3D pose by multi-view triangulation of deep network-based 2D pose estimates. However, triangulation requires multiple, synchronised cameras per keypoint and elaborate calibration protocols that hinder its widespread adoption in laboratory studies. Here, we describe LiftPose3D, a deep network-based method that overcomes these barriers by reconstructing 3D poses from a single 2D camera view. We illustrate LiftPose3D’s versatility by applying it to multiple experimental systems using flies, mice, and macaque monkeys and in circumstances where 3D triangulation is impractical or impossible. Thus, LiftPose3D permits high-quality 3D pose estimation in the absence of complex camera arrays, tedious calibration procedures, and despite occluded keypoints in freely behaving animals.

scholarly journals RobotP: A Benchmark Dataset for 6D Object Pose Estimation

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1299
Author(s):  
Honglin Yuan ◽  
Tim Hoogenkamp ◽  
Remco C. Veltkamp
Keyword(s):  
Pose Estimation ◽  
Ground Truth ◽  
3D Models ◽  
Depth Image ◽  
Great Success ◽  
Estimation Algorithms ◽  
Depth Images ◽  
Object Pose Estimation ◽  
Image Pairs ◽  
Bounding Boxes

Deep learning has achieved great success on robotic vision tasks. However, when compared with other vision-based tasks, it is difficult to collect a representative and sufficiently large training set for six-dimensional (6D) object pose estimation, due to the inherent difficulty of data collection. In this paper, we propose the RobotP dataset consisting of commonly used objects for benchmarking in 6D object pose estimation. To create the dataset, we apply a 3D reconstruction pipeline to produce high-quality depth images, ground truth poses, and 3D models for well-selected objects. Subsequently, based on the generated data, we produce object segmentation masks and two-dimensional (2D) bounding boxes automatically. To further enrich the data, we synthesize a large number of photo-realistic color-and-depth image pairs with ground truth 6D poses. Our dataset is freely distributed to research groups by the Shape Retrieval Challenge benchmark on 6D pose estimation. Based on our benchmark, different learning-based approaches are trained and tested by the unified dataset. The evaluation results indicate that there is considerable room for improvement in 6D object pose estimation, particularly for objects with dark colors, and photo-realistic images are helpful in increasing the performance of pose estimation algorithms.

scholarly journals Feature Boosting Network For 3D Pose Estimation

2020 ◽  
Vol 42 (2) ◽  
pp. 494-501 ◽  
Author(s):  
Jun Liu ◽  
Henghui Ding ◽  
Amir Shahroudy ◽  
Ling-Yu Duan ◽  
Xudong Jiang ◽  
...  
Keyword(s):  
Pose Estimation ◽  
3D Pose Estimation

scholarly journals Gait analysis for early Parkinson’s disease detection based on deep learning

2019 ◽  
Vol 5 (1) ◽  
pp. 9-12
Author(s):  
Jyothsna Kondragunta ◽  
Christian Wiede ◽  
Gangolf Hirtz
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Deep Learning ◽  
Gait Analysis ◽  
Pose Estimation ◽  
Early Identification ◽  
Human Gait ◽  
3D Pose Estimation ◽  
Wearable Technologies ◽  
Advantages And Disadvantages

AbstractBetter handling of neurological or neurodegenerative disorders such as Parkinson’s Disease (PD) is only possible with an early identification of relevant symptoms. Although the entire disease can’t be treated but the effects of the disease can be delayed with proper care and treatment. Due to this fact, early identification of symptoms for the PD plays a key role. Recent studies state that gait abnormalities are clearly evident while performing dual cognitive tasks by people suffering with PD. Researches also proved that the early identification of the abnormal gaits leads to the identification of PD in advance. Novel technologies provide many options for the identification and analysis of human gait. These technologies can be broadly classified as wearable and non-wearable technologies. As PD is more prominent in elderly people, wearable sensors may hinder the natural persons movement and is considered out of scope of this paper. Non-wearable technologies especially Image Processing (IP) approaches captures data of the person’s gait through optic sensors Existing IP approaches which perform gait analysis is restricted with the parameters such as angle of view, background and occlusions due to objects or due to own body movements. Till date there exists no researcher in terms of analyzing gait through 3D pose estimation. As deep leaning has proven efficient in 2D pose estimation, we propose an 3D pose estimation along with proper dataset. This paper outlines the advantages and disadvantages of the state-of-the-art methods in application of gait analysis for early PD identification. Furthermore, the importance of extracting the gait parameters from 3D pose estimation using deep learning is outlined.

Neurobiology of premature ejaculation - from animal experiments to evidence- based treatment in humans

2011 ◽  
Vol 26 (S2) ◽  
pp. 2034-2034
Author(s):  
M.D. Waldinger
Keyword(s):  
Premature Ejaculation ◽  
Animal Experiments ◽  
Animal Research ◽  
Laboratory Animals ◽  
Evidence Based ◽  
Female Rat ◽  
Neural Basis ◽  
Behavioral Studies ◽  
Female Wistar Rats ◽  
Lifelong Premature Ejaculation

In the last two decades an increasing number of sexual behavioral studies in laboratory animals has contributed to a better understanding of the neural basis of ejaculatory functioning. In addition, these studies, which mainly involved male and female Wistar rats, elucidated basic mechanisms that underly the psychopharmacology of SSRI-induced ejaculation delay. Notably, a new animal model for premature and retarded ejaculation has been developed. This model has been shown to be of eminent importance for the investigation of ejaculatory disorders. Moreover, it has been proven useful into the investigation of female rat sexual motivation. Translational research translates findings of animal research into human application. Indeed, objective and systematic psychopharmacological research in men with lifelong premature ejaculation yields a remarkable similarity with data that have been found in animals. Sofar, animal research of premature ejaculation remarkably predicts data in men with lifelong PE, on the condition that clinical human research was performed according to evidence based research principles.

SGAN4AbSum: A Semantic-Enhanced Generative Adversarial Network for Abstractive Text Summarization

2021 ◽  
Author(s):  
Tham Vo
Keyword(s):  
Ground Truth ◽  
Text Summarization ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Training Strategy ◽  
Adversarial Network ◽  
Deep Recurrent Neural Network ◽  
Benchmark Datasets ◽  
Latent Representations ◽  
Abstractive Summarization

Abstract In abstractive summarization task, most of proposed models adopt the deep recurrent neural network (RNN)-based encoder-decoder architecture to learn and generate meaningful summary for a given input document. However, most of recent RNN-based models always suffer the challenges related to the involvement of much capturing high-frequency/reparative phrases in long documents during the training process which leads to the outcome of trivial and generic summaries are generated. Moreover, the lack of thorough analysis on the sequential and long-range dependency relationships between words within different contexts while learning the textual representation also make the generated summaries unnatural and incoherent. To deal with these challenges, in this paper we proposed a novel semantic-enhanced generative adversarial network (GAN)-based approach for abstractive text summarization task, called as: SGAN4AbSum. We use an adversarial training strategy for our text summarization model in which train the generator and discriminator to simultaneously handle the summary generation and distinguishing the generated summary with the ground-truth one. The input of generator is the jointed rich-semantic and global structural latent representations of training documents which are achieved by applying a combined BERT and graph convolutional network (GCN) textual embedding mechanism. Extensive experiments in benchmark datasets demonstrate the effectiveness of our proposed SGAN4AbSum which achieve the competitive ROUGE-based scores in comparing with state-of-the-art abstractive text summarization baselines.

Fast and Robust Multi-Person 3D Pose Estimation and Tracking from Multiple Views

2021 ◽  
pp. 1-1
Author(s):  
Junting Dong ◽  
Qi Fang ◽  
Wen Jiang ◽  
Yurou Yang ◽  
Qixing Huang ◽  
...  
Keyword(s):  
Pose Estimation ◽  
Multiple Views ◽  
3D Pose Estimation
Sign in / Sign up

Export Citation Format

Share Document