AbstractAberrant social behavior is a core feature of many neuropsychiatric disorders, yet the study of complex social behavior in freely moving rodents is relatively infrequently incorporated into preclinical models. This likely contributes to limited translational impact. A major bottleneck for the adoption of socially complex, ethology-rich, preclinical procedures are the technical limitations for consistently annotating detailed behavioral repertoires of rodent social behavior. Manual annotation is subjective, prone to observer drift, and extremely time-intensive. Commercial approaches are expensive and inferior to manual annotation. Open-source alternatives often require significant investments in specialized hardware and significant computational and programming knowledge. By combining recent computational advances in convolutional neural networks and pose-estimation with further machine learning analysis, complex rodent social behavior is primed for inclusion under the umbrella of computational neuroethology.Here we present an open-source package with graphical interface and workflow (Simple Behavioral Analysis, SimBA) that uses pose-estimation to create supervised machine learning predictive classifiers of rodent social behavior, with millisecond resolution and accuracies that can out-perform human observers. SimBA does not require specialized video acquisition hardware nor extensive computational background. Standard descriptive statistical analysis, along with graphical region of interest annotation, are provided in addition to predictive classifier generation. To increase ease-of-use for behavioural neuroscientists, we designed SimBA with accessible menus for pre-processing videos, annotating behavioural training datasets, selecting advanced machine learning options, robust classifier validation functions and flexible visualizations tools. This allows for predictive classifier transparency, explainability and tunability prior to, and during, experimental use. We demonstrate that this approach is flexible and robust in both mice and rats by classifying social behaviors that are commonly central to the study of brain function and social motivation. Finally, we provide a library of poseestimation weights and behavioral predictive classifiers for resident-intruder behaviors in mice and rats. All code and data, together with detailed tutorials and documentation, are available on the SimBA GitHub repository.Graphical abstractSimBA graphical interface (GUI) for creating supervised machine learning classifiers of rodent social behavior.(a) Pre-process videos. SimBA supports common video pre-processing functions (e.g., cropping, clipping, sampling, format conversion, etc.) that can be performed either on single videos, or as a batch.(b) Managing poseestimation data and creating classification projects. Pose-estimation tracking projects in DeepLabCut and DeepPoseKit can be either imported or created and managed within the SimBA graphical user interface, and the tracking results are imported into SimBA classification projects.SimBA also supports userdrawn region-of-interests (ROIs) for descriptive statistics of animal movements, or as features in machine learning classification projects.(c) Create classifiers, perform classifications, and analyze classification data. SimBA has graphical tools for correcting poseestimation tracking inaccuracies when multiple subjects are within a single frame, annotating behavioral events from videos, and optimizing machine learning hyperparameters and discrimination thresholds. A number of validation checkpoints and logs are included for increased classifier explainability and tunability prior to, and during, experimental use. Both detailed and summary data are provided at the end of classifier analysis. SimBA accepts behavioral annotations generated elsewhere (such as through JWatcher) that can be imported into SimBA classification projects.(d) Visualize classification results. SimBA has several options for visualizing machine learning classifications, animal movements and ROI data, and analyzing the durations and frequencies of classified behaviors.See the SimBA GitHub repository for a comprehensive documentation and user tutorials.
Human Behavior Prediction and Analysis Using Machine Learning-A Review
