Spatio-temporal Pain Recognition in CNN-Based Super-Resolved Facial Images

Non-contact Pain Recognition from Video Sequences with Remote Physiological Measurements Prediction

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/170 ◽

2021 ◽

Author(s):

Ruijing Yang ◽

Ziyu Guan ◽

Zitong Yu ◽

Xiaoyi Feng ◽

Jinye Peng ◽

...

Keyword(s):

Long Range ◽

Medical Diagnosis ◽

State Of The Art ◽

Video Sequences ◽

Learning Framework ◽

Remote Photoplethysmography ◽

Spatio Temporal ◽

Pain Recognition ◽

Appearance Changes ◽

Over Time

Automatic pain recognition is paramount for medical diagnosis and treatment. The existing works fall into three categories: assessing facial appearance changes, exploiting physiological cues, or fusing them in a multi-modal manner. However, (1) appearance changes are easily affected by subjective factors which impedes objective pain recognition. Besides, the appearance-based approaches ignore long-range spatial-temporal dependencies that are important for modeling expressions over time; (2) the physiological cues are obtained by attaching sensors on human body, which is inconvenient and uncomfortable. In this paper, we present a novel multi-task learning framework which encodes both appearance changes and physiological cues in a non-contact manner for pain recognition. The framework is able to capture both local and long-range dependencies via the proposed attention mechanism for the learned appearance representations, which are further enriched by temporally attended physiological cues (remote photoplethysmography, rPPG) that are recovered from videos in the auxiliary task. This framework is dubbed rPPG-enriched Spatio-Temporal Attention Network (rSTAN) and allows us to establish the state-of-the-art performance of non-contact pain recognition on publicly available pain databases. It demonstrates that rPPG predictions can be used as an auxiliary task to facilitate non-contact automatic pain recognition.

Download Full-text

Deep Multimodal Pain Recognition: A Database and Comparison of Spatio-Temporal Visual Modalities

2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) ◽

10.1109/fg.2018.00044 ◽

2018 ◽

Cited By ~ 8

Author(s):

Mohammad A. Haque ◽

Ruben B. Bautista ◽

Fatemeh Noroozi ◽

Kaustubh Kulkarni ◽

Christian B. Laursen ◽

...

Keyword(s):

Spatio Temporal ◽

Pain Recognition

Download Full-text

E3 ubiquitin ligases

Essays in Biochemistry ◽

10.1042/bse0410015 ◽

2005 ◽

Vol 41 ◽

pp. 15-30 ◽

Cited By ~ 123

Author(s):

Helen C. Ardley ◽

Philip A. Robinson

Keyword(s):

Protein Complexes ◽

Loss Of Function ◽

Direct Role ◽

Cellular Processes ◽

Substrate Protein ◽

Protein Ubiquitination ◽

C Terminus ◽

Full Complement ◽

Spatio Temporal ◽

Eukaryotic Organisms

The selectivity of the ubiquitin–26 S proteasome system (UPS) for a particular substrate protein relies on the interaction between a ubiquitin-conjugating enzyme (E2, of which a cell contains relatively few) and a ubiquitin–protein ligase (E3, of which there are possibly hundreds). Post-translational modifications of the protein substrate, such as phosphorylation or hydroxylation, are often required prior to its selection. In this way, the precise spatio-temporal targeting and degradation of a given substrate can be achieved. The E3s are a large, diverse group of proteins, characterized by one of several defining motifs. These include a HECT (homologous to E6-associated protein C-terminus), RING (really interesting new gene) or U-box (a modified RING motif without the full complement of Zn2+-binding ligands) domain. Whereas HECT E3s have a direct role in catalysis during ubiquitination, RING and U-box E3s facilitate protein ubiquitination. These latter two E3 types act as adaptor-like molecules. They bring an E2 and a substrate into sufficiently close proximity to promote the substrate's ubiquitination. Although many RING-type E3s, such as MDM2 (murine double minute clone 2 oncoprotein) and c-Cbl, can apparently act alone, others are found as components of much larger multi-protein complexes, such as the anaphase-promoting complex. Taken together, these multifaceted properties and interactions enable E3s to provide a powerful, and specific, mechanism for protein clearance within all cells of eukaryotic organisms. The importance of E3s is highlighted by the number of normal cellular processes they regulate, and the number of diseases associated with their loss of function or inappropriate targeting.

Download Full-text

Elucidating cyclic AMP signaling in subcellular domains with optogenetic tools and fluorescent biosensors

Biochemical Society Transactions ◽

10.1042/bst20190246 ◽

2019 ◽

Vol 47 (6) ◽

pp. 1733-1747 ◽

Cited By ~ 3

Author(s):

Christina Klausen ◽

Fabian Kaiser ◽

Birthe Stüven ◽

Jan N. Hansen ◽

Dagmar Wachten

Keyword(s):

Cyclic Amp ◽

Adenosine Monophosphate ◽

Adenylyl Cyclases ◽

Temporal Precision ◽

Fluorescent Biosensors ◽

Subcellular Domains ◽

Spatio Temporal ◽

Hydrolysis Of ◽

Shed Light ◽

Cyclic Amp Signaling

The second messenger 3′,5′-cyclic nucleoside adenosine monophosphate (cAMP) plays a key role in signal transduction across prokaryotes and eukaryotes. Cyclic AMP signaling is compartmentalized into microdomains to fulfil specific functions. To define the function of cAMP within these microdomains, signaling needs to be analyzed with spatio-temporal precision. To this end, optogenetic approaches and genetically encoded fluorescent biosensors are particularly well suited. Synthesis and hydrolysis of cAMP can be directly manipulated by photoactivated adenylyl cyclases (PACs) and light-regulated phosphodiesterases (PDEs), respectively. In addition, many biosensors have been designed to spatially and temporarily resolve cAMP dynamics in the cell. This review provides an overview about optogenetic tools and biosensors to shed light on the subcellular organization of cAMP signaling.

Download Full-text