scholarly journals Distilling Deep Structural Facial Relationships for FAU Intensity Estimation

2021 ◽  
Author(s):  
Yingruo Fan ◽  
Jacqueline CK Lam ◽  
Victor On Kwok Li
Keyword(s):  
Structural Characteristics ◽  
Model Performance ◽  
Training Model ◽  
Model Parameters ◽  
Intensity Estimation ◽  
Structural Relationships ◽  
Proposed Model ◽  
Comparable Performance ◽  
Knowledge Distillation ◽  
The Difference

<div> <div> <div> <p>Facial emotions are expressed through a combination of facial muscle movements, namely, the Facial Action Units (FAUs). FAU intensity estimation aims to estimate the intensity of a set of structurally dependent FAUs. Contrary to the existing works that focus on improving FAU intensity estimation, this study investigates how knowledge distillation (KD) incorporated into a training model can improve FAU intensity estimation efficiency while achieving the same level of performance. Given the intrinsic structural characteristics of FAU, it is desirable to distill deep structural relationships, namely, DSR-FAU, using heatmap regression. Our methodology is as follows: First, a feature map-level distillation loss was applied to ensure that the student network and the teacher network share similar feature distributions. Second, the region-wise and channel-wise relationship distillation loss functions were introduced to penalize the difference in structural relationships. Specifically, the region-wise relationship can be represented by the structural correlations across the facial features, whereas the channel-wise relationship is represented by the implicit FAU co-occurrence dependencies. Third, we compared the model performance of DSR-FAU with the state-of-the-art models, based on two benchmarking datasets. Our proposed model achieves comparable performance with other baseline models, though requiring a lower number of model parameters and lower computation complexities. </p> </div> </div> </div>

scholarly journals Distilling Deep Structural Facial Relationships for FAU Intensity Estimation

2021 ◽  
Author(s):  
Yingruo Fan ◽  
Jacqueline CK Lam ◽  
Victor On Kwok Li
Keyword(s):  
Structural Characteristics ◽  
Model Performance ◽  
Training Model ◽  
Model Parameters ◽  
Intensity Estimation ◽  
Structural Relationships ◽  
Proposed Model ◽  
Comparable Performance ◽  
Knowledge Distillation ◽  
The Difference

<div> <div> <div> <p>Facial emotions are expressed through a combination of facial muscle movements, namely, the Facial Action Units (FAUs). FAU intensity estimation aims to estimate the intensity of a set of structurally dependent FAUs. Contrary to the existing works that focus on improving FAU intensity estimation, this study investigates how knowledge distillation (KD) incorporated into a training model can improve FAU intensity estimation efficiency while achieving the same level of performance. Given the intrinsic structural characteristics of FAU, it is desirable to distill deep structural relationships, namely, DSR-FAU, using heatmap regression. Our methodology is as follows: First, a feature map-level distillation loss was applied to ensure that the student network and the teacher network share similar feature distributions. Second, the region-wise and channel-wise relationship distillation loss functions were introduced to penalize the difference in structural relationships. Specifically, the region-wise relationship can be represented by the structural correlations across the facial features, whereas the channel-wise relationship is represented by the implicit FAU co-occurrence dependencies. Third, we compared the model performance of DSR-FAU with the state-of-the-art models, based on two benchmarking datasets. Our proposed model achieves comparable performance with other baseline models, though requiring a lower number of model parameters and lower computation complexities. </p> </div> </div> </div>

scholarly journals An Updated Analytical Structural Pounding Force Model Based on Viscoelasticity of Materials

2016 ◽  
Vol 2016 ◽  
pp. 1-15 ◽  
Author(s):  
Qichao Xue ◽  
Chunwei Zhang ◽  
Jian He ◽  
Guangping Zou ◽  
Jingcai Zhang
Keyword(s):  
Vibration Control ◽  
Viscoelastic Model ◽  
Control Process ◽  
Model Performance ◽  
Analytical Models ◽  
Model Parameters ◽  
Force Model ◽  
Control Analysis ◽  
Proposed Model ◽  
Force Calculation

Based on the summary of existing pounding force analytical models, an updated pounding force analysis method is proposed by introducing viscoelastic constitutive model and contact mechanics method. Traditional Kelvin viscoelastic pounding force model can be expanded to 3-parameter linear viscoelastic model by separating classic pounding model parameters into geometry parameters and viscoelastic material parameters. Two existing pounding examples, the poundings of steel-to-steel and concrete-to-concrete, are recalculated by utilizing the proposed method. Afterwards, the calculation results are compared with other pounding force models. The results show certain accuracy in proposed model. The relative normalized errors of steel-to-steel and concrete-to-concrete experiments are 19.8% and 12.5%, respectively. Furthermore, a steel-to-polymer pounding example is calculated, and the application of the proposed method in vibration control analysis for pounding tuned mass damper (TMD) is simulated consequently. However, due to insufficient experiment details, the proposed model can only give a rough trend for both single pounding process and vibration control process. Regardless of the cheerful prospect, the study in this paper is only the first step of pounding force calculation. It still needs a more careful assessment of the model performance, especially in the presence of inelastic response.

scholarly journals Dynamics of Nonlocal Rod by Means of Fractional Laplacian

Symmetry ◽  
2020 ◽  
Vol 12 (12) ◽  
pp. 1933
Author(s):  
Vittorio Gusella ◽  
Giuseppina Autuori ◽  
Patrizia Pucci ◽  
Federico Cluni
Keyword(s):  
Fractional Laplacian ◽  
Free Vibrations ◽  
Dynamic Loads ◽  
Model Parameters ◽  
Local Response ◽  
Proposed Model ◽  
Fractional Models ◽  
Static And Dynamic Loads ◽  
The Difference ◽  
The Time Domain

The use of fractional models to analyse nonlocal behaviour of solids has acquired great importance in recent years. The aim of this paper is to propose a model that uses the fractional Laplacian in order to obtain the equation ruling the dynamics of nonlocal rods. The solution is found by means of numerical techniques with a discretisation in the space domain. At first, the proposed model is compared to a model that uses Eringen’s classical approach to derive the differential equation ruling the problem, showing how the parameters used in the proposed fractional model can be estimated. Moreover, the physical meaning of the model parameters is assessed. The model is then extended in dynamics by means of a discretisation in the time domain using Newmark’s method, and the responses to different dynamic conditions, such as an external load varying with time and free vibrations due to an initial deformation, are estimated, showing the difference of behaviour between the local response and the nonlocal response. The obtained results show that the proposed model can be used efficiently to estimate the response of the nonlocal rod both to static and dynamic loads.

scholarly journals Personalized Federated Learning with Semisupervised Distillation

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Xianxian Li ◽  
Yanxia Gong ◽  
Yuan Liang ◽  
Li-e Wang
Keyword(s):  
Model Performance ◽  
Personal Data ◽  
Training Model ◽  
Heterogeneous Data ◽  
Global Model ◽  
Model Parameters ◽  
Quality Model ◽  
Learning Framework ◽  
Class Distribution ◽  
Adaptive Aggregation

Heterogeneous data and models pose critical challenges for federated learning. However, the traditional federated learning framework, which trains the global model by transferring model parameters, has major limitations; it requires that all participants have the same training model architectures, and the trained global model does not guarantee accurate projections for participants’ personal data. To solve this problem, we propose a new federal framework named personalized federated learning with semisupervised distillation (pFedSD), which ensures the privacy of the participants’ model architectures and improves the communication efficiency by transmitting the model’s predicted class distribution rather than model parameters. First, the server adopts the adaptive aggregation method to reduce the weight of low-quality model predictions for the model’s predicted class distributions uploaded by all clients, which helps to improve the quality of the aggregation of the prediction class distribution. Then, the server sends it back to the clients for local training to obtain the personalized model. We finally conducted experiments on different datasets (MNIST, FMNIST, and CIFAR10), and the results show that the model performance of pFedSD exceeds the latest federated distillation algorithms.

scholarly journals Automatic Diabetic Retinopathy Grading via Self-Knowledge Distillation

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1337
Author(s):  
Ling Luo ◽  
Dingyu Xue ◽  
Xinglong Feng
Keyword(s):  
Diabetic Retinopathy ◽  
Deep Learning ◽  
Large Scale ◽  
Model Performance ◽  
Side Branch ◽  
Superior Performance ◽  
Model Parameters ◽  
Hard Exudates ◽  
Knowledge Distillation ◽  
Self Knowledge

Diabetic retinopathy (DR) is a common fundus disease that leads to irreversible blindness, which plagues the working-age population. Automatic medical imaging diagnosis provides a non-invasive method to assist ophthalmologists in timely screening of suspected DR cases, which prevents its further deterioration. However, the state-of-the-art deep-learning-based methods generally have a large amount of model parameters, which makes large-scale clinical deployment a time-consuming task. Moreover, the severity of DR is associated with lesions, and it is difficult for the model to focus on these regions. In this paper, we propose a novel deep-learning technique for grading DR with only image-level supervision. Specifically, we first customize the model with the help of self-knowledge distillation to achieve a trade-off between model performance and time complexity. Secondly, CAM-Attention is used to allow the network to focus on discriminative zone, e.g., microaneurysms, soft/hard exudates, etc.. Considering that directly attaching a classifier after the Side branch will disrupt the hierarchical nature of convolutional neural networks, a Mimicking Module is employed that allows the Side branch to actively mimic the main branch structure. Extensive experiments are conducted on two benchmark datasets, with an AUC of 0.965 and an accuracy of 92.9% for the Messidor dataset and 67.96% accuracy achieved for the challenging IDRID dataset, which demonstrates the superior performance of our proposed method.

scholarly journals Modeling sleep onset misperception in insomnia

SLEEP ◽  
2020 ◽  
Vol 43 (8) ◽  
Author(s):  
Lieke W A Hermans ◽  
Merel M van Gilst ◽  
Marta Regis ◽  
Leonie C E van den Heuvel ◽  
Hanneke Langen ◽  
...  
Keyword(s):  
Sleep Onset ◽  
Model Performance ◽  
Minimum Length ◽  
Model Parameters ◽  
Sleep Onset Latency ◽  
Sleep State ◽  
Data Set ◽  
Sleep Diaries ◽  
The Difference ◽  
Mean Square Errors

Abstract Objectives To extend and validate a previously suggested model of the influence of uninterrupted sleep bouts on sleep onset misperception in a large independent data set. Methods Polysomnograms and sleep diaries of 139 insomnia patients and 92 controls were included. We modeled subjective sleep onset as the start of the first uninterrupted sleep fragment longer than Ls minutes, where parameter Ls reflects the minimum length of a sleep fragment required to be perceived as sleep. We compared the so-defined sleep onset latency (SOL) for various values of Ls. Model parameters were compared between groups, and across insomnia subgroups with respect to sleep onset misperception, medication use, age, and sex. Next, we extended the model to incorporate the length of wake fragments. Model performance was assessed by calculating root mean square errors (RMSEs) of the difference between estimated and perceived SOL. Results Participants with insomnia needed a median of 34 minutes of undisturbed sleep to perceive sleep onset, while healthy controls needed 22 minutes (Mann–Whitney U = 4426, p &lt; 0.001). Similar statistically significant differences were found between sleep onset misperceivers and non-misperceivers (median 40 vs. 20 minutes, Mann–Whitney U = 984.5, p &lt; 0.001). Model outcomes were similar across other subgroups. Extended models including wake bout lengths resulted in only marginal improvements of model outcome. Conclusions Patients with insomnia, particularly sleep misperceivers, need larger continuous sleep bouts to perceive sleep onset. The modeling approach yields a parameter for which we coin the term Sleep Fragment Perception Index, providing a useful measure to further characterize sleep state misperception.

scholarly journals Modelling of simple shear tests on volcanic unsaturated sands

2020 ◽  
Vol 195 ◽  
pp. 02021
Author(s):  
Mariagiovanna Moscariello ◽  
Yanni Chen ◽  
Sabatino Cuomo ◽  
Giuseppe Buscarnera
Keyword(s):  
Simple Shear ◽  
Unsaturated Soils ◽  
Model Performance ◽  
Frictional Resistance ◽  
Flow Rule ◽  
Model Parameters ◽  
Shear Tests ◽  
Soil Response ◽  
Proposed Model ◽  
Unsaturated Sands

In landslide susceptibility analysis, a relevant issue is the proper modelling of the complex mechanisms that regulate the failure and post-failure stages. In this paper, simple shear experiments replicating the kinematics of failure in landslide-prone areas are interpreted through an elastoplastic strain-hardening constitutive model for both saturated and unsaturated soils. The material tested is an air-fall volcanic (pyroclastic) soil from Southern Italy which originated from the explosive activity of the Somma-Vesuvius volcanic apparatus. Data from triaxial and shear tests performed on remoulded specimens characterized by saturated and unsaturated conditions are used to calibrate the model parameters. The evolution of shear stress, volumetric and shear strain measured during the experiments are reproduced by means of a model formulation specific for simple shear conditions. To capture the strength emerging under different states of saturation, non-associated flow rule, and a suction-dependent yield surface are used. Examination of the experimental data available for various testing conditions enabled the quantification of the variability of fundamental model constants, such as those controlling frictional resistance and water retention behaviour. To account for such scatter in the physical properties, the constitutive analyses are performed by employing varying model constants within a band of admissible values. The resulting model performance is validated by comparing the simulations with the experimental results at different saturation conditions. The results show that the combination of the proposed model with a data-driven determination of the range of variation of hydro-mechanical properties is crucial to satisfactorily simulate the essential features of the soil response under a variety of simple shear testing regimes.

scholarly journals A Car-Following Model Based on Quantified Homeostatic Risk Perception

2013 ◽  
Vol 2013 ◽  
pp. 1-13 ◽  
Author(s):  
Guangquan Lu ◽  
Bo Cheng ◽  
Yunpeng Wang ◽  
Qingfeng Lin
Keyword(s):  
Perceived Risk ◽  
Safety Margin ◽  
Model Parameters ◽  
Perceived Safety ◽  
Car Following ◽  
Stimulus Response ◽  
Proposed Model ◽  
Car Following Model ◽  
The Difference ◽  
Level Of Risk

This study attempts to elucidate individual car-following behavior using risk homeostasis theory (RHT). On the basis of this theory and the stimulus-response concept, we develop a desired safety margin (DSM) model. Safety margin, defined as the level of perceived risk in car-following processes, is proposed and considered to be a stimulus parameter. Acceleration is assessed in accordance with the difference between the perceived safety margin (perceived level of risk) and desired safety margin (acceptable level of risk) of a driver in a car-following situation. Sixty-three cases selected from Next Generation Simulation (NGSIM) are used to calibrate the parameters of the proposed model for general car-following behavior. Other eight cases with two following cars taken from NGSIM are used to validate the model. A car-following case with stop-and-go processes is also used to demonstrate the performance of the proposed model. The simulation results are then compared with the calculations derived using the Gazis-Herman-Rothery (GHR) model. As a result, the DSM and GHR models yield similar results and the proposed model is effective for simulation of car following. By adjusting model parameters, the proposed model can simulate different driving behaviors. The proposed model gives a new way to explain car-following process by RHT.

scholarly journals Fuzzy Modeling of Urban Water Supply Crisis

2021 ◽  
Author(s):  
Welitom Ttatom Pereira da Silva ◽  
Marco Antonio Almeida de Souza
Keyword(s):  
Water Supply ◽  
Conceptual Model ◽  
Influencing Factors ◽  
Water Consumption ◽  
Model Performance ◽  
Real Data ◽  
Model Parameters ◽  
Urban Water ◽  
Urban Water Supply ◽  
Proposed Model

The chaotic growth of cities results in numerous problems related to public health and urban environment. One of these problems is the urban water supply system crisis. This research aims to develop a mathematical model for urban water supply crises (UWC) able to deal with the ambiguity of the real available data. The applied methodology comprises the following steps: (i) identifying the influencing factors in UWC; (ii) proposing a conceptual model for the description of UWC; (iii) collecting and simulating the necessary and available data; (iv) optimizing the conceptual model parameters; and (v) verifying the proposed model performance. The results indicate that there are many influencing factors in UWC. The model developed comprises two parts or two sub-models. The first sub-model explains water consumption, and the second sub-model explains water availability. In the first sub-model, the functions are related to the factors that influence water consumption. In the second sub-model, the functions are related to the factors that influence the availability of water. This research also aims to analyze the possibility of applying Fuzzy Logic to deal with the ambiguity of real data. It was concluded that, with the proposed model, the UWC was modeled appropriately. The model proposed can help to predict the impact of actions such as reducing losses, reducing pressure on the water supply network and intermittent supply on the intensity of water crisis cases in cities.

scholarly journals Automatic Model Calibration: A New Way to Improve Numerical Weather Forecasting

2017 ◽  
Vol 98 (5) ◽  
pp. 959-970 ◽  
Author(s):  
Q. Duan ◽  
Z. Di ◽  
J. Quan ◽  
C. Wang ◽  
W. Gong ◽  
...  
Keyword(s):  
Model Calibration ◽  
Weather Forecasting ◽  
Weather Prediction ◽  
Model Performance ◽  
Model Parameters ◽  
Mathematical Methods ◽  
Numerical Weather ◽  
Calibration Methods ◽  
The Difference ◽  
Nwp Model

Abstract Weather forecasting skill has been improved over recent years owing to advances in the representation of physical processes by numerical weather prediction (NWP) models, observational systems, data assimilation and postprocessing, new computational capability, and effective communications and training. There is an area that has received less attention so far but can bring significant improvement to weather forecasting—the calibration of NWP models, a process in which model parameters are tuned using certain mathematical methods to minimize the difference between predictions and observations. Model calibration of the NWP models is difficult because 1) there are a formidable number of model parameters and meteorological variables to tune, and 2) a typical NWP model is very expensive to run, and conventional model calibration methods require many model runs (up to tens of thousands) or cannot handle the high dimensionality of NWP models. This study demonstrates that a newly developed automatic model calibration platform can overcome these difficulties and improve weather forecasting through parameter optimization. We illustrate how this is done with a case study involving 5-day weather forecasting during the summer monsoon in the greater Beijing region using the Weather Research and Forecasting Model. The keys to automatic model calibration are to use global sensitivity analysis to screen out the most important parameters influencing model performance and to employ surrogate models to reduce the need for a large number of model runs. Through several optimization and validation studies, we have shown that automatic model calibration can improve precipitation and temperature forecasting significantly according to a number of performance measures.

Sign in / Sign up

Export Citation Format

Share Document