scholarly journals Differentiated Distribution Recovery for Neural Text Generation

2019 ◽  
Vol 33 ◽  
pp. 6682-6689
Author(s):  
Jianing Li ◽  
Yanyan Lan ◽  
Jiafeng Guo ◽  
Jun Xu ◽  
Xueqi Cheng
Keyword(s):  
Synthetic Data ◽  
Likelihood Estimation ◽  
Parameter Tuning ◽  
Turing Test ◽  
Language Models ◽  
Pass Rate ◽  
Text Generation ◽  
True Probability ◽  
Adversarial Training

Neural language models based on recurrent neural networks (RNNLM) have significantly improved the performance for text generation, yet the quality of generated text represented by Turing Test pass rate is still far from satisfying. Some researchers propose to use adversarial training or reinforcement learning to promote the quality, however, such methods usually introduce great challenges in the training and parameter tuning processes. Through our analysis, we find the problem of RNNLM comes from the usage of maximum likelihood estimation (MLE) as the objective function, which requires the generated distribution to precisely recover the true distribution. Such requirement favors high generation diversity which restricted the generation quality. This is not suitable when the overall quality is low, since high generation diversity usually indicates lot of errors rather than diverse good samples. In this paper, we propose to achieve differentiated distribution recovery, DDR for short. The key idea is to make the optimal generation probability proportional to the β-th power of the true probability, where β > 1. In this way, the generation quality can be greatly improved by sacrificing diversity from noises and rare patterns. Experiments on synthetic data and two public text datasets show that our DDR method achieves more flexible quality-diversity trade-off and higher Turing Test pass rate, as compared with baseline methods including RNNLM, SeqGAN and LeakGAN.

scholarly journals Generating Synthetic Training Data for Supervised De-Identification of Electronic Health Records

2021 ◽  
Vol 13 (5) ◽  
pp. 136
Author(s):  
Claudia Alessandra Libbi ◽  
Jan Trienes ◽  
Dolf Trieschnigg ◽  
Christin Seifert
Keyword(s):  
Electronic Health Records ◽  
Named Entity Recognition ◽  
Synthetic Data ◽  
Real Data ◽  
Entity Recognition ◽  
Language Models ◽  
Text Generation ◽  
Health Records ◽  
Named Entity ◽  
Electronic Health

A major hurdle in the development of natural language processing (NLP) methods for Electronic Health Records (EHRs) is the lack of large, annotated datasets. Privacy concerns prevent the distribution of EHRs, and the annotation of data is known to be costly and cumbersome. Synthetic data presents a promising solution to the privacy concern, if synthetic data has comparable utility to real data and if it preserves the privacy of patients. However, the generation of synthetic text alone is not useful for NLP because of the lack of annotations. In this work, we propose the use of neural language models (LSTM and GPT-2) for generating artificial EHR text jointly with annotations for named-entity recognition. Our experiments show that artificial documents can be used to train a supervised named-entity recognition model for de-identification, which outperforms a state-of-the-art rule-based baseline. Moreover, we show that combining real data with synthetic data improves the recall of the method, without manual annotation effort. We conduct a user study to gain insights on the privacy of artificial text. We highlight privacy risks associated with language models to inform future research on privacy-preserving automated text generation and metrics for evaluating privacy-preservation during text generation.

Simple Index to Assess the Calibration Quality of Safety Performance Functions Based on Multiple Goodness-of-Fit Metrics

2021 ◽  
pp. 036119812110088
Author(s):  
Raul E. Avelar ◽  
Karen Dixon ◽  
Boniphace Kutela ◽  
Sam Klump ◽  
Beth Wemple ◽  
...  
Keyword(s):  
Goodness Of Fit ◽  
Synthetic Data ◽  
Calibration Procedure ◽  
Safety Performance ◽  
Absolute Deviation ◽  
Data Set ◽  
Safety Database ◽  
Simple Index ◽  
Safety Performance Functions

The calibration of safety performance functions (SPFs) is a mechanism included in the Highway Safety Manual (HSM) to adjust SPFs in the HSM for use in intended jurisdictions. Critically, the quality of the calibration procedure must be assessed before using the calibrated SPFs. Multiple resources to aid practitioners in calibrating SPFs have been developed in the years following the publication of the HSM 1st edition. Similarly, the literature suggests multiple ways to assess the goodness-of-fit (GOF) of a calibrated SPF to a data set from a given jurisdiction. This paper uses the calibration results of multiple intersection SPFs to a large Mississippi safety database to examine the relations between multiple GOF metrics. The goal is to develop a sensible single index that leverages the joint information from multiple GOF metrics to assess overall quality of calibration. A factor analysis applied to the calibration results revealed three underlying factors explaining 76% of the variability in the data. From these results, the authors developed an index and performed a sensitivity analysis. The key metrics were found to be, in descending order: the deviation of the cumulative residual (CURE) plot from the 95% confidence area, the mean absolute deviation, the modified R-squared, and the value of the calibration factor. This paper also presents comparisons between the index and alternative scoring strategies, as well as an effort to verify the results using synthetic data. The developed index is recommended to comprehensively assess the quality of the calibrated intersection SPFs.

scholarly journals Quality of Automated Stereotactic Radiosurgery Plans in Patients with 4 to 10 Brain Metastases

Cancers ◽  
2021 ◽  
Vol 13 (14) ◽  
pp. 3458
Author(s):  
Anna Petoukhova ◽  
Roland Snijder ◽  
Rudolf Wiggenraad ◽  
Linda de Boer-de Wit ◽  
Ivonne Mudde-van der Wouden ◽  
...  
Keyword(s):  
Brain Metastases ◽  
Stereotactic Radiosurgery ◽  
Irradiation Time ◽  
Intensity Modulated Radiotherapy ◽  
Pass Rate ◽  
Conformity Index ◽  
Index Gradient ◽  
Single Isocenter ◽  
Volumetric Modulated Arc Radiotherapy

The purpose was to compare linac-based stereotactic radiosurgery and hypofractionated radiotherapy plan quality of automated planning, intensity modulated radiotherapy (IMRT) and manual dynamic conformal arc (DCA) plans as well as single- and multiple-isocenter techniques for multiple brain metastases (BM). For twelve patients with four to ten BM, seven non-coplanar linac-based plans were created: a manually planned DCA plan with a separate isocenter for each metastasis, a single-isocenter dynamic IMRT plan, an automatically generated single-isocenter volumetric modulated arc radiotherapy (VMAT) plan, four automatically generated single-isocenter DCA plans with three or five couch angles, with high or low sparing of normal tissue. Paddick conformity index, gradient index (GI), mean dose, total V12Gy and V5Gy of uninvolved brain, number of monitor units (MUs), irradiation time and pass rate were compared. The GI was significantly higher for VMAT than for separate-isocenter, IMRT, and all automatically generated plans. The number of MUs was lowest for VMAT, followed by automatically generated DCA and IMRT plans and highest for manual DCA plans. Irradiation time was the shortest for automatically planned DCA plans. Automatically generated linac-based single-isocenter plans for multiple BM reduce the number of MUs and irradiation time with at least comparable GI and V5Gy relative to the reference separate-isocenter DCA plans.

scholarly journals Identifying Stabilising Effects on Survey Based Life Satisfaction Using Quasi-maximum Likelihood Estimation

2021 ◽  
Author(s):  
Johannes Klement
Keyword(s):  
Life Satisfaction ◽  
Maximum Likelihood ◽  
Maximum Likelihood Estimation ◽  
Negative Binomial ◽  
Measurement Instrument ◽  
Likelihood Estimation ◽  
Satisfaction Scale ◽  
Quasi Maximum Likelihood ◽  
The Stability

AbstractTo which extent do happiness correlates contribute to the stability of life satisfaction? Which method is appropriate to provide a conclusive answer to this question? Based on life satisfaction data of the German SOEP, we show that by Negative Binomial quasi-maximum likelihood estimation statements can be made as to how far correlates of happiness contribute to the stabilisation of life satisfaction. The results show that happiness correlates which are generally associated with a positive change in life satisfaction, also stabilise life satisfaction and destabilise dissatisfaction with life. In such as they lower the probability of leaving positive states of life satisfaction and increase the probability of leaving dissatisfied states. This in particular applies to regular exercise, volunteering and living in a marriage. We further conclude that both patterns in response behaviour and the quality of the measurement instrument, the life satisfaction scale, have a significant effect on the variation and stability of reported life satisfaction.

scholarly journals Benchmarking Unsupervised Outlier Detection with Realistic Synthetic Data

2021 ◽  
Vol 15 (4) ◽  
pp. 1-20
Author(s):  
Georg Steinbuss ◽  
Klemens Böhm
Keyword(s):  
Outlier Detection ◽  
Synthetic Data ◽  
Real Data ◽  
Detection Methods ◽  
Quality Of Data ◽  
Benchmark Data ◽  
Core Idea ◽  
Generic Process ◽  
Unsupervised Outlier Detection

Benchmarking unsupervised outlier detection is difficult. Outliers are rare, and existing benchmark data contains outliers with various and unknown characteristics. Fully synthetic data usually consists of outliers and regular instances with clear characteristics and thus allows for a more meaningful evaluation of detection methods in principle. Nonetheless, there have only been few attempts to include synthetic data in benchmarks for outlier detection. This might be due to the imprecise notion of outliers or to the difficulty to arrive at a good coverage of different domains with synthetic data. In this work, we propose a generic process for the generation of datasets for such benchmarking. The core idea is to reconstruct regular instances from existing real-world benchmark data while generating outliers so that they exhibit insightful characteristics. We propose and describe a generic process for the benchmarking of unsupervised outlier detection, as sketched so far. We then describe three instantiations of this generic process that generate outliers with specific characteristics, like local outliers. To validate our process, we perform a benchmark with state-of-the-art detection methods and carry out experiments to study the quality of data reconstructed in this way. Next to showcasing the workflow, this confirms the usefulness of our proposed process. In particular, our process yields regular instances close to the ones from real data. Summing up, we propose and validate a new and practical process for the benchmarking of unsupervised outlier detection.

scholarly journals Detraining Slows and Maintenance Training Over 6 Years Halts Parkinsonian Symptoms-Progression

2021 ◽  
Vol 12 ◽  
Author(s):  
Tibor Hortobágyi ◽  
Dávid Sipos ◽  
Gábor Borbély ◽  
György Áfra ◽  
Emese Reichardt-Varga ◽  
...  
Keyword(s):  
Quality Of Life ◽  
High Intensity ◽  
Exercise Program ◽  
Likelihood Estimation ◽  
Drug Dose ◽  
Mixed Effects Modeling ◽  
Maintenance Program ◽  
Before And After ◽  
Exercise Induced

Introduction: There are scant data to demonstrate that the long-term non-pharmaceutical interventions can slow the progression of motor and non-motor symptoms and lower drug dose in Parkinson's disease (PD).Methods: After randomization, the Exercise-only (E, n = 19) group completed an initial 3-week-long, 15-session supervised, high-intensity sensorimotor agility exercise program designed to improve the postural stability. The Exercise + Maintenance (E + M, n = 22) group completed the 3-week program and continued the same program three times per week for 6 years. The no exercise and no maintenance control (C, n = 26) group continued habitual living. In each patient, 11 outcomes were measured before and after the 3-week initial exercise program and then, at 3, 6, 12, 18, 24, 36, 48, 60, and 72 months.Results: The longitudinal linear mixed effects modeling of each variable was fitted with maximum likelihood estimation and adjusted for baseline and covariates. The exercise program strongly improved the primary outcome, Motor Experiences of Daily Living, by ~7 points and all secondary outcomes [body mass index (BMI), disease and no disease-specific quality of life, depression, mobility, and standing balance]. In E group, the detraining effects lasted up to 12 months. E+M group further improved the initial exercise-induced gains up to 3 months and the gains were sustained until year 6. In C group, the symptoms worsened steadily. By year 6, levodopa (L-dopa) equivalents increased in all the groups but least in E + M group.Conclusion: A short-term, high-intensity sensorimotor agility exercise program improved the PD symptoms up to a year during detraining but the subsequent 6-year maintenance program was needed to further increase or sustain the initial improvements in the symptoms, quality of life, and drug dose.

scholarly journals Importance-Aware Learning for Neural Headline Editing

2020 ◽  
Vol 34 (05) ◽  
pp. 9282-9289
Author(s):  
Qingyang Wu ◽  
Lei Li ◽  
Hao Zhou ◽  
Ying Zeng ◽  
Zhou Yu
Keyword(s):  
Social Media ◽  
Large Scale ◽  
Network Models ◽  
Language Models ◽  
Neural Network Models ◽  
Generation Task ◽  
Social Media Platforms ◽  
Editing Process ◽  
Different Levels

Many social media news writers are not professionally trained. Therefore, social media platforms have to hire professional editors to adjust amateur headlines to attract more readers. We propose to automate this headline editing process through neural network models to provide more immediate writing support for these social media news writers. To train such a neural headline editing model, we collected a dataset which contains articles with original headlines and professionally edited headlines. However, it is expensive to collect a large number of professionally edited headlines. To solve this low-resource problem, we design an encoder-decoder model which leverages large scale pre-trained language models. We further improve the pre-trained model's quality by introducing a headline generation task as an intermediate task before the headline editing task. Also, we propose Self Importance-Aware (SIA) loss to address the different levels of editing in the dataset by down-weighting the importance of easily classified tokens and sentences. With the help of Pre-training, Adaptation, and SIA, the model learns to generate headlines in the professional editor's style. Experimental results show that our method significantly improves the quality of headline editing comparing against previous methods.

scholarly journals RobOMP: Robust variants of Orthogonal Matching Pursuit for sparse representations

2019 ◽  
Author(s):  
Carlos A Loza
Keyword(s):  
Mean Squared Error ◽  
Matching Pursuit ◽  
Synthetic Data ◽  
Optimal Solution ◽  
Parameter Tuning ◽  
Weight Vector ◽  
Orthogonal Matching Pursuit ◽  
Main Mode ◽  
Observation Matrix ◽  
And Performance

Sparse coding aims to find a parsimonious representation of an example given an observation matrix or dictionary. In this regard, Orthogonal Matching Pursuit (OMP) provides an intuitive, simple and fast approximation of the optimal solution. However, its main building block is anchored on the minimization of the Mean Squared Error cost function (MSE). This approach is only optimal if the errors are distributed according to a Gaussian distribution without samples that strongly deviate from the main mode, i.e. outliers. If such assumption is violated, the sparse code will likely be biased and performance will degrade accordingly. In this paper, we introduce five robust variants of OMP (RobOMP) fully based on the theory of M-Estimators under a linear model. The proposed framework exploits efficient Iteratively Reweighted Least Squares (IRLS) techniques to mitigate the effect of outliers and emphasize the samples corresponding to the main mode of the data. This is done adaptively via a learned weight vector that models the distribution of the data in a robust manner. Experiments on synthetic data under several noise distributions and image recognition under different combinations of occlusion and missing pixels thoroughly detail the superiority of RobOMP over MSE-based approaches and similar robust alternatives. We also introduce a denoising framework based on robust, sparse and redundant representations that open the door to potential further applications of the proposed techniques. The five different variants of RobOMP do not require parameter tuning from the user and, hence, constitute principled alternatives to OMP.

scholarly journals Joint optimization on decoding graphs using minimum classification error criterion

2014 ◽  
Vol 3 ◽  
Author(s):  
Abdelaziz A. Abdelhamid ◽  
Waleed H. Abdulla
Keyword(s):  
Likelihood Estimation ◽  
Discriminative Training ◽  
Language Models ◽  
Classification Error ◽  
Error Criterion ◽  
Speech Corpora ◽  
Finite State ◽  
Minimum Classification Error ◽  
Speech Features ◽  
Weighted Finite State Transducers

Motivated by the inherent correlation between the speech features and their lexical words, we propose in this paper a new framework for learning the parameters of the corresponding acoustic and language models jointly. The proposed framework is based on discriminative training of the models' parameters using minimum classification error criterion. To verify the effectiveness of the proposed framework, a set of four large decoding graphs is constructed using weighted finite-state transducers as a composition of two sets of context-dependent acoustic models and two sets of n-gram-based language models. The experimental results conducted on this set of decoding graphs validated the effectiveness of the proposed framework when compared with four baseline systems based on maximum likelihood estimation and separate discriminative training of acoustic and language models in benchmark testing of two speech corpora, namely TIMIT and RM1.

Sign in / Sign up

Export Citation Format

Share Document