scholarly journals What Can a Generative Language Model Answer About a Passage?

2021 ◽  
Author(s):  
Douglas Summers-Stay ◽  
Claire Bonial ◽  
Clare Voss
Author(s):  
Kelvin Guu ◽  
Tatsunori B. Hashimoto ◽  
Yonatan Oren ◽  
Percy Liang

We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.


2020 ◽  
Vol 389 ◽  
pp. 93-107
Author(s):  
Jinmeng Wu ◽  
Tingting Mu ◽  
Jeyarajan Thiyagalingam ◽  
John Y. Goulermas

2021 ◽  
pp. 1-14
Author(s):  
Ethan Porter ◽  
Yamil R. Velez

Abstract Although placebo conditions are ubiquitous in survey experiments, little evidence guides common practices for their use and selection. How should scholars choose and construct placebos? First, we review the role of placebos in published survey experiments, finding that placebos are used inconsistently. Then, drawing on the medical literature, we clarify the role that placebos play in accounting for nonspecific effects (NSEs), or the effects of ancillary features of experiments. We argue that, in the absence of precise knowledge of NSEs that placebos are adjusting for, researchers should average over a corpus of many placebos. We demonstrate this agnostic approach to placebo construction through the use of GPT-2, a generative language model trained on a database of over 1 million internet news pages. Using GPT-2, we devise 5,000 distinct placebos and administer two experiments (N = 2,975). Our results illustrate how researchers can minimize their role in placebo selection through automated processes. We conclude by offering tools for incorporating computer-generated placebo text vignettes into survey experiments and developing recommendations for best practice.


2021 ◽  
Author(s):  
Richard W. Shuai ◽  
Jeffrey A. Ruffolo ◽  
Jeffrey J. Gray

Successful development of monoclonal antibodies (mAbs) for therapeutic applications is hindered by developability issues such as low solubility, low thermal stability, high aggregation, and high immunogenicity. The discovery of more developable mAb candidates relies on high-quality antibody libraries for isolating candidates with desirable properties. We present Immunoglobulin Language Model (IgLM), a deep generative language model for generating synthetic libraries by re-designing variable-length spans of antibody sequences. IgLM formulates antibody design as an autoregressive sequence generation task based on text-infilling in natural language. We trained IgLM on approximately 558M antibody heavy- and light-chain variable sequences, conditioning on each sequence's chain type and species-of-origin. We demonstrate that IgLM can be applied to generate synthetic libraries that may accelerate the discovery of therapeutic antibody candidates


2008 ◽  
Vol 19 (9) ◽  
pp. 2449-2460 ◽  
Author(s):  
Mei WANG ◽  
Xiang-Dong ZHOU ◽  
Jun-Qi ZHANG ◽  
Hong-Tao XU ◽  
Bai-Le SHI

Author(s):  
Larisa V. Kalashnikova

The article enlightens the probem of nonsense and its role in the development of creative thinking and fantasy, and the way how the interpretation of nonsense affects children imagination. The function of imagination inherent to a person, and especially to a child, has a powerful potential – to create artificially new metaphorical models, absurd and most incredible situations based on self-amazement. Children are able to measure the properties of unfamiliar objects with the properties of known things. It is not difficult for small researchers to replace incomprehensible meanings with familiar ones; to think over situations, to make analogies, to transfer signs and properties of one object to another. The problem of nonsense research is interesting and relevant. The element of the game is an integral component of nonsense. In the process of playing, children cognize the world, learn to interact with the world, imitating the adults behavior. Imagination and fantasy help the child to invent his own rules of the game, to choose language elements that best suit his ideas. The child uses the learned productive models of the language system to create their own models and their own language, attracting language signs: words, morphs, sentences. Children’s dictionary stimulates word formation and language nomination processes. Nonsense-words are the result of children’s dictionary, speech errors and occazional formations, presented in the form of contamination, phonetic transformations, lexical substitution, implemented on certain models. The first two models are phonetic imitation and hybrid speech, based on the natural language model. The third model of designing nonsense is represented by words that have no meaning at all and can be attributed to words-portmonaie. Due to the flexibility of interframe relationships and the lack of algorithmic thinking, children can not only capture the implicit similarity of objects and phenomena, but also create it through their imagination. Interpretation of nonsense is an effective method of developing imagination in children, because metaphors, nonsense as a means of creating new meanings, modeling new content from fragments of one’s own experience, are a powerful incentive for creative thinking.


Sign in / Sign up

Export Citation Format

Share Document