scholarly journals The Rediscovery Hypothesis: Language Models Need to Meet Linguistics

2021 ◽  
Vol 72 ◽  
pp. 1343-1384
Author(s):  
Vassilina Nikoulina ◽  
Maxat Tezekbayev ◽  
Nuradil Kozhakhmet ◽  
Madina Babazhanova ◽  
Matthias Gallé ◽  
...  
Keyword(s):  
Language Modeling ◽  
Language Models ◽  
Linguistic Knowledge ◽  
Necessary Condition ◽  
Ongoing Debate ◽  
Linguistic Information ◽  
Information Theoretic ◽  
Modern Language ◽  
The Impact ◽  
Linguistic Structures

There is an ongoing debate in the NLP community whether modern language models contain linguistic knowledge, recovered through so-called probes. In this paper, we study whether linguistic knowledge is a necessary condition for the good performance of modern language models, which we call the rediscovery hypothesis. In the first place, we show that language models that are significantly compressed but perform well on their pretraining objectives retain good scores when probed for linguistic structures. This result supports the rediscovery hypothesis and leads to the second contribution of our paper: an information-theoretic framework that relates language modeling objectives with linguistic information. This framework also provides a metric to measure the impact of linguistic information on the word prediction task. We reinforce our analytical results with various experiments, both on synthetic and on real NLP tasks in English.

Reducing the Risks of the Financial Sector to Ensure Sustainable Growth of the Russian Еconomy

2019 ◽  
Vol 12 (3) ◽  
pp. 86-92
Author(s):  
T. I. Minina ◽  
V. V. Skalkin
Keyword(s):  
Economic Growth ◽  
Monetary Policy ◽  
Financial Sector ◽  
Developed Countries ◽  
Sustainable Growth ◽  
Necessary Condition ◽  
Real Sector ◽  
Financial Instruments ◽  
The Real ◽  
The Impact

Russia’s entry into the top five economies of the world depends, among other things, on the development of the financial sector, being a necessary condition for the economic growth of a developed macroeconomic and macro-financial system. The financial sector represents a system of relationships for the effective collection and distribution of economic resources, their deployment according to public demand, reducing the risk of overproduction and overheating of the economy.Therefore, the subject of the research is the financial sector of the Russian economy.The purpose of the research was to formulate an approach to alleviating the risks of increasing financial costs in the real sector of the economy by reducing the impact of endogenous risks expressed as financial asset “bubbles” using the experience of developed countries in the monetary policy.The paper analyzes a macroeconomic model applied to the financial sector. It is established that the economic growth is determined by the growth and, more important, the qualitative development of the financial sector, which leads to two phenomena: overproduction in the real sector and an increase in asset prices in the financial sector, with a debt load in both the real and financial sectors. This results in decreasing the interest rate of the mega-regulator to near-zero values. In this case, since the mechanisms of the conventional monetary policy do not work, the unconventional monetary policy is used when the mega-regulator buys out derivative financial instruments from systemically important institutions. As a conclusion, given deflationally low rates, it is proposed that the megaregulator should issue its own derivative financial instruments and place them in the financial market.

Workplace Spirituality and Organizational Commitment: An Empirical Exploration

2017 ◽  
Vol 10 (02) ◽  
Author(s):  
Naval Garg
Keyword(s):  
Organizational Commitment ◽  
Sense Of Community ◽  
Workplace Spirituality ◽  
Necessary Condition ◽  
Continuance Commitment ◽  
Six Dimensions ◽  
The Impact ◽  
Structured Questionnaire

The paper aims to empirically explore the impact of six dimensions of workplace spirituality on three types of organizational commitment. Six dimensions of workplace spirituality used for the study are Swadharma, Lokasangraha, authenticity, sense of community, Karma capital and Krityagyata. Components of organizational commitment are affective, normative and continuance commitment. A sample of 541 employees working in various organizations was given a structured questionnaire. Correlations, regressions and Necessary Condition Analysis(NCA) were carried out. The paper has enriched the field of workplace spirituality by contributing to existing literature via adding one more construct of Indian spirituality i.e. Krityagyata. Paper concludes that workplace spirituality climate helps in promoting organizational commitment. NCA elicited necessity of various dimensions of workplace spirituality for healthy organizational commitment.

scholarly journals Research on the Relationship between Shared Leadership and Individual Creativity-Qualitative Comparative Analysis on the Basis of Clear Set

2021 ◽  
Vol 13 (10) ◽  
pp. 5445
Author(s):  
Muyun Sun ◽  
Jigan Wang ◽  
Ting Wen
Keyword(s):  
Comparative Analysis ◽  
Environmental Factors ◽  
Shared Leadership ◽  
Management Practices ◽  
Qualitative Comparative Analysis ◽  
Necessary Condition ◽  
The Individual ◽  
The Impact ◽  
The Relationship ◽  
Sufficient And Necessary Condition

Creativity is the key to obtaining and maintaining competitiveness of modern organizations, and it has attracted much attention from academic circles and management practices. Shared leadership is believed to effectively influence team output. However, research on the impact of individual creativity is still in its infancy. This study adopts the qualitative comparative analysis method, taking 1584 individuals as the research objects, underpinned by a questionnaire-based survey. It investigates the influence of the team’s shared leadership network elements and organizational environmental factors on the individual creativity. We have found that there are six combination of conditions of shared leadership and organizational environmental factors constituting sufficient combination of conditions to increase or decrease individual creativity. Moreover, we have noticed that the low network density of shared leadership is a sufficient and necessary condition of reducing individual creativity. Our results also provide management suggestions for practical activities during the team management.

Patterns of interpredictability and principal parts in Latin verb paradigms: an entropy-based approach

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Matteo Pellegrini
Keyword(s):  
Principal Part ◽  
Quantitative Approach ◽  
Information Theoretic ◽  
Verb Inflection ◽  
Type Frequency ◽  
Principal Parts ◽  
The Impact

AbstractThis paper provides a fully word-based, abstractive analysis of predictability in Latin verb paradigms. After reviewing previous traditional and theoretically grounded accounts of Latin verb inflection, a procedure is outlined where the uncertainty in guessing the content of paradigm cells given knowledge of one or more inflected wordforms is measured by means of the information-theoretic notions of unary and n-ary implicative entropy, respectively, in a quantitative approach that uses the type frequency of alternation patterns between wordforms as an estimate of their probability of application. Entropy computations are performed by using the Qumin toolkit on data taken from the inflected lexicon LatInfLexi. Unary entropy values are used to draw a mapping of the verbal paradigm in zones of full interpredictability, composed of cells that can be inferred from one another with no uncertainty. N-ary entropy values are used to extract categorical and near principal part sets, that allow to fill the rest of the paradigm with little or no uncertainty. Lastly, the issue of the impact of information on the derivational relatedness of lexemes on uncertainty in inflectional predictions is tackled, showing that adding a classification of verbs in derivational families allows for a relevant reduction of entropy, not only for derived verbs, but also for simple ones.

scholarly journals Marsh rabbit mortalities tie pythons to the precipitous decline of mammals in the Everglades

2015 ◽  
Vol 282 (1805) ◽  
pp. 20150120 ◽  
Author(s):  
Robert A. McCleery ◽  
Adia Sovie ◽  
Robert N. Reed ◽  
Mark W. Cunningham ◽  
Margaret E. Hunter ◽  
...  
Keyword(s):  
National Park ◽  
Large Scale ◽  
Ongoing Debate ◽  
Faunal Communities ◽  
Python Molurus ◽  
Scale Experiment ◽  
Precipitous Decline ◽  
Python Molurus Bivittatus ◽  
The Impact ◽  
Ecological Functioning

To address the ongoing debate over the impact of invasive species on native terrestrial wildlife, we conducted a large-scale experiment to test the hypothesis that invasive Burmese pythons ( Python molurus bivittatus ) were a cause of the precipitous decline of mammals in Everglades National Park (ENP). Evidence linking pythons to mammal declines has been indirect and there are reasons to question whether pythons, or any predator, could have caused the precipitous declines seen across a range of mammalian functional groups. Experimentally manipulating marsh rabbits, we found that pythons accounted for 77% of rabbit mortalities within 11 months of their translocation to ENP and that python predation appeared to preclude the persistence of rabbit populations in ENP. On control sites, outside of the park, no rabbits were killed by pythons and 71% of attributable marsh rabbit mortalities were classified as mammal predations. Burmese pythons pose a serious threat to the faunal communities and ecological functioning of the Greater Everglades Ecosystem, which will probably spread as python populations expand their range.

Secret Error-Correcting Network Coding against Eavesdropping and Pollution Attacks

2012 ◽  
Vol 457-458 ◽  
pp. 1499-1507 ◽  
Author(s):  
Si Guang Chen ◽  
Meng Wu ◽  
Wei Feng Lu
Keyword(s):  
Network Coding ◽  
Necessary Condition ◽  
Information Theoretic ◽  
Transmitted Information ◽  
Information Theoretic Security ◽  
Secret Communication ◽  
Pollution Attacks ◽  
Rank Metric Codes ◽  
Shared Secret ◽  
And Performance

In this work we consider the problem of designing a secret error-correcting network coding scheme against an adversary that can re-select the tapping links in different time slot and inject z erroneous packets into network. We first derive a necessary condition for keeping the transmitted information secret from the adversary, while the network is only subject to the eavesdropping attack. We then design an error-correcting scheme by combining the rank-metric codes with shared secret model, which can decode the transmitted information correctly provided a sufficiently large q. With that, a secret error-correcting network coding is proposed by combining this error-correcting scheme with secret communication. We show that under the requirement of communication can achieve a rate of packets. Moreover, it ensures that the communicated information is reliable and information-theoretic security from the adversary. In particular, the requirement of packet length is not as large as the required in [12]. Finally, the security and performance analyses illustrate the characteristics of our scheme.

scholarly journals Language Modeling for Morphologically Rich Languages: Character-Aware Modeling for Word-Level Prediction

2018 ◽  
Vol 6 ◽  
pp. 451-465 ◽  
Author(s):  
Daniela Gerz ◽  
Ivan Vulić ◽  
Edoardo Ponti ◽  
Jason Naradowsky ◽  
Roi Reichart ◽  
...  
Keyword(s):  
Large Scale ◽  
Language Modeling ◽  
Language Models ◽  
Data Sets ◽  
High Type ◽  
Word Level ◽  
Level Information ◽  
Character Sequences ◽  
Novel Method ◽  
Morphologically Rich Languages

Neural architectures are prominent in the construction of language models (LMs). However, word-level prediction is typically agnostic of subword-level information (characters and character sequences) and operates over a closed vocabulary, consisting of a limited word set. Indeed, while subword-aware models boost performance across a variety of NLP tasks, previous work did not evaluate the ability of these models to assist next-word prediction in language modeling tasks. Such subword-level informed models should be particularly effective for morphologically-rich languages (MRLs) that exhibit high type-to-token ratios. In this work, we present a large-scale LM study on 50 typologically diverse languages covering a wide variety of morphological systems, and offer new LM benchmarks to the community, while considering subword-level information. The main technical contribution of our work is a novel method for injecting subword-level information into semantic word vectors, integrated into the neural language modeling training, to facilitate word-level prediction. We conduct experiments in the LM setting where the number of infrequent words is large, and demonstrate strong perplexity gains across our 50 languages, especially for morphologically-rich languages. Our code and data sets are publicly available.

scholarly journals Generating Sentences by Editing Prototypes

2018 ◽  
Vol 6 ◽  
pp. 437-450 ◽  
Author(s):  
Kelvin Guu ◽  
Tatsunori B. Hashimoto ◽  
Yonatan Oren ◽  
Percy Liang
Keyword(s):  
Language Model ◽  
Language Modeling ◽  
Language Models ◽  
Training Corpus ◽  
Human Evaluation ◽  
Sentence Level ◽  
Sentence Similarity ◽  
Traditional Language ◽  
Generative Language

We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.

Sign in / Sign up

Export Citation Format

Share Document