Personalized API Recommendations

2021 ◽  
Vol 31 (09) ◽  
pp. 1299-1327
Author(s):  
Wenhua Yang ◽  
Yu Zhou ◽  
Zhiqiu Huang
Keyword(s):  
Short Term Memory ◽  
Language Model ◽  
Personal Characteristics ◽  
Daily Basis ◽  
Experimental Results ◽  
Language Models ◽  
Model Structure ◽  
Personalized Services ◽  
Application Programming ◽  
N Gram

Application Programming Interfaces (APIs) play an important role in modern software development. Developers interact with APIs on a daily basis and thus need to learn and memorize those APIs suitable for implementing the required functions. This can be a burden even for experienced developers since there exists a mass of available APIs. API recommendation techniques focus on assisting developers in selecting suitable APIs. However, existing API recommendation techniques have not taken the developers personal characteristics into account. As a result, they cannot provide developers with personalized API recommendation services. Meanwhile, they lack the support for self-defined APIs in the recommendation. To this end, we aim to propose a personalized API recommendation method that considers developers’ differences. Our API recommendation method is based on statistical language. We propose a model structure that combines the N-gram model and the long short-term memory (LSTM) neural network and train predictive models using API invoking sequences extracted from GitHub code repositories. A general language model trained on all sorts of code data is first acquired, based on which two personalized language models that recommend personalized library APIs and self-defined APIs are trained using the code data of the developer who needs personalized services. We evaluate our personalized API recommendation method on real-world developers, and the experimental results show that our approach achieves better accuracy in recommending both library APIs and self-defined APIs compared with the state-of-the-art. The experimental results also confirm the effectiveness of our hybrid model structure and the choice of the LSTM’s size.

scholarly journals Character n-Gram Embeddings to Improve RNN Language Models

2019 ◽  
Vol 33 ◽  
pp. 5074-5082 ◽  
Author(s):  
Sho Takase ◽  
Jun Suzuki ◽  
Masaaki Nagata
Keyword(s):  
Neural Network ◽  
Machine Translation ◽  
Recurrent Neural Network ◽  
Language Model ◽  
Language Modeling ◽  
Word Embedding ◽  
Experimental Results ◽  
Language Models ◽  
Word Embeddings ◽  
N Gram

This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wieting et al. 2016). Our proposed method constructs word embeddings from character ngram embeddings and combines them with ordinary word embeddings. We demonstrate that the proposed method achieves the best perplexities on the language modeling datasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct experiments on application tasks: machine translation and headline generation. The experimental results indicate that our proposed method also positively affects these tasks

INTEGRATION OF n-GRAM LANGUAGE MODELS IN MULTIPLE CLASSIFIER SYSTEMS FOR OFFLINE HANDWRITTEN TEXT LINE RECOGNITION

2008 ◽  
Vol 22 (07) ◽  
pp. 1301-1321 ◽  
Author(s):  
ROMAN BERTOLAMI ◽  
HORST BUNKE
Keyword(s):  
Language Model ◽  
Language Models ◽  
Combination Method ◽  
Text Line ◽  
Multiple Classifier Systems ◽  
Classifier Systems ◽  
Handwritten Text ◽  
Handwritten Text Recognition ◽  
Multiple Classifier ◽  
N Gram

Current multiple classifier systems for unconstrained handwritten text recognition do not provide a straightforward way to utilize language model information. In this paper, we describe a generic method to integrate a statistical n-gram language model into the combination of multiple offline handwritten text line recognizers. The proposed method first builds a word transition network and then rescores this network with an n-gram language model. Experimental evaluation conducted on a large dataset of offline handwritten text lines shows that the proposed approach improves the recognition accuracy over a reference system as well as over the original combination method that does not include a language model.

scholarly journals COVID-19 sentiment analysis via deep learning during the rise of novel cases

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0255615 ◽  
Author(s):  
Rohitash Chandra ◽  
Aswin Krishna
Keyword(s):  
Deep Learning ◽  
Sentiment Analysis ◽  
Short Term Memory ◽  
Language Model ◽  
Deep Understanding ◽  
Language Models ◽  
Catastrophic Events ◽  
Social Scientists ◽  
Significant Group ◽  
Global Vector

Social scientists and psychologists take interest in understanding how people express emotions and sentiments when dealing with catastrophic events such as natural disasters, political unrest, and terrorism. The COVID-19 pandemic is a catastrophic event that has raised a number of psychological issues such as depression given abrupt social changes and lack of employment. Advancements of deep learning-based language models have been promising for sentiment analysis with data from social networks such as Twitter. Given the situation with COVID-19 pandemic, different countries had different peaks where rise and fall of new cases affected lock-downs which directly affected the economy and employment. During the rise of COVID-19 cases with stricter lock-downs, people have been expressing their sentiments in social media. This can provide a deep understanding of human psychology during catastrophic events. In this paper, we present a framework that employs deep learning-based language models via long short-term memory (LSTM) recurrent neural networks for sentiment analysis during the rise of novel COVID-19 cases in India. The framework features LSTM language model with a global vector embedding and state-of-art BERT language model. We review the sentiments expressed for selective months in 2020 which covers the major peak of novel cases in India. Our framework utilises multi-label sentiment classification where more than one sentiment can be expressed at once. Our results indicate that the majority of the tweets have been positive with high levels of optimism during the rise of the novel COVID-19 cases and the number of tweets significantly lowered towards the peak. We find that the optimistic, annoyed and joking tweets mostly dominate the monthly tweets with much lower portion of negative sentiments. The predictions generally indicate that although the majority have been optimistic, a significant group of population has been annoyed towards the way the pandemic was handled by the authorities.

Half-Context Language Models

2011 ◽  
Vol 37 (4) ◽  
pp. 843-865
Author(s):  
Hinrich Schütze ◽  
Michael Walsh
Keyword(s):  
Clustering Algorithm ◽  
Language Model ◽  
Model Performance ◽  
Language Models ◽  
Fine Grained ◽  
Specific Analysis ◽  
Distributional Information ◽  
N Gram ◽  
Context Specific ◽  
Context Models

This article investigates the effects of different degrees of contextual granularity on language model performance. It presents a new language model that combines clustering and half-contextualization, a novel representation of contexts. Half-contextualization is based on the half-context hypothesis that states that the distributional characteristics of a word or bigram are best represented by treating its context distribution to the left and right separately and that only directionally relevant distributional information should be used. Clustering is achieved using a new clustering algorithm for class-based language models that compares favorably to the exchange algorithm. When interpolated with a Kneser-Ney model, half-context models are shown to have better perplexity than commonly used interpolated n-gram models and traditional class-based approaches. A novel, fine-grained, context-specific analysis highlights those contexts in which the model performs well and those which are better treated by existing non-class-based models.

scholarly journals A General Technique to Train Language Models on Language Models

2005 ◽  
Vol 31 (2) ◽  
pp. 173-185 ◽  
Author(s):  
Mark-Jan Nederhof
Keyword(s):  
Second Language ◽  
Finite Automaton ◽  
Language Model ◽  
Language Models ◽  
General Technique ◽  
Context Free Grammar ◽  
N Gram ◽  
Context Free ◽  
Probabilistic Context

We show that under certain conditions, a language model can be trained on the basis of a second language model. The main instance of the technique trains a finite automaton on the basis of a probabilistic context-free grammar, such that the Kullback-Leibler distance between grammar and trained automaton is provably minimal. This is a substantial generalization of an existing algorithm to train an n-gram model on the basis of a probabilistic context-free grammar.

Using Pre-trained Language Model to Enhance Active Learning for Sentence Matching

2022 ◽  
Vol 21 (2) ◽  
pp. 1-19
Author(s):  
Guirong Bai ◽  
Shizhu He ◽  
Kang Liu ◽  
Jun Zhao
Keyword(s):  
Active Learning ◽  
Language Model ◽  
Experimental Results ◽  
Language Models ◽  
Data Driven ◽  
Learning Approach ◽  
Sentence Matching ◽  
Learning Language

Active learning is an effective method to substantially alleviate the problem of expensive annotation cost for data-driven models. Recently, pre-trained language models have been demonstrated to be powerful for learning language representations. In this article, we demonstrate that the pre-trained language model can also utilize its learned textual characteristics to enrich criteria of active learning. Specifically, we provide extra textual criteria with the pre-trained language model to measure instances, including noise, coverage, and diversity. With these extra textual criteria, we can select more efficient instances for annotation and obtain better results. We conduct experiments on both English and Chinese sentence matching datasets. The experimental results show that the proposed active learning approach can be enhanced by the pre-trained language model and obtain better performance.

A Comparison of Phrase Based and Word based Language Model for Punjabi

2017 ◽  
Vol 7 (7) ◽  
pp. 444
Author(s):  
Umrinderpal Singh
Keyword(s):  
Machine Translation ◽  
Great Improvement ◽  
Language Model ◽  
Language Models ◽  
Translation System ◽  
Information Base ◽  
Model Yield ◽  
Machine Translation System ◽  
N Gram

A language model provides connection to the decoding process to determine a precise word from several available options in the information base or phrase table. The language model can be generated using n-gram approach. Various language models and smoothing procedures are there to determine this model, like unigram, bigram, trigram, interpolation, backoff language model etc. We have done some experiments with different language models where we have used phrases in place of words as the smallest unit. Experiments have shown that phrase based language model yield more accurate results as compared to simple word based mode. We have also done some experiments with machine translation system where we have used phrase based language model rather than word based model and system yield great improvement.

scholarly journals A Generalized Language Model in Tensor Space

2019 ◽  
Vol 33 ◽  
pp. 7450-7458 ◽  
Author(s):  
Lipeng Zhang ◽  
Peng Zhang ◽  
Xindian Ma ◽  
Shuqin Gu ◽  
Zhan Su ◽  
...  
Keyword(s):  
Language Model ◽  
Tensor Decomposition ◽  
Expressive Power ◽  
Semantic Space ◽  
Language Models ◽  
Tensor Representation ◽  
Order Tensor ◽  
Tensor Space ◽  
Recursive Calculation ◽  
N Gram

In the literature, tensors have been effectively used for capturing the context information in language models. However, the existing methods usually adopt relatively-low order tensors, which have limited expressive power in modeling language. Developing a higher-order tensor representation is challenging, in terms of deriving an effective solution and showing its generality. In this paper, we propose a language model named Tensor Space Language Model (TSLM), by utilizing tensor networks and tensor decomposition. In TSLM, we build a high-dimensional semantic space constructed by the tensor product of word vectors. Theoretically, we prove that such tensor representation is a generalization of the n-gram language model. We further show that this high-order tensor representation can be decomposed to a recursive calculation of conditional probability for language modeling. The experimental results on Penn Tree Bank (PTB) dataset and WikiText benchmark demonstrate the effectiveness of TSLM.

scholarly journals Modelling and Optimizing on Syntactic N-Grams for Statistical Machine Translation

2015 ◽  
Vol 3 ◽  
pp. 169-182 ◽  
Author(s):  
Rico Sennrich
Keyword(s):  
Statistical Machine Translation ◽  
Language Model ◽  
Language Models ◽  
Translation Quality ◽  
N Gram ◽  
Evaluation Metric ◽  
Log Linear ◽  
Dependency Structures ◽  
Free Word

The role of language models in SMT is to promote fluent translation output, but traditional n-gram language models are unable to capture fluency phenomena between distant words, such as some morphological agreement phenomena, subcategorisation, and syntactic collocations with string-level gaps. Syntactic language models have the potential to fill this modelling gap. We propose a language model for dependency structures that is relational rather than configurational and thus particularly suited for languages with a (relatively) free word order. It is trainable with Neural Networks, and not only improves over standard n-gram language models, but also outperforms related syntactic language models. We empirically demonstrate its effectiveness in terms of perplexity and as a feature function in string-to-tree SMT from English to German and Russian. We also show that using a syntactic evaluation metric to tune the log-linear parameters of an SMT system further increases translation quality when coupled with a syntactic language model.

scholarly journals LanguageCrawl: a generic tool for building language models upon common Crawl

2021 ◽  
Author(s):  
Szymon Roziewski ◽  
Marek Kozłowski
Keyword(s):  
Language Processing ◽  
Deep Neural Networks ◽  
Language Model ◽  
Language Models ◽  
Unstructured Data ◽  
Web Pages ◽  
Data Intensive ◽  
The Common ◽  
Internet Community ◽  
N Gram

AbstractThe exponential growth of the internet community has resulted in the production of a vast amount of unstructured data, including web pages, blogs and social media. Such a volume consisting of hundreds of billions of words is unlikely to be analyzed by humans. In this work we introduce the tool LanguageCrawl, which allows Natural Language Processing (NLP) researchers to easily build web-scale corpora using the Common Crawl Archive—an open repository of web crawl information, which contains petabytes of data. We present three use cases in the course of this work: filtering of Polish websites, the construction of n-gram corpora and the training of a continuous skipgram language model with hierarchical softmax. Each of them has been implemented within the LanguageCrawl toolkit, with the possibility to adjust specified language and n-gram ranks. This paper focuses particularly on high computing efficiency by applying highly concurrent multitasking. Our tool utilizes effective libraries and design. LanguageCrawl has been made publicly available to enrich the current set of NLP resources. We strongly believe that our work will facilitate further NLP research, especially in under-resourced languages, in which the lack of appropriately-sized corpora is a serious hindrance to applying data-intensive methods, such as deep neural networks.

Sign in / Sign up

Export Citation Format

Share Document