scholarly journals Slovene and Croatian word embeddings in terms of gender occupational analogies

2021 ◽  
Vol 9 (1) ◽  
pp. 26-59
Author(s):  
Matej Ulčar ◽  
Anka Supej ◽  
Marko Robnik-Šikonja ◽  
Senja Pollak
Keyword(s):  
Neural Networks ◽  
Natural Language ◽  
Gender Bias ◽  
Deep Neural Networks ◽  
Word Embeddings ◽  
Text Representation ◽  
Occupational Gender

In recent years, the use of deep neural networks and dense vector embeddings for text representation have led to excellent results in the field of computational understanding of natural language. It has also been shown that word embeddings often capture gender, racial and other types of bias. The article focuses on evaluating Slovene and Croatian word embeddings in terms of gender bias using word analogy calculations. We compiled a list of masculine and feminine nouns for occupations in Slovene and evaluated the gender bias of fastText, word2vec and ELMo embeddings with different configurations and different approaches to analogy calculations. The lowest occupational gender bias was observed with the fastText embeddings. Similarly, we compared different fastText embeddings on Croatian occupational analogies.

scholarly journals A Survey on Bias in Deep NLP

2021 ◽  
Vol 11 (7) ◽  
pp. 3184
Author(s):  
Ismael Garrido-Muñoz  ◽  
Arturo Montejo-Ráez  ◽  
Fernando Martínez-Santiago  ◽  
L. Alfonso Ureña-López 
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Natural Language Processing ◽  
Probability Distribution ◽  
Natural Language ◽  
Network Design ◽  
Language Processing ◽  
Deep Neural Networks ◽  
Learning Processes ◽  
Relevant Issue

Deep neural networks are hegemonic approaches to many machine learning areas, including natural language processing (NLP). Thanks to the availability of large corpora collections and the capability of deep architectures to shape internal language mechanisms in self-supervised learning processes (also known as “pre-training”), versatile and performing models are released continuously for every new network design. These networks, somehow, learn a probability distribution of words and relations across the training collection used, inheriting the potential flaws, inconsistencies and biases contained in such a collection. As pre-trained models have been found to be very useful approaches to transfer learning, dealing with bias has become a relevant issue in this new scenario. We introduce bias in a formal way and explore how it has been treated in several networks, in terms of detection and correction. In addition, available resources are identified and a strategy to deal with bias in deep NLP is proposed.

scholarly journals Learning Eligibility in Cancer Clinical Trials Using Deep Neural Networks

2018 ◽  
Vol 8 (7) ◽  
pp. 1206 ◽  
Author(s):  
Aurelia Bustos ◽  
Antonio Pertusa
Keyword(s):  
Neural Networks ◽  
Clinical Trials ◽  
Deep Neural Networks ◽  
Medical Knowledge ◽  
Clinical Information ◽  
Representation Learning ◽  
Free Text ◽  
Cancer Clinical Trials ◽  
Word Embeddings ◽  
New Treatments

Interventional cancer clinical trials are generally too restrictive, and some patients are often excluded on the basis of comorbidity, past or concomitant treatments, or the fact that they are over a certain age. The efficacy and safety of new treatments for patients with these characteristics are, therefore, not defined. In this work, we built a model to automatically predict whether short clinical statements were considered inclusion or exclusion criteria. We used protocols from cancer clinical trials that were available in public registries from the last 18 years to train word-embeddings, and we constructed a dataset of 6M short free-texts labeled as eligible or not eligible. A text classifier was trained using deep neural networks, with pre-trained word-embeddings as inputs, to predict whether or not short free-text statements describing clinical information were considered eligible. We additionally analyzed the semantic reasoning of the word-embedding representations obtained and were able to identify equivalent treatments for a type of tumor analogous with the drugs used to treat other tumors. We show that representation learning using deep neural networks can be successfully leveraged to extract the medical knowledge from clinical trial protocols for potentially assisting practitioners when prescribing treatments.

Sign in / Sign up

Export Citation Format

Share Document