Voice Activation for Low-Resource Languages

Voice activation systems are used to find a pre-defined word or phrase in the audio stream. Industry solutions, such as “OK, Google” for Android devices, are trained with millions of samples. In this work, we propose and investigate several ways to train a voice activation system when the in-domain data set is small. We compare self-training exemplar pre-training, fine-tuning a model pre-trained on another domain, joint training on both an out-of-domain high-resource and a target low-resource data set, and unsupervised pre-training. In our experiments, the unsupervised pre-training and the joint-training with a high-resource data set from another domain significantly outperform a strong baseline of fine-tuning a model trained on another data set. We obtain 7–25% relative improvement depending on the model architecture. Additionally, we improve the best test accuracy on the Lithuanian data set from 90.77% to 93.85%.

Download Full-text

Modified Deep Neural Networks for Dog Breeds Identification

10.20944/preprints201812.0232.v1 ◽

2018 ◽

Cited By ~ 1

Author(s):

Aydin Ayanzadeh ◽

Sahand Vahidnia

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

State Of The Art ◽

The State ◽

Fine Tuning ◽

Test Accuracy ◽

Data Sets ◽

Data Set

In this paper, we leverage state of the art models on Imagenet data-sets. We use the pre-trained model and learned weighs to extract the feature from the Dog breeds identification data-set. Afterwards, we applied fine-tuning and dataaugmentation to increase the performance of our test accuracy in classification of dog breeds datasets. The performance of the proposed approaches are compared with the state of the art models of Image-Net datasets such as ResNet-50, DenseNet-121, DenseNet-169 and GoogleNet. we achieved 89.66% , 85.37% 84.01% and 82.08% test accuracy respectively which shows thesuperior performance of proposed method to the previous works on Stanford dog breeds datasets.

Download Full-text

A fast, resource efficient and reliable rule-based system for COVID-19 symptom identification

JAMIA Open ◽

10.1093/jamiaopen/ooab070 ◽

2021 ◽

Author(s):

Himanshu S Sahoo ◽

Greg M Silverman ◽

Nicholas E Ingraham ◽

Monica I Lupei ◽

Michael A Puskarich ◽

...

Keyword(s):

Real Time ◽

Resource Utilization ◽

Fine Tuning ◽

Rule Based ◽

Low Resource ◽

Annotation System ◽

High Resource ◽

Industry Standard ◽

Depth Analysis ◽

Current Annotation

Abstract Objective With COVID-19 there was a need for rapidly scalable annotation system that facilitated real-time integration with clinical decision support systems (CDS). Current annotation systems suffer from high resource utilization and poor scalability limiting real-world integration with CDS. A potential solution to mitigate these issues is to use the rule-based gazetteer developed at our institution. Materials and Methods Performance, resource utilization and runtime of the rule-based gazetteer was compared with five annotation systems: BioMedICUS, cTAKES, MetaMap, CLAMP and MedTagger. Results This rule-based gazetteer was fastest, had low resource footprint and similar performance for weighted micro-average and macro-average measures of precision, recall and f1-score compared to other annotation systems. Discussion Opportunities to increase its performance include fine-tuning lexical rules for symptom identification. Additionally, it could run on multiple compute nodes for faster runtime. Conclusion This rule-based gazetteer overcame key technical limitations facilitating real-time symptomatology identification for COVID-19 and integration of unstructured data elements into our CDS. It is ideal for large-scale deployment across a wide variety of health care settings for surveillance of acute COVID-19 symptoms for integration into prognostic modeling. Such a system is currently being leveraged for monitoring of post-acute sequelae of COVID-19 (PASC) progression in COVID-19 survivors. This study conducted the first in-depth analysis and developed a rule-based gazetteer for COVID-19 symptom extraction with the following key features: low processor and memory utilization, faster runtime and similar weighted micro-average and macro-average measures for precision, recall and f1-score compared to industry standard annotation systems. Lay Summary With COVID-19 came an unprecedented need to identify symptoms of COVID-19 patients under investigation (PUIs) in a time sensitive, resource-efficient and accurate manner. While available annotation systems perform well for smaller healthcare settings, they fail to scale in larger healthcare systems where 10,000+ clinical notes are generated a day. This study covers 3 improvements addressing key limitations of current annotation systems. (1) High resource utilization and poor scalability of existing annotation systems. The presented rule-based gazetteer is a high-throughput annotation system for processing high volume of notes, thus, providing opportunity for clinicians to make more informed time-sensitive decisions around patient care. (2) Equally important is our developed rule-based gazetteer performs similar or better than current annotation systems for symptom identification. (3) Due to minimal resource needs of the rule-based gazetteer, it could be deployed at healthcare sites lacking a robust infrastructure where industry standard annotation systems cannot be deployed because of low resource availability.

Download Full-text

POISE: Efficient Cross-Domain Chinese Named Entity Recognization via Transfer Learning

Symmetry ◽

10.3390/sym12101673 ◽

2020 ◽

Vol 12 (10) ◽

pp. 1673

Author(s):

Jiabao Sheng ◽

Aishan Wumaier ◽

Zhe Li

Keyword(s):

Transfer Learning ◽

Entity Recognition ◽

Target Domain ◽

Learning Methods ◽

Low Resource ◽

Named Entity ◽

High Resource ◽

Percentage Points ◽

Entity Annotation ◽

Resource Data

To improve the performance of deep learning methods in case of a lack of labeled data for entity annotation in entity recognition tasks, this study proposes transfer learning schemes that combine the character to be the word to convert low-resource data symmetry into high-resource data. We combine character embedding, word embedding, and the embedding of the label features using high- and low-resource data based on the BiLSTM-CRF model, and perform the feature-transfer and parameter-sharing tasks in two domains of the BiLSTM network to annotate with zero resources. Before transfer learning, we must first calculate the label similarity between two different domains and select the label features with large similarity for feature transfer mapping. All training parameters of the source domain in the model are shared during the BiLSTM network processing and CRF layer. In addition, we also use the method of combining characters and words to reduce the problem of word segmentation across domains and reduce the error rate in label mapping. The results of experiments show that in terms of the overall F1 score, the proposed model without supervision was superior by 9.76 percentage points to the general parametric shared transfer learning method, and by 9.08 and 12.38 percentage points, respectively, to two recent high–low resource learning methods. The proposed scheme improves performance in terms of transfer learning between the high- and low-resource data and can identify the predicted data in the target domain.

Download Full-text

Low-Resource Named Entity Recognition via the Pre-Training Model

Symmetry ◽

10.3390/sym13050786 ◽

2021 ◽

Vol 13 (5) ◽

pp. 786

Author(s):

Siqi Chen ◽

Yijie Pei ◽

Zunwang Ke ◽

Wushour Silamu

Keyword(s):

Data Augmentation ◽

Language Model ◽

Named Entity Recognition ◽

Name Entity Recognition ◽

Fine Tuning ◽

Entity Recognition ◽

Language Models ◽

Low Resource ◽

Named Entity ◽

High Resource

Named entity recognition (NER) is an important task in the processing of natural language, which needs to determine entity boundaries and classify them into pre-defined categories. For low-resource languages, most state-of-the-art systems require tens of thousands of annotated sentences to obtain high performance. However, there is minimal annotated data available about Uyghur and Hungarian (UH languages) NER tasks. There are also specificities in each task—differences in words and word order across languages make it a challenging problem. In this paper, we present an effective solution to providing a meaningful and easy-to-use feature extractor for named entity recognition tasks: fine-tuning the pre-trained language model. Therefore, we propose a fine-tuning method for a low-resource language model, which constructs a fine-tuning dataset through data augmentation; then the dataset of a high-resource language is added; and finally the cross-language pre-trained model is fine-tuned on this dataset. In addition, we propose an attention-based fine-tuning strategy that uses symmetry to better select relevant semantic and syntactic information from pre-trained language models and apply these symmetry features to name entity recognition tasks. We evaluated our approach on Uyghur and Hungarian datasets, which showed wonderful performance compared to some strong baselines. We close with an overview of the available resources for named entity recognition and some of the open research questions.

Download Full-text

Low Resource Sequence Tagging with Weak Labels

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6415 ◽

2020 ◽

Vol 34 (05) ◽

pp. 8862-8869

Author(s):

Edwin Simpson ◽

Jonas Pfeiffer ◽

Iryna Gurevych

Keyword(s):

Transfer Learning ◽

Domain Adaptation ◽

Ensemble Methods ◽

Training Data ◽

Target Domain ◽

Low Resource ◽

Standard Data ◽

High Resource ◽

Crowdsourced Data ◽

Resource Data

Current methods for sequence tagging depend on large quantities of domain-specific training data, limiting their use in new, user-defined tasks with few or no annotations. While crowdsourcing can be a cheap source of labels, it often introduces errors that degrade the performance of models trained on such crowdsourced data. Another solution is to use transfer learning to tackle low resource sequence labelling, but current approaches rely heavily on similar high resource datasets in different languages. In this paper, we propose a domain adaptation method using Bayesian sequence combination to exploit pre-trained models and unreliable crowdsourced data that does not require high resource data in a different language. Our method boosts performance by learning the relationship between each labeller and the target task and trains a sequence labeller on the target domain with little or no gold-standard data. We apply our approach to labelling diagnostic classes in medical and educational case studies, showing that the model achieves strong performance though zero-shot transfer learning and is more effective than alternative ensemble methods. Using NER and information extraction tasks, we show how our approach can train a model directly from crowdsourced labels, outperforming pipeline approaches that first aggregate the crowdsourced data, then train on the aggregated labels.

Download Full-text

Male gyn-oncologists reach conclusions faster, but female gyn-oncologists are more rational, both in high resource and low resource countries

10.26226/morressier.5770e29ed462b80290b4c595 ◽

2016 ◽

Author(s):

Walter Gotlieb

Keyword(s):

Low Resource ◽

High Resource

Download Full-text

Deep Learning Techniques for Grape Plant Species Identification in Natural Images

Sensors ◽

10.3390/s19224850 ◽

2019 ◽

Vol 19 (22) ◽

pp. 4850 ◽

Cited By ~ 6

Author(s):

Carlos S. Pereira ◽

Raul Morais ◽

Manuel J. C. S. Reis

Keyword(s):

Transfer Learning ◽

Climatic Conditions ◽

Fine Tuning ◽

Variety Identification ◽

Test Accuracy ◽

Accuracy Score ◽

Learning Techniques ◽

Four Corners ◽

Integrated Software ◽

Grape Varieties

Frequently, the vineyards in the Douro Region present multiple grape varieties per parcel and even per row. An automatic algorithm for grape variety identification as an integrated software component was proposed that can be applied, for example, to a robotic harvesting system. However, some issues and constraints in its development were highlighted, namely, the images captured in natural environment, low volume of images, high similarity of the images among different grape varieties, leaf senescence, and significant changes on the grapevine leaf and bunch images in the harvest seasons, mainly due to adverse climatic conditions, diseases, and the presence of pesticides. In this paper, the performance of the transfer learning and fine-tuning techniques based on AlexNet architecture were evaluated when applied to the identification of grape varieties. Two natural vineyard image datasets were captured in different geographical locations and harvest seasons. To generate different datasets for training and classification, some image processing methods, including a proposed four-corners-in-one image warping algorithm, were used. The experimental results, obtained from the application of an AlexNet-based transfer learning scheme and trained on the image dataset pre-processed through the four-corners-in-one method, achieved a test accuracy score of 77.30%. Applying this classifier model, an accuracy of 89.75% on the popular Flavia leaf dataset was reached. The results obtained by the proposed approach are promising and encouraging in helping Douro wine growers in the automatic task of identifying grape varieties.

Download Full-text

Classification of jujube defects in small data sets based on transfer learning

Neural Computing and Applications ◽

10.1007/s00521-021-05715-2 ◽

2021 ◽

Author(s):

Jianping Ju ◽

Hong Zheng ◽

Xiaohang Xu ◽

Zhongyuan Guo ◽

Zhaohui Zheng ◽

...

Keyword(s):

Transfer Learning ◽

Loss Function ◽

Training Model ◽

Parameter Distribution ◽

Test Accuracy ◽

Small Data ◽

Data Sets ◽

Data Set ◽

Small Data Sets

AbstractAlthough convolutional neural networks have achieved success in the field of image classification, there are still challenges in the field of agricultural product quality sorting such as machine vision-based jujube defects detection. The performance of jujube defect detection mainly depends on the feature extraction and the classifier used. Due to the diversity of the jujube materials and the variability of the testing environment, the traditional method of manually extracting the features often fails to meet the requirements of practical application. In this paper, a jujube sorting model in small data sets based on convolutional neural network and transfer learning is proposed to meet the actual demand of jujube defects detection. Firstly, the original images collected from the actual jujube sorting production line were pre-processed, and the data were augmented to establish a data set of five categories of jujube defects. The original CNN model is then improved by embedding the SE module and using the triplet loss function and the center loss function to replace the softmax loss function. Finally, the depth pre-training model on the ImageNet image data set was used to conduct training on the jujube defects data set, so that the parameters of the pre-training model could fit the parameter distribution of the jujube defects image, and the parameter distribution was transferred to the jujube defects data set to complete the transfer of the model and realize the detection and classification of the jujube defects. The classification results are visualized by heatmap through the analysis of classification accuracy and confusion matrix compared with the comparison models. The experimental results show that the SE-ResNet50-CL model optimizes the fine-grained classification problem of jujube defect recognition, and the test accuracy reaches 94.15%. The model has good stability and high recognition accuracy in complex environments.

Download Full-text

Phrase Table Combination Based on Symmetrization of Word Alignment for Low-Resource Languages

Applied Sciences ◽

10.3390/app11041868 ◽

2021 ◽

Vol 11 (4) ◽

pp. 1868

Author(s):

Sari Dewi Budiwati ◽

Al Hafiz Akbar Maulana Siagian ◽

Tirana Noor Fatyanosa ◽

Masayoshi Aritsugi

Keyword(s):

Language Model ◽

Word Alignment ◽

Low Resource ◽

Relative Improvement ◽

Direct Translation ◽

The Common ◽

Target Source

Phrase table combination in pivot approaches can be an effective method to deal with low-resource language pairs. The common practice to generate phrase tables in pivot approaches is to use standard symmetrization, i.e., grow-diag-final-and. Although some researchers found that the use of non-standard symmetrization could improve bilingual evaluation understudy (BLEU) scores, the use of non-standard symmetrization has not been commonly employed in pivot approaches. In this study, we propose a strategy that uses the non-standard symmetrization of word alignment in phrase table combination. The appropriate symmetrization is selected based on the highest BLEU scores in each direct translation of source–target, source–pivot, and pivot–target of Kazakh–English (Kk–En) and Japanese–Indonesian (Ja–Id). Our experiments show that our proposed strategy outperforms the direct translation in Kk–En with absolute improvements of 0.35 (a 11.3% relative improvement) and 0.22 (a 6.4% relative improvement) BLEU points for 3-gram and 5-gram, respectively. The proposed strategy shows an absolute gain of up to 0.11 (a 0.9% relative improvement) BLEU points compared to direct translation for 3-gram in Ja–Id. Our proposed strategy using a small phrase table obtains better BLEU scores than a strategy using a large phrase table. The size of the target monolingual and feature function weight of the language model (LM) could reduce perplexity scores.

Download Full-text

A framework for designing medical devices resilient to low-resource settings

Globalization and Health ◽

10.1186/s12992-021-00718-z ◽

2021 ◽

Vol 17 (1) ◽

Author(s):

Davide Piaggio ◽

Rossana Castaldo ◽

Marco Cinelli ◽

Sara Cinelli ◽

Alessia Maccaro ◽

...

Keyword(s):

Delphi Study ◽

Middle Income ◽

Low Resource Settings ◽

Health Technologies ◽

Low Resource ◽

Middle Income Countries ◽

High Resource ◽

International Experts ◽

Global Population ◽

Different Parts

Abstract Background To date (April 2021), medical device (MD) design approaches have failed to consider the contexts where MDs can be operationalised. Although most of the global population lives and is treated in Low- and Middle-Income Countries (LMCIs), over 80% of the MD market share is in high-resource settings, which set de facto standards that cannot be taken for granted in lower resource settings. Using a MD designed for high-resource settings in LMICs may hinder its safe and efficient operationalisation. In the literature, many criteria for frameworks to support resilient MD design were presented. However, since the available criteria (as of 2021) are far from being consensual and comprehensive, the aim of this study is to raise awareness about such challenges and to scope experts’ consensus regarding the essentiality of MD design criteria. Results This paper presents a novel application of Delphi study and Multiple Criteria Decision Analysis (MCDA) to develop a framework comprising 26 essential criteria, which were evaluated and chosen by international experts coming from different parts of the world. This framework was validated by analysing some MDs presented in the WHO Compendium of innovative health technologies for low-resource settings. Conclusions This novel holistic framework takes into account some domains that are usually underestimated by MDs designers. For this reason, it can be used by experts designing MDs resilient to low-resource settings and it can also assist policymakers and non-governmental organisations in shaping the future of global healthcare.

Download Full-text