scholarly journals An overview of multi-task learning

2017 ◽  
Vol 5 (1) ◽  
pp. 30-43 ◽  
Author(s):  
Yu Zhang ◽  
Qiang Yang
Keyword(s):  
Supervised Learning ◽  
Language Processing ◽  
Health Informatics ◽  
Web Applications ◽  
Task Learning ◽  
Learning Tasks ◽  
Promising Area ◽  
Speed Up ◽  
Definition Of ◽  
Theoretical Analyses

Abstract As a promising area in machine learning, multi-task learning (MTL) aims to improve the performance of multiple related learning tasks by leveraging useful information among them. In this paper, we give an overview of MTL by first giving a definition of MTL. Then several different settings of MTL are introduced, including multi-task supervised learning, multi-task unsupervised learning, multi-task semi-supervised learning, multi-task active learning, multi-task reinforcement learning, multi-task online learning and multi-task multi-view learning. For each setting, representative MTL models are presented. In order to speed up the learning process, parallel and distributed MTL models are introduced. Many areas, including computer vision, bioinformatics, health informatics, speech, natural language processing, web applications and ubiquitous computing, use MTL to improve the performance of the applications involved and some representative works are reviewed. Finally, recent theoretical analyses for MTL are presented.

scholarly journals An approach framework of transfer learning, adversarial training and hierarchical multi-task learning - a case study of disinformation detection with offensive text

2022 ◽  
Vol 2161 (1) ◽  
pp. 012049
Author(s):  
Shravan Chandra ◽  
Bhaskarjyoti Das
Keyword(s):  
Transfer Learning ◽  
Error Feedback ◽  
Task Learning ◽  
Learning Tasks ◽  
Linguistic Patterns ◽  
Social Media Platforms ◽  
Adversarial Training ◽  
Hierarchical Nature ◽  
Definition Of

Abstract With society going online and disinformation getting accepted as a phenomena that we have to live with, there is a growing need to automatically detect offensive text on modern social media platforms. But the lack of enough balanced labeled data, constantly evolving socio-linguistic patterns and ever-changing definition of offensive text make it a challenging task. This is a common pattern witnessed in all disinformation detection tasks such as detection of propaganda, rumour, fake news, hate etc. The work described in this paper improves upon the existing body of techniques by bringing in an approach framework that can surpass the existing benchmarks. Firstly, it addresses the imbalanced and insufficient nature of available labeled dataset. Secondly, learning using relates tasks through multi-task learning has been proved to be an effective approach in this domain but it has the unrealistic requirement of labeled data for all related tasks. The framework presented here suitably uses transfer learning in lieu of multi-task learning to address this issue. Thirdly, it builds a model explicitly addressing the hierarchical nature in the taxonomy of disinformation being detected as that delivers a stronger error feedback to the learning tasks. Finally, the model is made more robust by adversarial training. The work presented in this paper uses offensive text detection as a case study and shows convincing results for the chosen approach. The framework adopted can be easily replicated in other similar learning tasks facing a similar set of challenges.

Adversarial training for few-shot text classification

2021 ◽  
Vol 14 (2) ◽  
pp. 201-214
Author(s):  
Danilo Croce ◽  
Giuseppe Castellucci ◽  
Roberto Basili
Keyword(s):  
Supervised Learning ◽  
Language Processing ◽  
Reproducing Kernel ◽  
Generative Adversarial Networks ◽  
Training Material ◽  
Semantic Classification ◽  
Universal Sentence ◽  
Kernel Hilbert Spaces ◽  
Supervised Methods ◽  
Low Dimensional

In recent years, Deep Learning methods have become very popular in classification tasks for Natural Language Processing (NLP); this is mainly due to their ability to reach high performances by relying on very simple input representations, i.e., raw tokens. One of the drawbacks of deep architectures is the large amount of annotated data required for an effective training. Usually, in Machine Learning this problem is mitigated by the usage of semi-supervised methods or, more recently, by using Transfer Learning, in the context of deep architectures. One recent promising method to enable semi-supervised learning in deep architectures has been formalized within Semi-Supervised Generative Adversarial Networks (SS-GANs) in the context of Computer Vision. In this paper, we adopt the SS-GAN framework to enable semi-supervised learning in the context of NLP. We demonstrate how an SS-GAN can boost the performances of simple architectures when operating in expressive low-dimensional embeddings; these are derived by combining the unsupervised approximation of linguistic Reproducing Kernel Hilbert Spaces and the so-called Universal Sentence Encoders. We experimentally evaluate the proposed approach over a semantic classification task, i.e., Question Classification, by considering different sizes of training material and different numbers of target classes. By applying such adversarial schema to a simple Multi-Layer Perceptron, a classifier trained over a subset derived from 1% of the original training material achieves 92% of accuracy. Moreover, when considering a complex classification schema, e.g., involving 50 classes, the proposed method outperforms state-of-the-art alternatives such as BERT.

scholarly journals A Survey on Contrastive Self-Supervised Learning

Technologies ◽  
2020 ◽  
Vol 9 (1) ◽  
pp. 2
Author(s):  
Ashish Jaiswal ◽  
Ashwin Ramesh Babu ◽  
Mohammad Zaki Zadeh ◽  
Debapriya Banerjee ◽  
Fillia Makedon
Keyword(s):  
Computer Vision ◽  
Supervised Learning ◽  
Language Processing ◽  
Large Scale ◽  
Performance Comparison ◽  
Extensive Review ◽  
Future Directions ◽  
Dominant Component ◽  
Supervised Methods ◽  
The Cost

Self-supervised learning has gained popularity because of its ability to avoid the cost of annotating large-scale datasets. It is capable of adopting self-defined pseudolabels as supervision and use the learned representations for several downstream tasks. Specifically, contrastive learning has recently become a dominant component in self-supervised learning for computer vision, natural language processing (NLP), and other domains. It aims at embedding augmented versions of the same sample close to each other while trying to push away embeddings from different samples. This paper provides an extensive review of self-supervised methods that follow the contrastive approach. The work explains commonly used pretext tasks in a contrastive learning setup, followed by different architectures that have been proposed so far. Next, we present a performance comparison of different methods for multiple downstream tasks such as image classification, object detection, and action recognition. Finally, we conclude with the limitations of the current methods and the need for further techniques and future directions to make meaningful progress.

scholarly journals PKT: fast color-based spatial model for human skin detection

2021 ◽  
Author(s):  
Grace L. Samson ◽  
Joan Lu
Keyword(s):  
Spatial Structure ◽  
Human Skin ◽  
Skin Color ◽  
Skin Detection ◽  
Quad Tree ◽  
Learning Tasks ◽  
Color Detection ◽  
Speed Up ◽  
Image Pixels ◽  
Speed Accuracy

AbstractWe present a new detection method for color-based object detection, which can improve the performance of learning procedures in terms of speed, accuracy, and efficiency, using spatial inference, and algorithm. We applied the model to human skin detection from an image; however, the method can also work for other machine learning tasks involving image pixels. We propose (1) an improved RGB/HSL human skin color threshold to tackle darker human skin color detection problem. (2), we also present a new rule-based fast algorithm (packed k-dimensional tree --- PKT) that depends on an improved spatial structure for human skin/face detection from colored 2D images. We also implemented a novel packed quad-tree (PQT) to speed up the quad-tree performance in terms of indexing. We compared the proposed system to traditional pixel-by-pixel (PBP)/pixel-wise (PW) operation, and quadtree based procedures. The results show that our proposed spatial structure performs better (with a very low false hit rate, very high precision, and accuracy rate) than most state-of-the-art models.

scholarly journals SIMULTANEOUS DEFINITION OF KEY CHARACTERISTICS IN ORDER TO FACILITATE ROBUST DESIGN IN EARLY PRODUCT DEVELOPMENT STAGES

2021 ◽  
Vol 1 ◽  
pp. 2691-2700
Author(s):  
Stefan Goetz ◽  
Dennis Horber ◽  
Benjamin Schleich ◽  
Sandro Wartzack
Keyword(s):  
Product Development ◽  
Robust Design ◽  
Language Processing ◽  
Design Stage ◽  
Concept Design ◽  
Clear Definition ◽  
Implicit Information ◽  
Key Characteristics ◽  
Associated Functions ◽  
Definition Of

AbstractThe success of complex product development projects strongly depends on the clear definition of target factors that allow a reliable statement about the fulfilment of the product requirements. In the context of tolerancing and robust design, Key Characteristics (KCs) have been established for this purpose and form the basis for all downstream activities. In order to integrate the activities related to the KC definition into product development as early as possible, the often vaguely formulated requirements must be translated into quantifiable KCs. However, this is primarily a manual process, so the results strongly depend on the experience of the design engineer.In order to overcome this problem, a novel computer-aided approach is presented, which automatically derives associated functions and KCs already during the definition of product requirements. The approach uses natural language processing and formalized design knowledge to extract and provide implicit information from the requirements. This leads to a clear definition of the requirements and KCs and thus creates a founded basis for robustness evaluation at the beginning of the concept design stage. The approach is exemplarily applied to a window lifter.

scholarly journals A Sequential and Intensive Weighted Language Modeling Scheme for Multi-Task Learning-Based Natural Language Understanding

2021 ◽  
Vol 11 (7) ◽  
pp. 3095
Author(s):  
Suhyune Son ◽  
Seonjeong Hwang ◽  
Sohyeun Bae ◽  
Soo Jun Park ◽  
Jang-Hwan Choi
Keyword(s):  
Natural Language ◽  
Language Processing ◽  
Empirical Investigation ◽  
Natural Language Understanding ◽  
Language Modeling ◽  
Language Understanding ◽  
Task Learning ◽  
Language Representation ◽  
Internal Transfer ◽  
Two Stages

Multi-task learning (MTL) approaches are actively used for various natural language processing (NLP) tasks. The Multi-Task Deep Neural Network (MT-DNN) has contributed significantly to improving the performance of natural language understanding (NLU) tasks. However, one drawback is that confusion about the language representation of various tasks arises during the training of the MT-DNN model. Inspired by the internal-transfer weighting of MTL in medical imaging, we introduce a Sequential and Intensive Weighted Language Modeling (SIWLM) scheme. The SIWLM consists of two stages: (1) Sequential weighted learning (SWL), which trains a model to learn entire tasks sequentially and concentrically, and (2) Intensive weighted learning (IWL), which enables the model to focus on the central task. We apply this scheme to the MT-DNN model and call this model the MTDNN-SIWLM. Our model achieves higher performance than the existing reference algorithms on six out of the eight GLUE benchmark tasks. Moreover, our model outperforms MT-DNN by 0.77 on average on the overall task. Finally, we conducted a thorough empirical investigation to determine the optimal weight for each GLUE task.

scholarly journals Heliceculture as a promising area of agricultural production

2020 ◽  
pp. 33-41
Author(s):  
Ivan Zubar ◽  
Yuliia Onyshchuk
Keyword(s):  
Agricultural Production ◽  
Systematic Approach ◽  
Production Capacity ◽  
World Market ◽  
Economic Science ◽  
Limited Information ◽  
Current State ◽  
Promising Area ◽  
Definition Of ◽  
Effective Development

Purpose. The aim of the article is research of theoretical, organizational and economic aspects of functioning and effective development of farms for growing and processing of snails and substantiation of prospects of heliceculture as a branch of agriculture on the basis of analysis of world experience. Methodology of research. In the course of the research the methods of general scientific (analysis, synthesis, abstraction) and empirical methods (observations, questionnaires, conversations) of economic science are used, which are based on a systematic approach, which allowed to characterize the current state of production and export of heliceculture products, as well as identify key issues in this area of agricultural production. Findings. The concept of “heliceculture” is substantiated and its content is analysed in terms of prospects as a new direction of agricultural production. The historical genesis of the development of traditions of snail products consumption has been studied, as a result of which the first mentions in the history of Ancient Rome, as well as France and Italy have been revealed. An analysis of the dynamics and current state of development of domestic heliculture entrepreneurship, which showed a rapid increase in the number of snail farms and increasing exports of snails to Europe. An overview of the world market for edible snails is made, where there is a noticeable tendency to a gradual annual increase in the consumption of heliculture products. The key elements of the technological process of growing edible snails are analysed, which allowed to systematize a number of basic technological processes and to conclude about the complexity and complexity of this production. The commodity assortment of heliculture has been determined, which includes meat, caviar and snail secretion. The key advantages of Ukraine as a producer and exporter of heliculture products are highlighted, including the availability of labour, proximity to markets, high land supply and dissatisfaction with global demand for these products, which makes it significant for the development of heliculture as an agricultural production. The key problems that hinder the development of snail farming are summarized, namely: legislative unregulation, limited industrial production capacity, lack of diversified processing, limited information and scientific research. The key directions of development of the heliceculture industry are determined, among which: organization of production and marketing heliceculture cooperatives, provision of in-depth processing and year-round uninterrupted production, development of agro-tourism on the basis of snail farms. Originality. A systematic approach to clarifying the definition of “heliceculture” is proposed. On the basis of a thorough study of official statistical information on the state of production of snail products, the importance of heliculture as a promising area of agricultural production is substantiated. Practical value. The obtained research results can be used in the development of an effective concept for the development of the heliculture industry. Key words: heliceculture, heliceculture market, snail farming, small business, family farm.

scholarly journals Comparison of Quantitative PCR (qPCR) Paenibacillus larvae Targeted Assays and Definition of Optimal Conditions for Its Detection/Quantification in Honey and Hive Debris

2018 ◽  
Author(s):  
Franca Rossi ◽  
Carmela Amadoro ◽  
Addolorato Ruberto ◽  
Luciano Ricchiuti
Keyword(s):  
Quantitative Pcr ◽  
Clinical Symptoms ◽  
Sequence Data ◽  
Standard Procedure ◽  
Paenibacillus Larvae ◽  
Optimal Conditions ◽  
Target Species ◽  
Speed Up ◽  
Definition Of ◽  
Culture Dependent

The application of quantitative PCR (qPCR) as a routine method to detect and enumerate Paenibacillus larvae in honey and hive debris could greatly speed up the estimation of prevalence and outbreak risk of the American foulbrood (AFB) disease of Apis mellifera. However, none of the qPCR tests described so far has been officially proposed as a standard procedure for P. larvae detection and enumeration for surveillance purposes. Therefore, in this study inclusivity, exclusivity and sensitivity in detection of P. larvae spores directly in samples of honey and hive debris were re-evaluated for the previously published qPCR methods. To this aim recently acquired P. larvae sequence data were considered to assess inclusivity in silico and more appropriate non-target species were used to verify exclusivity experimentally. This led to the modification of one of the previously described methods resulting in a new test capable to allow the detection of P. larvae spores in honey and hive debris down to 1 CFU/g. The application of the qPCR test optimized in this study can allow to reliably detect and quantify P. larvae in honey and hive debris, thus circumventing the disadvantages of late AFB diagnosis based on clinical symptoms and possible underestimation of spore numbers that is the main drawback of culture-dependent procedures.

DEVELOPMENT OF MODELS FOR DIAGNOSTIC DIABETES BASED ON FUZZY LOGIC METODS

2021 ◽  
pp. 90-95
Author(s):  
Евгений Николаевич Коровин ◽  
Екатерина Ивановна Новикова ◽  
Олег Валерьевич Родионов
Keyword(s):  
Mathematical Modeling ◽  
Decision Making ◽  
Information Technology ◽  
Fuzzy Logic ◽  
Human Body ◽  
Patient Information ◽  
Decision Making Process ◽  
Speed Up ◽  
Definition Of

В статье рассматриваются разработки методов интеллектуальной поддержки процесса диагностики сахарного диабета, а также определение его типа. В последние годы количество людей, страдающих данным заболеванием, неуклонно растет, а без своевременной диагностики эта патология может нанести огромный вред организму человека. Сахарный диабет 1 типа опасен тем, что в основном возникает у людей молодого возраста. Оперативное обнаружение диабета, а также определение его типа, поможет не только избежать возможных осложнений, но и в некоторых случаях предотвратить смерть пациента. Информационные технологии все чаще используются в различных сферах деятельности для разработки новых или совершенствования существующих методов обработки данных, особенно это можно заметить в сфере медицины. В настоящее время врач самостоятельно ставит диагноз, основываясь на результатах различных анализов, однако, для ускорения процесса принятия решения, можно воспользоваться методами математического моделирования, а именно: моделями диагностики диабета на основе нечеткой логики. Для наибольшего удобства данный способ распознавания заболевания впоследствии можно реализовать в информационно-программное обеспечение, которое сможет еще больше увеличить эффективность и скорость распознавания патологии The article discusses the issues of the incidence of diabetes in the population, in particular, the definition of its type. In recent years, the number of people suffering from this disease has been steadily growing, and without timely diagnosis, this pathology can cause enormous harm to the human body. Prompt detection of diabetes, as well as determination of its type, will help not only avoid possible complications, but also in some cases prevent the death of the patient. Information technology is increasingly being used in various fields of activity to develop new or improve existing methods of data processing, especially in the field of medicine. Currently, the doctor independently makes a diagnosis based on the results of various analyzes, however, to speed up the decision-making process, you can use the methods of mathematical modeling, namely, models of diabetes diagnostics based on fuzzy logic. For the greatest convenience, this method of disease recognition can subsequently be implemented in information software, which can further increase the efficiency and speed of pathology recognition

Informatics and Socio-Technical Challenges when Designing Solutions for Integrated eCare

2017 ◽  
pp. 261-289
Author(s):  
Sabine Koch ◽  
Maria Hägglund ◽  
Isabella Scandurra
Keyword(s):  
Integrated Care ◽  
Case Studies ◽  
Health Informatics ◽  
Information Needs ◽  
System Development ◽  
Collaborative Work ◽  
Successful Implementation ◽  
Data Sets ◽  
Technical Challenges ◽  
Definition Of

The central role of eHealth to enable the successful implementation of integrated care is commonly acknowledged today. This is easier said than done. To provide correct, understandable, and timely information at the point of need and to facilitate communication and decision support for a network of actors with different prerequisites and needs are some of the big challenges of integrated care. This book chapter focuses on the specific challenges related to informatics and socio-technical issues when designing solutions for integrated eCare. Methods for requirements elicitation, evaluation, and system development using user-centred design in collaborative environments involving a variety of stakeholders are presented. Case studies in homecare of older patients, in the care of stroke patients, and regarding citizen eHealth services in general illustrate the application of these methods. Possible solutions and pitfalls are discussed based on the experiences drawn from the case studies. To address the main informatics and socio-technical challenges in integrated eCare, namely informatics-supported collaborative work and to provide coordinated continuity for the patient, top-down activities such as health informatics standardisation, and bottom-up activities resulting in the definition of concrete patient journey descriptions, interaction points, information needs (that can be transformed into standardised data sets), as well as visualisation and interaction patterns need to go hand in hand.

Sign in / Sign up

Export Citation Format

Share Document