scholarly journals Masking Orchestration: Multi-Task Pretraining for Multi-Role Dialogue Representation Learning

2020 ◽  
Vol 34 (05) ◽  
pp. 9217-9224
Author(s):  
Tianyi Wang ◽  
Yating Zhang ◽  
Xiaozhong Liu ◽  
Changlong Sun ◽  
Qiong Zhang
Keyword(s):  
Knowledge Integration ◽  
Question Answering ◽  
Representation Learning ◽  
Essential Information ◽  
External Knowledge ◽  
Learning Tasks ◽  
Wide Range ◽  
Specific Learning

Multi-role dialogue understanding comprises a wide range of diverse tasks such as question answering, act classification, dialogue summarization etc. While dialogue corpora are abundantly available, labeled data, for specific learning tasks, can be highly scarce and expensive. In this work, we investigate dialogue context representation learning with various types unsupervised pretraining tasks where the training objectives are given naturally according to the nature of the utterance and the structure of the multi-role conversation. Meanwhile, in order to locate essential information for dialogue summarization/extraction, the pretraining process enables external knowledge integration. The proposed fine-tuned pretraining mechanism is comprehensively evaluated via three different dialogue datasets along with a number of downstream dialogue-mining tasks. Result shows that the proposed pretraining mechanism significantly contributes to all the downstream tasks without discrimination to different encoders.

scholarly journals Clustering-Based Relational Unsupervised Representation Learning with an Explicit Distributed Representation

2017 ◽  
Author(s):  
Sebastijan Dumancic ◽  
Hendrik Blockeel
Keyword(s):  
Relational Learning ◽  
Representation Learning ◽  
Relational Data ◽  
Distributed Representation ◽  
Learning Tasks ◽  
Wide Range ◽  
Lower Complexity ◽  
Classification Tasks ◽  
Latent Representations

The goal of unsupervised representation learning is to extract a new representation of data, such that solving many different tasks becomes easier. Existing methods typically focus on vectorized data and offer little support for relational data, which additionally describes relationships among instances. In this work we introduce an approach for relational unsupervised representation learning. Viewing a relational dataset as a hypergraph, new features are obtained by clustering vertices and hyperedges. To find a representation suited for many relational learning tasks, a wide range of similarities between relational objects is considered, e.g. feature and structural similarities. We experimentally evaluate the proposed approach and show that models learned on such latent representations perform better, have lower complexity, and outperform the existing approaches on classification tasks.

scholarly journals Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge

2022 ◽  
Vol 40 (1) ◽  
pp. 1-33
Author(s):  
Yang Deng ◽  
Yuexiang Xie ◽  
Yaliang Li ◽  
Min Yang ◽  
Wai Lam ◽  
...  
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Question Answering ◽  
Representation Learning ◽  
Attention Mechanism ◽  
Convolutional Network ◽  
Structure Information ◽  
Dialog Systems ◽  
External Knowledge ◽  
Knowledge Based

Answer selection, which is involved in many natural language processing applications, such as dialog systems and question answering (QA), is an important yet challenging task in practice, since conventional methods typically suffer from the issues of ignoring diverse real-world background knowledge. In this article, we extensively investigate approaches to enhancing the answer selection model with external knowledge from knowledge graph (KG). First, we present a context-knowledge interaction learning framework, Knowledge-aware Neural Network, which learns the QA sentence representations by considering a tight interaction with the external knowledge from KG and the textual information. Then, we develop two kinds of knowledge-aware attention mechanism to summarize both the context-based and knowledge-based interactions between questions and answers. To handle the diversity and complexity of KG information, we further propose a Contextualized Knowledge-aware Attentive Neural Network, which improves the knowledge representation learning with structure information via a customized Graph Convolutional Network and comprehensively learns context-based and knowledge-based sentence representation via the multi-view knowledge-aware attention mechanism. We evaluate our method on four widely used benchmark QA datasets, including WikiQA, TREC QA, InsuranceQA, and Yahoo QA. Results verify the benefits of incorporating external knowledge from KG and show the robust superiority and extensive applicability of our method.

scholarly journals Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering

2020 ◽  
Vol 34 (05) ◽  
pp. 8449-8456 ◽  
Author(s):  
Shangwen Lv ◽  
Daya Guo ◽  
Jingjing Xu ◽  
Duyu Tang ◽  
Nan Duan ◽  
...  
Keyword(s):  
Question Answering ◽  
Structural Information ◽  
Representation Learning ◽  
Knowledge Bases ◽  
Knowledge Sources ◽  
Convolutional Network ◽  
External Knowledge ◽  
Relational Structures ◽  
Word Representation ◽  
Final Answer

Commonsense question answering aims to answer questions which require background knowledge that is not explicitly expressed in the question. The key challenge is how to obtain evidence from external knowledge and make predictions based on the evidence. Recent studies either learn to generate evidence from human-annotated evidence which is expensive to collect, or extract evidence from either structured or unstructured knowledge bases which fails to take advantages of both sources simultaneously. In this work, we propose to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence. Specifically, we extract evidence from both structured knowledge base (i.e. ConceptNet) and Wikipedia plain texts. We construct graphs for both sources to obtain the relational structures of evidence. Based on these graphs, we propose a graph-based approach consisting of a graph-based contextual word representation learning module and a graph-based inference module. The first module utilizes graph structural information to re-define the distance between words for learning better contextual word representations. The second module adopts graph convolutional network to encode neighbor information into the representations of nodes, and aggregates evidence with graph attention mechanism for predicting the final answer. Experimental results on CommonsenseQA dataset illustrate that our graph-based approach over both knowledge sources brings improvement over strong baselines. Our approach achieves the state-of-the-art accuracy (75.3%) on the CommonsenseQA dataset.

scholarly journals Table2Vec-automated universal representation learning of enterprise data DNA for benchmarkable and explainable enterprise data science

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Longbing Cao ◽  
Chengzhang Zhu
Keyword(s):  
Data Science ◽  
Representation Learning ◽  
Heterogeneous Data ◽  
Learning Goals ◽  
Customer Data ◽  
Domain Specific ◽  
External Data ◽  
Learning Tasks ◽  
Universal Representation ◽  
Specific Learning

AbstractEnterprise data typically involves multiple heterogeneous data sources and external data that respectively record business activities, transactions, customer demographics, status, behaviors, interactions and communications with the enterprise, and the consumption and feedback of its products, services, production, marketing, operations, and management, etc. They involve enterprise DNA associated with domain-oriented transactions and master data, informational and operational metadata, and relevant external data. A critical challenge in enterprise data science is to enable an effective ‘whole-of-enterprise’ data understanding and data-driven discovery and decision-making on all-round enterprise DNA. Accordingly, here we introduce a neural encoder Table2Vec for automated universal representation learning of entities such as customers from all-round enterprise DNA with automated data characteristics analysis and data quality augmentation. The learned universal representations serve as representative and benchmarkable enterprise data genomes (similar to biological genomes and DNA in organisms) and can be used for enterprise-wide and domain-specific learning tasks. Table2Vec integrates automated universal representation learning on low-quality enterprise data and downstream learning tasks. Such automated universal enterprise representation and learning cannot be addressed by existing enterprise data warehouses (EDWs), business intelligence and corporate analytics systems, where ‘enterprise big tables’ are constructed with reporting and analytics conducted by specific analysts on respective domain subjects and goals. It addresses critical limitations and gaps of existing representation learning, enterprise analytics and cloud analytics, which are analytical subject, task and data-specific, creating analytical silos in an enterprise. We illustrate Table2Vec in characterizing all-round customer data DNA in an enterprise on complex heterogeneous multi-relational big tables to build universal customer vector representations. The learned universal representation of each customer is all-round, representative and benchmarkable to support both enterprise-wide and domain-specific learning goals and tasks in enterprise data science. Table2Vec significantly outperforms the existing shallow, boosting and deep learning methods typically used for enterprise analytics. We further discuss the research opportunities, directions and applications of automated universal enterprise representation and learning and the learned enterprise data DNA for automated, all-purpose, whole-of-enterprise and ethical machine learning and data science.

scholarly journals Bridging the gap: integrating external knowledge from open innovation platforms

2021 ◽  
Vol 1 (7) ◽  
Author(s):  
Marion A. Weissenberger-Eibl ◽  
Tim Hampel
Keyword(s):  
Open Innovation ◽  
Knowledge Integration ◽  
Organizational Factors ◽  
Project Teams ◽  
Negative Attitudes ◽  
Organizational Resistance ◽  
External Knowledge ◽  
Positive Effects ◽  
Innovation Activities ◽  
Project Level

AbstractWhile external knowledge has the potential to benefit a firm’s innovation activities, research shows that the positive effects of a more open model of innovation do not come naturally. This paper draws on the project level to explore the organizational factors that hamper or impede the integration of external knowledge sourced from an open innovation platform and to suggest interventions to overcome these barriers. While open innovation is mainly discussed as a concept that resides at the level of the organization, this paper draws on the project level to contribute to a multi-level understanding of open innovation and to offer a deeper understanding of the challenges project teams face, when integrating external knowledge. To investigate occurring barriers, four cases of external knowledge integration within a multinational corporation are analyzed. The results show that due to the external nature of the knowledge, an additional effort of project teams is required such as forming alliances with key individuals and changing negative attitudes towards external sources to overcome organizational resistance. Theoretical as well as practical implications are discussed.

A content spectral-based text representation

2021 ◽  
pp. 1-12
Author(s):  
Melesio Crespo-Sanchez ◽  
Ivan Lopez-Arevalo ◽  
Edwin Aldana-Bobadilla ◽  
Alejandro Molina-Villegas
Keyword(s):  
Machine Learning ◽  
Text Analysis ◽  
Question Answering ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Text Representation ◽  
Feature Vectors ◽  
Learning Tasks ◽  
Semantic Component ◽  
Vector Representations

In the last few years, text analysis has grown as a keystone in several domains for solving many real-world problems, such as machine translation, spam detection, and question answering, to mention a few. Many of these tasks can be approached by means of machine learning algorithms. Most of these algorithms take as input a transformation of the text in the form of feature vectors containing an abstraction of the content. Most of recent vector representations focus on the semantic component of text, however, we consider that also taking into account the lexical and syntactic components the abstraction of content could be beneficial for learning tasks. In this work, we propose a content spectral-based text representation applicable to machine learning algorithms for text analysis. This representation integrates the spectra from the lexical, syntactic, and semantic components of text producing an abstract image, which can also be treated by both, text and image learning algorithms. These components came from feature vectors of text. For demonstrating the goodness of our proposal, this was tested on text classification and complexity reading score prediction tasks obtaining promising results.

scholarly journals Transcriptome-wide Identification and Characterization of microRNAs and Their Targets in a Highly Adaptable Conifer Platycladus orientalis

2022 ◽  
Vol 147 (1) ◽  
pp. 7-17
Author(s):  
Ying Yang ◽  
Xian-Ge Hu ◽  
Bingsong Zheng ◽  
Yue Li ◽  
Tongli Wang ◽  
...  
Keyword(s):  
Stress Tolerance ◽  
Leucine Zipper ◽  
Class Iii ◽  
Conifer Species ◽  
Essential Information ◽  
Sequencing Platform ◽  
Platycladus Orientalis ◽  
Wide Range ◽  
Identification And Characterization

MicroRNAs (miRNAs) are short noncoding RNAs (20–25 nucleotides) that regulate gene expression posttranscriptionally. However, identification and characterization of miRNAs remain limited for conifer species. In this study, we applied transcriptome-wide miRNAs sequencing to a conifer species Platycladus orientalis, which is highly adaptable to a wide range of environmental adversities, including drought, barren soil, and mild salinity. A total of 17,181,542 raw reads were obtained from the Illumina sequencing platform; 31 conserved and 91 novel miRNAs were identified, and their unique characteristics were further analyzed. Ten randomly selected miRNAs were validated by quantificational real-time polymerase chain reaction. Through miRNA target predictions based on psRNATarget, 2331 unique mRNAs were predicted to be targets of P. orientalis miRNAs that involved in 187 metabolic pathways in KEGG database. These targets included not only important transcription factors (e.g., class III homeodomain leucine zipper targeted by por-miR166d) but also indispensable nontranscriptional factor proteins (i.e., por-miR482a-3p regulated nucleotide-binding site leucine-rich repeat protein). Interestingly, six miRNAs (por-miR16, -miR44, -miR60-5p, -miR69–3p, -miR166b-5p, and -miR395c) were found in adaptation-related pathways (e.g., drought), indicating their possible involved in this species’ stress-tolerance characteristics. The present study provided essential information for understanding the regulatory role of miRNAs in P. orientalis and sheds light on their possible use in tree improvement for stress tolerance.

A Critical Review of Organizing Knowledge Management for Innovation

2022 ◽  
Vol 8 (2) ◽  
pp. 7-15
Author(s):  
Asa Romeo Asa ◽  
Harold Campbell ◽  
Johanna Pangeiko Nautwima
Keyword(s):  
Knowledge Management ◽  
Competitive Advantage ◽  
Diffusion Of Innovation ◽  
Business Intelligence ◽  
Network Dynamics ◽  
Knowledge Networks ◽  
External Knowledge ◽  
Knowledge Processes ◽  
Wide Range ◽  
Integrated Knowledge

This study critically reviews the literature that demonstrates the relevance of knowledge management process and business intelligence, as well as the challenges arising when it comes to organising for innovation in today’s business organisations. Hence, the to attain desired innovation it is important to integrate business intelligence (BI) and knowledge management (KM) for the diffusion of innovation. Hence, importance of integrating business intelligence (BI) and knowledge management (KM) for the diffusion of innovation. Organisations’ innovation dynamics and knowledge processes that lead competitive advantage of organisations are examined. Literature points that many organisations rely on individual employees’ knowledge and skills. As a result, information systems that enable knowledge management (KM) as a critical tool for gaining a competitive advantage (Campbell, 2012). The seminal argument in this study is that knowledge diffusion and knowledge externalities are the main drive of increase in economy. As a result, this is expected to be a win-win value proposition for such organisations integrating business intelligence and knowledge management. However, owing to changing business conditions and the rapidity of technological development, as well as the rising expenses involved with carrying out R&D operations in many of these organisations, maintaining competitive advantage through internal R&D alone is becoming increasingly challenging. The importance of innovation processes and network dynamics in the context of Integrated Knowledge Networks is explored, which provide feasible possibilities for utilising innovation as an interactive process as well as knowledge processes for creating business intelligence in organisations. Due to the challenges of organising for innovation, the organisations figured to rely on “Open innovation” approach to intentionally seek out unique knowledge and information outside of their organisational bounds. This study also discusses the challenges that organisations hurdle on in managing inter-organizational cooperation because of external knowledge sourcing techniques (Campbell, 2009). This is due, in part, to the fact that they span a wide range of organisations, people, and resources, as well as the interactions that exist between them. The creative processes and network dynamics are facilitated by an architecture that blends organisational and technical aspects in Integrated Knowledge Networks. Hence, the study focuses on twofold to sourcing external knowledge in particular: learning from international business environments and corporate venturing strategy for corporate incubators.

Applied Novel Quality Check Method for PVT Data with High Impurities Using Various Samples from Malaysian Fields

2021 ◽  
Author(s):  
Luky Hendraningrat ◽  
Intan Khalida Salleh
Keyword(s):  
Molecular Weight ◽  
Equation Of State ◽  
Gas Phase ◽  
Oil And Gas ◽  
Comprehensive Evaluation ◽  
Gas Condensate ◽  
Fluid Composition ◽  
Essential Information ◽  
Pvt Data ◽  
Wide Range

Abstract PVT analysis of reservoir fluid samples provides essential information for determining hydrocarbon in place, depletion strategy, and hydrocarbon flowability. Hence, quality checking (QC) is necessary to ensure the best representative sample for further analysis. Recently, a novel tool based on Equation of State (EOS) was introduced to tackle the limitation of the Hoffmann method for surface samples with high impurities and heavier components. This paper presents comprehensively evaluating a novel EOS-based method using various PVT data from Malaysian fields. Numerous PVT separator samples from 30 fields with various reservoir fluids (Black Oil, Volatile, and Gas Condensate) were carried out and evaluated. The impurities contain a wide range of up to 60%. The 2-phase P-T (pressure and temperature) diagram of each oil and gas phase before recombination was calculated using PVT software based on Equation of State (EOS). The 2-phase P-T diagram was created and observed the intersection point as calculated equilibrium at separator conditions. Once it is observed and compared with written separator condition in the laboratory report and observed its deviation. Eventually, the result will be compared with the Hoffmann method. The Hoffmann method is well-known as a traditional QC method that was initially developed using gas condensate PVT data to identify possible errors in measured separator samples. If the sample has high impurities and/or heavier components, the Hoffmann method will only show a straight line to the lighter components and those impurities and heavier components will be an outlier that engineers will misinterpret that it has errors and cannot be used for further analysis such PVT characterization. The QC using EOS-based were conducted using actual fields data. It shows potential as novel QC tools but observed only less than 10% of data with complete information that can meet intersection points located precisely similar with reported in the laboratory. There is some investigation and evaluation of the EOS-based QC method. First, most of the molecular weight of the heavier fluid composition of gas and oil phase was not reported or used assumptions especially when its mole fraction is not zero. Second, properties of heavier components of the oil phase (molecular weight and specific gravity) were not measured and assumed similar as wellstream. Third, pressure and temperature data are inconsistent between the oil and gas phase at the separator condition. This study can provide improvement in laboratory measurement quality and help engineers to have a better understanding of PVT Report, essential data requirements, and assumptions used in the laboratory. Nevertheless, the Hoffmann method can be used as an inexpensive QC tool because it can be generated in a spreadsheet without a PVT software license. Both combination techniques can provide a comprehensive evaluation for separator samples with high impurities before identifying representative fluid for further analysis.

Sign in / Sign up

Export Citation Format

Share Document