scholarly journals A Survey of Domain Knowledge Elicitation in Applied Machine Learning

2021 ◽  
Vol 5 (12) ◽  
pp. 73
Author(s):  
Daniel Kerrigan ◽  
Jessica Hullman ◽  
Enrico Bertini
Keyword(s):  
Machine Learning ◽  
Domain Knowledge ◽  
Development Process ◽  
Model Development ◽  
Knowledge Elicitation ◽  
Future Directions ◽  
Domain Experts ◽  
Applied Machine Learning ◽  
Elicitation Process ◽  
Model Development Process

Eliciting knowledge from domain experts can play an important role throughout the machine learning process, from correctly specifying the task to evaluating model results. However, knowledge elicitation is also fraught with challenges. In this work, we consider why and how machine learning researchers elicit knowledge from experts in the model development process. We develop a taxonomy to characterize elicitation approaches according to the elicitation goal, elicitation target, elicitation process, and use of elicited knowledge. We analyze the elicitation trends observed in 28 papers with this taxonomy and identify opportunities for adding rigor to these elicitation approaches. We suggest future directions for research in elicitation for machine learning by highlighting avenues for further exploration and drawing on what we can learn from elicitation research in other fields.

scholarly journals Value Creation in Collaborative Analysis Model Development Processes

2014 ◽  
Author(s):  
Göknur Sirin ◽  
Torgeir Welo ◽  
Bernard Yannou ◽  
Eric Landel
Keyword(s):  
Domain Knowledge ◽  
Development Process ◽  
Coherent Control ◽  
Early Stage ◽  
Current Model ◽  
Model Development ◽  
Manufacturing Companies ◽  
Analysis Model ◽  
Pull Control ◽  
Model Development Process

Integration and coordination of engineering analysis model is a vast development field in the context of complex product development. Engineers’ siloed way of working in combination with lack of efficiency in current model development process may cause inconsistency based on model interfaces, human errors, miscommunication between teams and misinterpretations. In lean terms, this may create multiple wastes, including waiting, overproduction leading to excess inventory, unnecessary processing and may be the most harmful: defects (e.g., incorrect models) with rework consequences. Hence, product manufacturing companies must establish effective processes to add value throughout the multidisciplinary distributed modeling environment. The goal of this paper is to propose a pull-control model development process, providing model architecture integration and coherent control in early design phase. This paper proposes also an appropriate reuse strategy; this allows for utilizing plug-and-play type modular product models managed through a single-source of authority concept. A pull-control development process helps prevent potential rework arising from inconsistencies related to definitions, know-how and stakeholders communication at an early stage of the design process. Also, the proposed black box models reuse strategy helps reduce human-related error such as lack of domain knowledge, experience and misinterpretations. The proposed method is used to identify and visualize potential improvement in terms of increased model transparency and reuse when transforming from the present to the suggested future modeling strategy. The research has been conducted by synthesizing findings from a literature review, in combination with observations and analysis of current analysis model development practices within the automotive OEM Renault in France.

scholarly journals Strategic Business Model Development in Ecosystems

2021 ◽  
Author(s):  
◽  
Dominik Mann
Keyword(s):  
Business Model ◽  
Development Process ◽  
Business Models ◽  
Model Development ◽  
Entrepreneurial Ventures ◽  
And Performance ◽  
Integration Strategies ◽  
Model Components ◽  
Business Model Development ◽  
Model Development Process

<p>Designing and strategically developing viable business models is vital for value creation and capture and in turn for the survival and performance of entrepreneurial ventures. However, the widely held firm-centric and static business model perspective appears inadequate to reflect the realities of increasingly blurred industry boundaries, interconnected economies, and the resulting collapse of incumbent value chains. This PhD thesis adds understanding of the dynamic business model development process from an ecosystem perspective. The evolution of ten entrepreneurial ventures’ business models was documented and investigated through longitudinal in-depth case studies over twelve months. Analysing and comparing the cases revealed strategies that resulted in the development of effective interactive structures and robust value co-creation and capture mechanisms. The development of interactive structures, i.e. firm-ecosystem fits, was either supported by a focused or diversified ecosystem integration approach underpinned by heterogeneous interdependencies of value proposition and business model components across ecosystems. The obtained insights allowed the derivation of sets of capabilities that supported the business model development process and enhanced entrepreneurial ventures’ chances of survival. The findings have several implications for advancements of the business model theory. In particular they indicate what integration strategies can inform entrepreneurs’ and managers’ business model design and execution strategies for operating in increasingly complex ecosystems.</p>

Exposing System and Model Disparity and Agreement Using Wavelets

2018 ◽  
Vol 3 (2) ◽  
Author(s):  
Andrew D. Atkinson ◽  
Raymond R. Hill ◽  
Joseph J. Pignatiello ◽  
G. Geoffrey Vining ◽  
Edward D. White ◽  
...  
Keyword(s):  
Development Process ◽  
Model Development ◽  
Model Verification ◽  
Verification And Validation ◽  
High Dimensional ◽  
Model Data ◽  
Accurate Representation ◽  
Model Verification And Validation ◽  
Model Development Process ◽  
Functional Output

Model verification and validation (V&V) remain a critical step in the simulation model development process. A model requires verification to ensure that it has been correctly transitioned from a conceptual form to a computerized form. A model also requires validation to substantiate the accurate representation of the system it is meant to simulate. Validation assessments are complex when the system and model both generate high-dimensional functional output. To handle this complexity, this paper reviews several wavelet-based approaches for assessing models of this type and introduces a new concept for highlighting the areas of contrast and congruity between system and model data. This concept identifies individual wavelet coefficients that correspond to the areas of discrepancy between the system and model.

Design of the Engineering-Oriented Equipment Requirement Demonstration Model Base

2014 ◽  
Vol 513-517 ◽  
pp. 3612-3616
Author(s):  
Yan Ping Fan ◽  
Qi Sheng Guo ◽  
Jie Bai ◽  
Jin Liang Wang
Keyword(s):  
Design Pattern ◽  
Development Process ◽  
Model Development ◽  
Model Management ◽  
Model Base ◽  
Model Driven ◽  
Management Technology ◽  
Application Requirements ◽  
Model Development Process

Aiming at the engineering-oriented application requirements of the equipment requirement demonstration, the demonstration process driven by models is regarded as the essential goal. Firstly, the activities and the modeling requirements of the equipment requirement demonstration are analyzed in detail. Thereafter, the model system of the equipment requirement demonstration is built. Focusing on the design of the model base, the model-driven model development process and the design pattern of the models based on MVC are put forward and discussed emphatically. The VV&A mechanism is designed to improve the quality of the models. According to the management requirements of the model base, the model management technology based on ontology is put forward, and then the run mechanism of the model base is studied. Throughout all these designs, the models simulating the equipment requirement demonstration activities can be understood and reused better, and can satisfy the elementary requirements building the toolkits based on these models. Simultaneity, the equipment requirement demonstration activities can be regulated and formalized with the toolkits.

A Wavelet Synthesis Technique for Creating Realistic Virtual Environment Sounds

2002 ◽  
Vol 11 (5) ◽  
pp. 493-507 ◽  
Author(s):  
Nadine E. Miner ◽  
Thomas P. Caudell
Keyword(s):  
Virtual Environments ◽  
Virtual Environment ◽  
Development Process ◽  
Model Development ◽  
Sound Synthesis ◽  
Model Parameters ◽  
Experiment Validation ◽  
Environment Simulation ◽  
A New Technique ◽  
Model Development Process

This paper describes a new technique for synthesizing realistic sounds for virtual environments. The four-phase technique described uses wavelet analysis to create a sound model. Parameters are extracted from the model to provide dynamic sound synthesis control from a virtual environment simulation. Sounds can be synthesized in real time using the fast inverse wavelet transform. Perceptual experiment validation is an integral part of the model development process. This paper describes the four-phase process for creating the parameterized sound models. Several developed models and perceptual experiments for validating the sound synthesis veracity are described. The developed models and results demonstrate proof of the concept and illustrate the potential of this approach.

scholarly journals GAMES: A dynamic model development workflow for rigorous characterization of synthetic genetic systems

2021 ◽  
Author(s):  
Kate Dray ◽  
Joseph J Muldoon ◽  
Niall J Mangan ◽  
Neda Bagheri ◽  
Joshua Nathaniel Leonard
Keyword(s):  
Development Process ◽  
Ad Hoc ◽  
Dynamic Models ◽  
Model Development ◽  
Human In The Loop ◽  
Potential Impact ◽  
Model Development Process ◽  
Reducing Potential

Mathematical modeling is invaluable for advancing understanding and design of synthetic biological systems. However, the model development process is complicated and often unintuitive, requiring iteration on various computational tasks and comparisons with experimental data. Ad hoc model development can pose a barrier to reproduction and critical analysis of the development process itself, reducing potential impact and inhibiting further model development and collaboration. To help practitioners manage these challenges, we introduce GAMES: a workflow for Generation and Analysis of Models for Exploring Synthetic systems that includes both automated and human-in-the-loop processes. We systematically consider the process of developing dynamic models, including model formulation, parameter estimation, parameter identifiability, experimental design, model reduction, model refinement, and model selection. We demonstrate the workflow with a case study on a chemically responsive transcription factor. The generalizable workflow presented in this tutorial can enable biologists to more readily build and analyze models for various applications.

scholarly journals Explainable deep convolutional learning for intuitive model development by non–machine learning domain experts

2020 ◽  
Vol 6 ◽  
Author(s):  
Sundaravelpandian Singaravel ◽  
Johan Suykens ◽  
Hans Janssen ◽  
Philipp Geyer
Keyword(s):  
Machine Learning ◽  
Model Development ◽  
Design Stage ◽  
Domain Experts ◽  
Model Generalization ◽  
Computational Speed ◽  
Interpretable Models ◽  
Low Dimensional ◽  
Fully Connected ◽  
T Distribution

Abstract During the design stage, quick and accurate predictions are required for effective design decisions. Model developers prefer simple interpretable models for high computation speed. Given that deep learning (DL) has high computational speed and accuracy, it will be beneficial if these models are explainable. Furthermore, current DL development tools simplify the model development process. The article proposes a method to make the learning of the DL model explainable to enable non–machine learning (ML) experts to infer on model generalization and reusability. The proposed method utilizes dimensionality reduction (t-Distribution Stochastic Neighbour Embedding) and mutual information (MI). Results indicate that the convolutional layers capture design-related interpretations, and the fully connected layer captures performance-related interpretations. Furthermore, the global geometric structure within a model that generalized well and poorly is similar. The key difference indicating poor generalization is smoothness in the low-dimensional embedding. MI enables quantifying the reason for good and poor generalization. Such interpretation adds more information on model behaviour to a non-ML expert.

Sign in / Sign up

Export Citation Format

Share Document