Managing Machine Learning Workflow Components

Machine Learning Workflows (MLWfs) have become an essential and disruptive approach in problem-solving over several industries. However, the development process of MLWfs may be complex, time-consuming, and error-prone. To handle this problem, we introduce machine learning workflow management (MLWfM) as a technique to aid the development and reuse of MLWfs and their components through three aspects: representation, execution, and creation. We introduce our approach to structure MLWfs’ components and metadata in order to aid component retrieval and reuse of new MLWfs. We also consider the execution of these components within a tool. A hybrid knowledge representation, called Hyperknowledge, frames our methodology, supporting the three MLWfM’s aspects. To validate our approach, we show a practical use case in the Oil & Gas industry. In addition, to evaluate the feasibility of the proposed technique, we create a dataset of MLWfs executions and discuss the MLWfM’s performance in loading and querying this dataset.

Download Full-text

Paper2Wire – A Case Study of User-Centred Development of Machine Learning Tools for UX Designers

i-com ◽

10.1515/icom-2021-0002 ◽

2021 ◽

Vol 20 (1) ◽

pp. 19-32

Author(s):

Daniel Buschek ◽

Charlotte Anlauff ◽

Florian Lachner

Keyword(s):

Machine Learning ◽

Development Process ◽

User Study ◽

Concept Development ◽

Lessons Learned ◽

Design Tool ◽

Learning Tools ◽

Interface Elements ◽

Industry Partner

Abstract This paper reflects on a case study of a user-centred concept development process for a Machine Learning (ML) based design tool, conducted at an industry partner. The resulting concept uses ML to match graphical user interface elements in sketches on paper to their digital counterparts to create consistent wireframes. A user study (N=20) with a working prototype shows that this concept is preferred by designers, compared to the previous manual procedure. Reflecting on our process and findings we discuss lessons learned for developing ML tools that respect practitioners’ needs and practices.

Download Full-text

A machine learning approach to estimate product costs in the early product design phase: a use case from the automotive industry

Procedia CIRP ◽

10.1016/j.procir.2021.05.137 ◽

2021 ◽

Vol 100 ◽

pp. 643-648

Author(s):

Frank Bodendorf ◽

Jörg Franke

Keyword(s):

Machine Learning ◽

Product Design ◽

Automotive Industry ◽

Learning Approach ◽

Use Case ◽

Design Phase ◽

Machine Learning Approach ◽

Product Costs

Download Full-text

Construction of a quality model for machine learning systems

Software Quality Journal ◽

10.1007/s11219-021-09557-y ◽

2021 ◽

Author(s):

Julien Siebert ◽

Lisa Joeckel ◽

Jens Heidrich ◽

Adam Trendowicz ◽

Koji Nakamichi ◽

...

Keyword(s):

Machine Learning ◽

Lessons Learned ◽

Training Data ◽

Use Case ◽

Construction Process ◽

Quality Model ◽

Quality Models ◽

Quality Properties ◽

Reference Quality ◽

Industrial Use

AbstractNowadays, systems containing components based on machine learning (ML) methods are becoming more widespread. In order to ensure the intended behavior of a software system, there are standards that define necessary qualities of the system and its components (such as ISO/IEC 25010). Due to the different nature of ML, we have to re-interpret existing qualities for ML systems or add new ones (such as trustworthiness). We have to be very precise about which quality property is relevant for which entity of interest (such as completeness of training data or correctness of trained model), and how to objectively evaluate adherence to quality requirements. In this article, we present how to systematically construct quality models for ML systems based on an industrial use case. This quality model enables practitioners to specify and assess qualities for ML systems objectively. In addition to the overall construction process described, the main outcomes include a meta-model for specifying quality models for ML systems, reference elements regarding relevant views, entities, quality properties, and measures for ML systems based on existing research, an example instantiation of a quality model for a concrete industrial use case, and lessons learned from applying the construction process. We found that it is crucial to follow a systematic process in order to come up with measurable quality properties that can be evaluated in practice. In the future, we want to learn how the term quality differs between different types of ML systems and come up with reference quality models for evaluating qualities of ML systems.

Download Full-text