Real-time combination of material flow simulation, digital twins of manufacturing cells, an AGV and a mixed-reality application

Die hier dargestellten Arbeiten wurden im Applikationszentrum Industrie 4.0 als ein Teil des Showcases „Autonome Produktion“ durchgeführt. Das Fraunhofer IPA und das IFF haben gemeinsam eine echtzeitnahe Materialflusssimulation mit in-situ Visualisierung entwickelt, die automatisch ein digitales Abbild der Produktion am Beispiel einer verketteten Montagelinie erzeugt und über eine Deckeninstallation aus 6x2 Projektoren auf den Hallenboden projiziert. Durch diese in-situ Visualisierung können Abweichungen vom geplanten Ablauf in Echtzeit sichtbar gemacht, Probleme behoben und Stillstandszeiten verringert werden, bevor sie auftreten.   The work presented in this article was carried out in the Application Center for Industrie 4.0 as part of the showcase “Autonomous Production“. The IPA and IFF have jointly developed a material flow simulation close to real-time with in-situ visualization that automati cally generates a digital image of the production (e.g., a chained assembly line) and projects a 2D factory layout onto the floor using a 6x2 projector ceiling installation. In-situ visualization makes it possible to visualize the need for optimization and the prognosis of disruptive effects in the simulation model in real-time. Problems can be solved before they occur and downtime can be reduced.

Download Full-text

Development of environment design support mixed reality system capable of environment estimation using deep learning

Impact ◽

10.21820/23987073.2020.2.9 ◽

2020 ◽

Vol 2020 (2) ◽

pp. 9-11

Author(s):

Tomohiro Fukuda

Keyword(s):

Deep Learning ◽

Real Time ◽

Computer Games ◽

Construction Projects ◽

Mixed Reality ◽

Semantic Segmentation ◽

Environment Design ◽

Aviation Training ◽

Architecture And Design ◽

World Environment

Mixed reality (MR) is rapidly becoming a vital tool, not just in gaming, but also in education, medicine, construction and environmental management. The term refers to systems in which computer-generated content is superimposed over objects in a real-world environment across one or more sensory modalities. Although most of us have heard of the use of MR in computer games, it also has applications in military and aviation training, as well as tourism, healthcare and more. In addition, it has the potential for use in architecture and design, where buildings can be superimposed in existing locations to render 3D generations of plans. However, one major challenge that remains in MR development is the issue of real-time occlusion. This refers to hiding 3D virtual objects behind real articles. Dr Tomohiro Fukuda, who is based at the Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering at Osaka University in Japan, is an expert in this field. Researchers, led by Dr Tomohiro Fukuda, are tackling the issue of occlusion in MR. They are currently developing a MR system that realises real-time occlusion by harnessing deep learning to achieve an outdoor landscape design simulation using a semantic segmentation technique. This methodology can be used to automatically estimate the visual environment prior to and after construction projects.

Download Full-text

Real-time, High-resolution Depth Upsampling on Embedded Accelerators

ACM Transactions on Embedded Computing Systems ◽

10.1145/3436878 ◽

2021 ◽

Vol 20 (3) ◽

pp. 1-22

Author(s):

David Langerman ◽

Alan George

Keyword(s):

High Resolution ◽

Low Power ◽

Real Time ◽

Mixed Reality ◽

Graphics Processing Unit ◽

Processing Unit ◽

Reconfigurable Logic ◽

Depth Sensors ◽

Time Requirements ◽

Graphics Processing

High-resolution, low-latency apps in computer vision are ubiquitous in today’s world of mixed-reality devices. These innovations provide a platform that can leverage the improving technology of depth sensors and embedded accelerators to enable higher-resolution, lower-latency processing for 3D scenes using depth-upsampling algorithms. This research demonstrates that filter-based upsampling algorithms are feasible for mixed-reality apps using low-power hardware accelerators. The authors parallelized and evaluated a depth-upsampling algorithm on two different devices: a reconfigurable-logic FPGA embedded within a low-power SoC; and a fixed-logic embedded graphics processing unit. We demonstrate that both accelerators can meet the real-time requirements of 11 ms latency for mixed-reality apps. 1

Download Full-text

Real-time Upper Body Reconstruction and Streaming for Mixed Reality Applications

2020 International Conference on Cyberworlds (CW) ◽

10.1109/cw49994.2020.00027 ◽

2020 ◽

Author(s):

Dimitrios Laskos ◽

Konstantinos Moustakas

Keyword(s):

Real Time ◽

Mixed Reality ◽

Upper Body

Download Full-text

Simulationsgestützte Produktionsplanung flexibler Fertigungssysteme*/Simulation-based production planning of flexible manufacturing systems

wt Werkstattstechnik online ◽

10.37544/1436-4980-2019-04-40 ◽

2019 ◽

Vol 109 (04) ◽

pp. 242-249

Author(s):

A. Selmaier ◽

T. Donhauser ◽

T. Lechler ◽

J. Zeitler ◽

J. Franke

Keyword(s):

Production Planning ◽

Flexible Manufacturing ◽

Material Flow ◽

Flexible Manufacturing Systems ◽

Manufacturing Systems ◽

Flow Simulation ◽

Limited Extent ◽

Time Data ◽

High Data ◽

Simulation Based

Während sich das Verhalten starr verketteter Systeme relativ einfach mittels Materialflusssimulationen modellieren lässt, sind herkömmliche Simulationsansätze für flexible Fertigungssysteme aufgrund des hohen Datenerhebungs- sowie Parametrisieraufwands nur bedingt geeignet. Jedoch kann durch das automatische Übertragen von Echtzeitdaten in das Simulationsmodell der aktuelle Zustand solcher Systeme deutlich verbessert abgebildet werden. Der Beitrag stellt ein Konzept für die simulationsgestützte Produktionsplanung schnellveränderlicher Systeme vor.   While the behaviour of rigidly linked systems is relatively easy to model by means of material flow simulation, traditional simulation approaches are only suitable to a limited extent for flexible manufacturing systems due to the high data collection and parameterization effort. However, the use of real-time data can significantly improve the simulation of such systems. This paper presents an approach for simulation-based production planning of rapidly changing systems.

Download Full-text

A LEARNING-BASED REAL-TIME CONTROLLER FOR DYNAMIC MANUFACTURING CELLS

Journal of the Chinese Institute of Industrial Engineers ◽

10.1080/10170660409509406 ◽

2004 ◽

Vol 21 (3) ◽

pp. 251-262

Author(s):

Y. L. SUN ◽

Y. YIH

Keyword(s):

Real Time ◽

Manufacturing Cells

Download Full-text

Real-Time Affective Measurements in Medical Education, Using Virtual and Mixed Reality

Brain Function Assessment in Learning - Lecture Notes in Computer Science ◽

10.1007/978-3-030-60735-7_9 ◽

2020 ◽

pp. 87-95

Author(s):

Panagiotis Antoniou ◽

George Arfaras ◽

Niki Pandria ◽

George Ntakakis ◽

Emmanuil Bambatsikos ◽

...

Keyword(s):

Medical Education ◽

Real Time ◽

Mixed Reality

Download Full-text

Digital matter as interdisciplinary commodity

The Journal of Public Space ◽

10.5204/jps.v2i3.119 ◽

2017 ◽

Vol 2 (3) ◽

pp. 103

Author(s):

Uwe Rieger

Keyword(s):

Real Time ◽

Spatial Data ◽

Architectural Design ◽

Large Scale ◽

Mixed Reality ◽

Digital Fabrication ◽

Digital Information ◽

Design Environment ◽

Digital World ◽

Starting Point

With the current exponential growth in the sector of Spatial Data Technology and Mixed Reality display devises we experience an increasing overlap of the physical and digital world. Next to making data spatially visible the attempt is to connect digital information with physical properties. Over the past years a number of research institutions have been laying the ground for these developments. In contemporary architecture architectural design the dominant application of data technology is connected to graphical presentation, form finding and digital fabrication. The arc/sec Lab for Digital Spatial Operations at the University of Auckland takes a further step. The Lab explores concepts for a new condition of buildings and urban patterns in which digital information is connected with spatial appearance and linked to material properties. The approach focuses on the step beyond digital re-presentation and digital fabrication, where data is re-connected to the multi-sensory human perceptions and physical skills. The work at the Lab is conducted in a cross disciplinary design environment and based on experiential investigations. The arc/sec Lab utilizes large-scale interactive installations as the driving vehicle for the exploration and communication of new dimensions in architectural space. The experiments are aiming to make data “touchable” and to demonstrate real time responsive environments. In parallel they are the starting point for both the development of practice oriented applications and speculation on how our cities and buildings might change in the future. The article gives an overview of the current experiments being undertaken at the arc/sec Lab. It discusses how digital technologies allow for innovation between the disciplines by introducing real time adaptive behaviours to our build environment and it speculates on the type of spaces we can construct when digital matter is used as a new dynamic building material.

Download Full-text