scholarly journals Towards the Internet of Augmented Things: An Open-source Framework to Interconnect IoT Devices and Augmented Reality Systems

Proceedings ◽  
2019 ◽  
Vol 42 (1) ◽  
pp. 50 ◽  
Author(s):  
Óscar Blanco-Novoa ◽  
Paula Fraga-Lamas ◽  
Miguel Vilar-Montesinos ◽  
Tiago Fernández-Caramés
Keyword(s):  
Augmented Reality ◽  
Open Source ◽  
Mixed Reality ◽  
User Interaction ◽  
Physical World ◽  
The Internet ◽  
Full Potential ◽  
Smart Power ◽  
Open Source Framework ◽  
Iot Devices

The latest Augmented Reality (AR) and Mixed Reality (MR) systems are able to provide innovative methods for user interaction, but their full potential can only be achieved when they are able to exchange bidirectional information with the physical world that surround them, including the objects that belong to the Internet of Things (IoT). The problem is that elements like AR display devices or IoT sensors/actuators often use heterogeneous technologies that make it difficult to intercommunicate them in an easy way, thus requiring a high degree of specialization to carry out such a task. This paper presents an open-source framework that eases the integration of AR and IoT devices as well as the transfer of information among them, both in real time and in a dynamic way. The proposed framework makes use of widely used standard protocols and open-source tools like MQTT, HTTPS or Node-RED. In order to illustrate the operation of the framework, this paper presents the implementation of a practical home automation example: an AR/MR application for energy consumption monitoring that allows for using a pair of Microsoft HoloLens smart glasses to interact with smart power outlets.

scholarly journals Creating the Internet of Augmented Things: An Open-Source Framework to Make IoT Devices and Augmented and Mixed Reality Systems Talk to Each Other

Sensors ◽  
2020 ◽  
Vol 20 (11) ◽  
pp. 3328 ◽  
Author(s):  
Óscar Blanco-Novoa ◽  
Paula Fraga-Lamas ◽  
Miguel A. Vilar-Montesinos ◽  
Tiago M. Fernández-Caramés
Keyword(s):  
Open Source ◽  
Mixed Reality ◽  
Normal Operation ◽  
The Internet ◽  
Full Potential ◽  
Use Case ◽  
Smart Power ◽  
Operation Conditions ◽  
Heterogeneous Technologies ◽  
Iot Devices

Augmented Reality (AR) and Mixed Reality (MR) devices have evolved significantly in the last years, providing immersive AR/MR experiences that allow users to interact with virtual elements placed on the real-world. However, to make AR/MR devices reach their full potential, it is necessary to go further and let them collaborate with the physical elements around them, including the objects that belong to the Internet of Things (IoT). Unfortunately, AR/MR and IoT devices usually make use of heterogeneous technologies that complicate their intercommunication. Moreover, the implementation of the intercommunication mechanisms requires involving specialized developers with have experience on the necessary technologies. To tackle such problems, this article proposes the use of a framework that makes it easy to integrate AR/MR and IoT devices, allowing them to communicate dynamically and in real time. The presented AR/MR-IoT framework makes use of standard and open-source protocols and tools like MQTT, HTTPS or Node-RED. After detailing the inner workings of the framework, it is illustrated its potential through a practical use case: a smart power socket that can be monitored and controlled through Microsoft HoloLens AR/MR glasses. The performance of such a practical use case is evaluated and it is demonstrated that the proposed framework, under normal operation conditions, enables to respond in less than 100 ms to interaction and data update requests.

Real-Time 3D Design Modelling of Outdoor Structures Using Mobile Augmented Reality Systems

2009 ◽  
pp. 937-951
Author(s):  
Wayne Piekarski
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
User Interaction ◽  
Physical World ◽  
Iterative Refinement ◽  
3D Design ◽  
Real Objects ◽  
Modelling Techniques ◽  
Head Gestures ◽  
The Mind

This chapter presents a series of new augmented reality user interaction techniques to support the capture and creation of 3D geometry of large outdoor structures. Named construction at a distance, these techniques are based on the action at a distance concepts employed by other virtual environments researchers. These techniques address the problem of AR systems traditionally being consumers of information, rather than being used to create new content. By using information about the user’s physical presence along with hand and head gestures, AR systems can be used to capture and create the geometry of objects that are orders of magnitude larger than the user, with no prior information or assistance. While existing scanning techniques can only be used to capture existing physical objects, construction at a distance also allows the creation of new models that exist only in the mind of the user. Using a single AR interface, users can enter geometry and verify its accuracy in real-time. Construction at a distance is a collection of 3D modelling techniques based on the concept of AR working planes, landmark alignment, constructive solid geometry operations, and iterative refinement to form complex shapes. This chapter presents a number of different construction at a distance techniques, and are demonstrated with examples of real objects that have been modelled in the physical world.

Expert system for identifying and analyzing the IoT devices using Augmented Reality Approach

2021 ◽  
pp. 43-57
Author(s):  
WASIN ALKISHRI ◽  
Mahmood Al-Bahri
Keyword(s):  
Augmented Reality ◽  
Internet Of Things ◽  
Specific Area ◽  
Relevant Information ◽  
The Internet ◽  
Test Results ◽  
New Development ◽  
Server Software ◽  
Iot Devices ◽  
The Internet Of Things

Biometrics In conjunction with the new development of the Internet of Things (IoT), augmented reality (AR) systems are evolving to visualize 3D virtual models of the real world into an intelligent and interactive virtual reality environment that facilitates physical identification of objects and defines their specifications efficiently. The integration between AR and IoT in a complementary way helps identify network-related items' specifications and interact with the Internet of Things more efficiently. An identity is a dedicated, publicly known attribute or set of names for an individual device. Typically, identifiers operate within a specific area or network, making it difficult to identify things globally. This paper explores the use of Augmented Reality (AR) Technology for identifying devices and displaying relevant information about the device to the user. Based on the developed model network, the developed system of identification of IoT devices was tested. Also, the traffic generated by the AR device when generating requests to the organization server was investigated. According to the test results, the system is undemanding to the main network indicators. The system-generated traffic is self-similar. The test results show that the server software can solve the problems of identifying IoT devices through interaction with augmented reality devices.

Ratatoskr: An Open-Source Framework for In-Depth Power, Performance, and Area Analysis and Optimization in 3D NoCs

2022 ◽  
Vol 32 (1) ◽  
pp. 1-21
Author(s):  
Jan Moritz Joseph ◽  
Lennart Bamberg ◽  
Imad Hajjar ◽  
Behnam Razi Perjikolaei ◽  
Alberto García-Ortiz ◽  
...  
Keyword(s):  
Open Source ◽  
Single Point ◽  
User Interaction ◽  
Design Parameters ◽  
Power Performance ◽  
Levels Of Abstraction ◽  
Simulation Speed ◽  
Point Of Entry ◽  
Open Source Framework ◽  
High Level

We introduce Ratatoskr , an open-source framework for in-depth power, performance, and area (PPA) analysis in Networks-on-Chips (NoCs) for 3D-integrated and heterogeneous System-on-Chips (SoCs). It covers all layers of abstraction by providing an NoC hardware implementation on Register Transfer Level (RTL), an NoC simulator on cycle-accurate level and an application model on transaction level. By this comprehensive approach, Ratatoskr can provide the following specific PPA analyses: Dynamic power of links can be measured within 2.4% accuracy of bit-level simulations while maintaining cycle-accurate simulation speed. Router power is determined from RTL-to-gate-level synthesis combined with cycle-accurate simulations. The performance of the whole NoC can be measured both via cycle-accurate and RTL simulations. The performance (i.e., timing) of individual routers and the NoC area are obtained from RTL synthesis results. Despite these manifold features, Ratatoskr offers easy two-step user interaction: (1) A single point-of-entry allows setting design parameters. (2) PPA reports are generated automatically. For both the input and the output, different levels of abstraction can be chosen for high-level rapid network analysis or low-level improvement of architectural details. The synthesizable NoC-RTL model shows improved total router power and area in comparison to a conventional standard router. As a forward-thinking and unique feature not found in other NoC PPA-measurement tools, Ratatoskr supports heterogeneous 3D integration that is one of the most promising integration paradigms for upcoming SoCs. Thereby, Ratatoskr lays the groundwork to design their communication architectures. The framework is publicly available at https://github.com/ratatoskr-project .

scholarly journals SAR.IoT: Secured Augmented Reality for IoT Devices Management

Sensors ◽  
2021 ◽  
Vol 21 (18) ◽  
pp. 6001
Author(s):  
Daniel Fuentes ◽  
Luís Correia ◽  
Nuno Costa ◽  
Arsénio Reis ◽  
João Barroso ◽  
...  
Keyword(s):  
Augmented Reality ◽  
The Internet ◽  
Use Case ◽  
Home Automation ◽  
Technological Solution ◽  
Industrial Laboratory ◽  
Research Environments ◽  
Iot Devices ◽  
Output Information ◽  
The Internet Of Things

Currently, solutions based on the Internet of Things (IoT) concept are increasingly being adopted in several fields, namely, industry, agriculture, and home automation. The costs associated with this type of equipment is reasonably small, as IoT devices usually do not have output peripherals to display information about their status (e.g., a screen or a printer), although they may have informative LEDs, which is sometimes insufficient. For most IoT devices, the price of a minimalist display, to output and display the device’s running status (i.e., what the device is doing), might cost much more than the actual IoT device. Occasionally, it might become necessary to visualize the IoT device output, making it necessary to find solutions to show the hardware output information in real time, without requiring extra equipment, only what the administrator usually has with them. In order to solve the above, a technological solution that allows for the visualization of IoT device information in actual time, using augmented reality and a simple smartphone, was developed and analyzed. In addition, the system created integrates a security layer, at the level of AR, to secure the shown data from unwanted eyes. The results of the tests carried out allowed us to validate the operation of the solution when accessing the information of the IoT devices, verify the operation of the security layer in AR, analyze the interaction between smartphones, the platform, and the devices, and check which AR markers are most optimized for this use case. This work results in a secure augmented reality solution, which can be used with a simple smartphone, to monitor/manage IoT devices in industrial, laboratory or research environments.

Side-Channel Attacks in the Internet of Things

2018 ◽  
pp. 325-357 ◽  
Author(s):  
Andreas Zankl ◽  
Hermann Seuschek ◽  
Gorka Irazoqui ◽  
Berk Gulmezoglu
Keyword(s):  
Internet Of Things ◽  
Physical World ◽  
The Internet ◽  
Side Channel ◽  
Sensitive Information ◽  
Side Channel Attacks ◽  
Subsequent Discussion ◽  
Iot Devices ◽  
The Internet Of Things ◽  
Security Of Iot

The Internet of Things (IoT) rapidly closes the gap between the virtual and the physical world. As more and more information is processed through this expanding network, the security of IoT devices and backend services is increasingly important. Yet, side-channel attacks pose a significant threat to systems in practice, as the microarchitectures of processors, their power consumption, and electromagnetic emanation reveal sensitive information to adversaries. This chapter provides an extensive overview of previous attack literature. It illustrates that microarchitectural attacks can compromise the entire IoT ecosystem: from devices in the field to servers in the backend. A subsequent discussion illustrates that many of today's security mechanisms integrated in modern processors are in fact vulnerable to the previously outlined attacks. In conclusion to these observations, new countermeasures are needed that effectively defend against both microarchitectural and power/EM based side-channel attacks.

scholarly journals IoT Network Security: Threats, Risks, and a Data-Driven Defense Framework

IoT ◽  
2020 ◽  
Vol 1 (2) ◽  
pp. 259-285 ◽  
Author(s):  
Charles Wheelus ◽  
Xingquan Zhu
Keyword(s):  
Machine Learning ◽  
Predictive Models ◽  
Data Science ◽  
Physical World ◽  
Machine Learning Algorithms ◽  
The Internet ◽  
Security Threats ◽  
Criminal Organizations ◽  
Iot Devices ◽  
Using Data

The recent surge in Internet of Things (IoT) deployment has increased the pace of integration and extended the reach of the Internet from computers, tablets and phones to a myriad of devices in our physical world. Driven by the IoT, with each passing day, the Internet becomes more integrated with everyday life. While IoT devices provide endless new capabilities and make life more convenient, they also vastly increase the opportunity for nefarious individuals, criminal organizations and even state actors to spy on, and interfere with, unsuspecting users of IoT systems. As this looming crisis continues to grow, calls for data science approaches to address these problems have increased, and current research shows that predictive models trained with machine learning algorithms hold great potential to mitigate some of these issues. In this paper, we first carry out an analytics approach to review security risks associated with IoT systems, and then propose a machine learning-based solution to characterize and detect IoT attacks. We use a real-world IoT system with secured gate access as a platform, and introduce the IoT system in detail, including features to capture security threats/attacks to the system. By using data collected from a nine month period as our testbed, we evaluate the efficacy of predictive models trained by means of machine learning, and propose design principles and a loose framework for implementing secure IoT systems.

Real-Time 3D Design and Modelling of Outdoor Structures Using Mobile Augmented Reality Systems

2011 ◽  
pp. 181-197
Author(s):  
Wayne Piekarski
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
User Interaction ◽  
Physical World ◽  
Iterative Refinement ◽  
3D Design ◽  
Real Objects ◽  
Modelling Techniques ◽  
Head Gestures ◽  
The Mind

This chapter presents a series of new augmented reality user interaction techniques to support the capture and creation of 3D geometry of large outdoor structures. Named construction at a distance, these techniques are based on the action at a distance concepts employed by other virtual environments researchers. These techniques address the problem of AR systems traditionally being consumers of information, rather than being used to create new content. By using information about the user’s physical presence along with hand and head gestures, AR systems can be used to capture and create the geometry of objects that are orders of magnitude larger than the user, with no prior information or assistance. While existing scanning techniques can only be used to capture existing physical objects, construction at a distance also allows the creation of new models that exist only in the mind of the user. Using a single AR interface, users can enter geometry and verify its accuracy in real-time. Construction at a distance is a collection of 3D modelling techniques based on the concept of AR working planes, landmark alignment, constructive solid geometry operations, and iterative refinement to form complex shapes. This chapter presents a number of different construction at a distance techniques, and are demonstrated with examples of real objects that have been modelled in the physical world.

scholarly journals HoloFlows: modelling of processes for the Internet of Things in mixed reality

2021 ◽  
Author(s):  
Ronny Seiger ◽  
Romina Kühn ◽  
Mandy Korzetz ◽  
Uwe Aßmann
Keyword(s):  
Internet Of Things ◽  
Mixed Reality ◽  
Process Modelling ◽  
Control Individual ◽  
Smart Devices ◽  
The Internet ◽  
Technical Expertise ◽  
End User ◽  
Iot Devices ◽  
The Internet Of Things

AbstractOur everyday lives are increasingly pervaded by digital assistants and smart devices forming the Internet of Things (IoT). While user interfaces to directly monitor and control individual IoT devices are becoming more sophisticated and end-user friendly, applications to connect standalone IoT devices and create more complex IoT processes for automating and assisting users with repetitive tasks still require a high level of technical expertise and programming knowledge. Related approaches for process modelling in IoT mostly suggest extensions to complex modelling languages, require high levels of abstraction and technical knowledge, and rely on unintuitive tools. We present a novel approach for end-user oriented-no-code-IoT process modelling using Mixed Reality (MR) technology: HoloFlows. Users are able to explore the IoT environment and model processes among sensors and actuators as first-class citizens by simply “drawing” virtual wires among physical IoT devices. MR technology hereby facilitates the understanding of the physical contexts and relations among the IoT devices and provides a new and more intuitive way of modelling IoT processes. The results of a user study comparing HoloFlows with classical modelling approaches show an increased user experience and decrease in required modelling knowledge and technical expertise to create IoT processes.

Sign in / Sign up

Export Citation Format

Share Document