scholarly journals Intelligent motivation framework based on Q-network for multiple agents in Internet of Things simulations

2019 ◽  
Vol 15 (7) ◽  
pp. 155014771986638
Author(s):  
Hieu Trong Nguyen ◽  
Phuong Minh Chu ◽  
Jisun Park ◽  
Yunsick Sung ◽  
Kyungeun Cho
Keyword(s):  
Internet Of Things ◽  
Autonomous Vehicle ◽  
Three Dimensional ◽  
Virtual Agents ◽  
Daily Lives ◽  
Agent Based ◽  
Q Learning ◽  
Hierarchical Task Network ◽  
Average Accuracy ◽  
Motivation Model

Internet of Things simulations play significant roles in the diverse kinds of activities in our daily lives and have been extensively researched. Creating and controlling virtual agents in three-dimensional Internet of Things simulations is a key technology for achieving realism in three-dimensional simulations. Given that traditional virtual agent-based approaches have limitations for realism, it is necessary to improve the realism of three-dimensional Internet of Things simulations. This article proposes a Q-Network-based motivation framework that applies a Q-Network to select motivations from desires and hierarchical task network planning to execute actions based on goals of the selected motivations. The desires are to be identified and calculated based on states. Selected motivations will be chosen to determine the goals that agents must achieve. In the experiments, the proposed framework achieved an average accuracy of up to 85.5% when the Q-Network-based motivation model was trained. To verify the Q-Network-based motivation framework, a traditional Q-learning is also applied in the three-dimensional virtual environment. Comparing the results of the two frameworks, the Q-Network-based motivation framework shows better results than those of traditional Q-learning, as the accuracy of the Q-Network-based motivation is higher by 15.58%. The proposed framework can be applied to the diverse kinds of Internet of Things systems such as a training autonomous vehicle. Moreover, the proposed framework can generate big data on animal behaviors for other training systems.

scholarly journals An Agent-Based Approach to Realize Emergent Configurations in the Internet of Things

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1347 ◽  
Author(s):  
Fahed Alkhabbas ◽  
Romina Spalazzese ◽  
Paul Davidsson
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Internet Of Things ◽  
The Internet ◽  
Daily Lives ◽  
Agent Based ◽  
Design Time ◽  
Underlying Processes ◽  
The Internet Of Things

The Internet of Things (IoT) has enabled physical objects and devices, often referred to as things, to connect and communicate. This has opened up for the development of novel types of services that improve the quality of our daily lives. The dynamicity and uncertainty of IoT environments, including the mobility of users and devices, make it hard to foresee at design time available things and services. Further, users should be able to achieve their goals seamlessly in arbitrary environments. To address these challenges, we exploit Artificial Intelligence (AI) to engineer smart IoT systems that can achieve user goals and cope with the dynamicity and uncertainty of their environments. More specifically, the main contribution of this paper is an approach that leverages the notion of Belief-Desire-Intention agents and Machine Learning (ML) techniques to realize Emergent Configurations (ECs) in the IoT. An EC is an IoT system composed of a dynamic set of things that connect and cooperate temporarily to achieve a user goal. The approach enables the distributed formation, enactment, adaptation of ECs, and conflict resolution among them. We present a conceptual model of the entities of the approach, its underlying processes, and the guidelines for using it. Moreover, we report about the simulations conducted to validate the feasibility of the approach and evaluate its scalability.

scholarly journals Cloud Manufacturing, Internet of Things-Assisted Manufacturing and 3D Printing Technology: Reliable Tools for Sustainable Construction

2021 ◽  
Vol 13 (13) ◽  
pp. 7327
Author(s):  
Rajesh Singh ◽  
Anita Gehlot ◽  
Shaik Vaseem Akram ◽  
Lovi Raj Gupta ◽  
Manoj Kumar Jena ◽  
...  
Keyword(s):  
3D Printing ◽  
Internet Of Things ◽  
Materials Science ◽  
Predictive Analytics ◽  
Residential Buildings ◽  
Three Dimensional ◽  
Cloud Manufacturing ◽  
Service Evaluation ◽  
Sustainable Construction ◽  
Access Technology

The United Nations (UN) 2030 agenda on sustainable development goals (SDGs) encourages us to implement sustainable infrastructure and services for confronting challenges such as large energy consumption, solid waste generation, depletion of water resources and emission of greenhouse gases in the construction industry. Therefore, to overcome challenges and establishing sustainable construction, there is a requirement to integrate information technology with innovative manufacturing processes and materials science. Moreover, the wide implementation of three-dimensional printing (3DP) technology in constructing monuments, artistic objects, and residential buildings has gained attention. The integration of the Internet of Things (IoT), cloud manufacturing (CM), and 3DP allows us to digitalize the construction for providing reliable and digitalized features to the users. In this review article, we discuss the opportunities and challenges of implementing the IoT, CM, and 3D printing (3DP) technologies in building constructions for achieving sustainability. The recent convergence research of cloud development and 3D printing (3DP) are being explored in the article by categorizing them into multiple sections including 3D printing resource access technology, 3D printing cloud platform (3D–PCP) service architectures, 3D printing service optimized configuration technology, 3D printing service evaluation technology, and 3D service control and monitoring technology. This paper also examines and analyzes the limitations of existing research and, moreover, the article provides key recommendations such as automation with robotics, predictive analytics in 3DP, eco-friendly 3DP, and 5G technology-based IoT-based CM for future enhancements.

scholarly journals Agent-Based Modeling Demonstrates How Local Chemotactic Behavior Can Shape Biofilm Architecture

mSphere ◽  
2019 ◽  
Vol 4 (3) ◽  
Author(s):  
Emily G. Sweeney ◽  
Andrew Nishida ◽  
Alexandra Weston ◽  
Maria S. Bañuelos ◽  
Kristin Potter ◽  
...  
Keyword(s):  
Helicobacter Pylori ◽  
Three Dimensional ◽  
Structural Features ◽  
Agent Based Modeling ◽  
Emergent Properties ◽  
Cellular Interactions ◽  
Biofilm Growth ◽  
Content Type ◽  
Agent Based ◽  
H Pylori

ABSTRACTBacteria are often found living in aggregated multicellular communities known as biofilms. Biofilms are three-dimensional structures that confer distinct physical and biological properties to the collective of cells living within them. We used agent-based modeling to explore whether local cellular interactions were sufficient to give rise to global structural features of biofilms. Specifically, we asked whether chemorepulsion from a self-produced quorum-sensing molecule, autoinducer-2 (AI-2), was sufficient to recapitulate biofilm growth and cellular organization observed for biofilms ofHelicobacter pylori, a common bacterial resident of human stomachs. To carry out this modeling, we modified an existing platform, Individual-based Dynamics of Microbial Communities Simulator (iDynoMiCS), to incorporate three-dimensional chemotaxis, planktonic cells that could join or leave the biofilm structure, and cellular production of AI-2. We simulated biofilm growth of previously characterizedH. pyloristrains with various AI-2 production and sensing capacities. Using biologically plausible parameters, we were able to recapitulate both the variation in biofilm mass and cellular distributions observed with these strains. Specifically, the strains that were competent to chemotax away from AI-2 produced smaller and more heterogeneously spaced biofilms, whereas the AI-2 chemotaxis-defective strains produced larger and more homogeneously spaced biofilms. The model also provided new insights into the cellular demographics contributing to the biofilm patterning of each strain. Our analysis supports the idea that cellular interactions at small spatial and temporal scales are sufficient to give rise to larger-scale emergent properties of biofilms.IMPORTANCEMost bacteria exist in aggregated, three-dimensional structures called biofilms. Although biofilms play important ecological roles in natural and engineered settings, they can also pose societal problems, for example, when they grow in plumbing systems or on medical implants. Understanding the processes that promote the growth and disassembly of biofilms could lead to better strategies to manage these structures. We had previously shown thatHelicobacter pyloribacteria are repulsed by high concentrations of a self-produced molecule, AI-2, and thatH. pylorimutants deficient in AI-2 sensing form larger and more homogeneously spaced biofilms. Here, we used computer simulations of biofilm formation to show that localH. pyloribehavior of repulsion from high AI-2 could explain the overall architecture ofH. pyloribiofilms. Our findings demonstrate that it is possible to change global biofilm organization by manipulating local cell behaviors, which suggests that simple strategies targeting cells at local scales could be useful for controlling biofilms in industrial and medical settings.

scholarly journals The Application of Visual Environmental Economics in the Study of Public Preference and Urban Greenspace

2009 ◽  
Vol 36 (2) ◽  
pp. 355-375 ◽  
Author(s):  
Richard Laing ◽  
Anne-Marie Davies ◽  
David Miller ◽  
Anna Conniff ◽  
Stephen Scott ◽  
...  
Keyword(s):  
Environmental Economic ◽  
Environmental Economics ◽  
Environmental Conditions ◽  
Three Dimensional ◽  
Computer Models ◽  
Time Of Day ◽  
Public Preference ◽  
Daily Lives ◽  
Urban Greenspace ◽  
Physical Variables

Urban greenspace has consistently been argued to be of great importance to the wellbeing, health, and daily lives of residents and users. This paper reports results from a study that combined the visualisation of public results from a study that combined the visualisation of public greenspace with environmental economics, and that aimed to develop a method by which realistic computer models of sites could be used within preference studies. As part of a methodology that employed contingent rating to establish the values placed on specific greenspace sites, three-dimensional computer models were used to produce visualisations of particular environmental conditions. Of particular importance to the study was the influence of variables including lighting, season, time of day, and weather on the perception of respondents. This study followed previous work that established a suitable approach to the modelling and testing of entirely moveable physical variables within the built environment. As such, the study has established firmly that computer-generated visualisations are appropriate for use within environmental economic surveys, and that there is potential for a holistic range of attributes to be included in such studies.

Blockchain-Based Software-Defined Industrial Internet of Things: A Dueling Deep ${Q}$ -Learning Approach

2019 ◽  
Vol 6 (3) ◽  
pp. 4627-4639 ◽  
Author(s):  
Chao Qiu ◽  
F. Richard Yu ◽  
Haipeng Yao ◽  
Chunxiao Jiang ◽  
Fangmin Xu ◽  
...  
Keyword(s):  
Internet Of Things ◽  
Learning Approach ◽  
Industrial Internet Of Things ◽  
Q Learning ◽  
Industrial Internet

scholarly journals Modules of Organizational and Technical Systems for Solving Problems of Adaptation in a Rapidly Changing Environment

2020 ◽  
Vol 24 (5) ◽  
pp. 82-90
Author(s):  
A. Mikryukov ◽  
V. M. Trembach ◽  
A. V. Danilov
Keyword(s):  
Internet Of Things ◽  
Knowledge Base ◽  
External Environment ◽  
The Internet ◽  
Cyberphysical Systems ◽  
Adaptation Algorithm ◽  
Cognitive Approach ◽  
Agent Based ◽  
Cyberphysical System ◽  
The Internet Of Things

Purpose of research. The aim of the research is to form modules of organizational and technical systems (OTS) using a cognitive approach to solve problems of adaptation of cyberphysical systems. Currently, there is a rapid development of elements of the Internet of things. New tasks related to self-organization and adaptation in a rapidly changing external environment are brought to the fore. These tasks occur when new elements appear in the telecommunications computer network, they fail, change the mode, new tasks occur, etc. To work out these tasks, the possibilities of approaches to support and decision-making such as situational, cognitive, and semiotic are considered. The authors consider the cognitive approach in more detail. Within the framework of the cognitive paradigm, the article describes the use of the cognitive approach for solving problems of adaptation of cyberphysical systems. To solve this problem on the basis of an agent-based approach, the structure of a cyberphysical system with the possibility of adaptation is presented and the functions of its agents are described. The main stages of solving problems of adaptation of cyberphysical systems are presented. An adaptation algorithm using the planning mechanism is presented. The demo example shows a knowledge base for solving the problem of adapting cyberphysical systems using a cognitive planning mechanism.Materials and methods of research. New approaches and methods are required to address adaptation issues in planning. The cognitive approach is one of the developing directions in solving many problems of the Internet of things. One of these tasks is the ability to adapt OTS modules in a rapidly changing external environment based on the planning mechanism. To solve the planning problem, we use the algorithm described by Aristotle more than 2,350 years ago and implemented in the GPS program. This algorithm can be considered the first description of the cognitive mechanism that a person uses. The knowledge base uses an integrated approach to knowledge representation. When developing OTS modules, an agent-based approach was used to solve the problem of adaptation.Results. The existing and developing approaches and methods for decision support and decision-making are considered for decisionmaking in newly emerging situations in OTS modules. The main provisions of such significant approaches as situational, cognitive and semiotic are presented. A cognitive approach to the adaptation of intelligent systems is proposed. The solution of the problem of adaptation of cyberphysical systems is considered within the framework of the cognitive paradigm. The structure of a cyberphysical system capable of solving adaptation problems is shown. The functions of OTS modules based on agent-oriented technology are described. A description of the adaptation algorithm using the cognitive planning mechanism is given. The main stages of solving problems of adaptation of cyberphysical systems are presented. A demo example of solving the problem of adaptation by a cyberphysical system-a cooking robot – is shown.Conclusion. Using the modular architecture of an intelligent system allows you to solve many problems. One of these tasks is to configure elements of the Internet of things when they deviate from their main function. The planning mechanisms proposed for parametric adaptation can be repeatedly applied in OTS modules as separate agents. This approach is relevant for elements of the Internet of things. In the case of expanding the functionality of the OTS modules of Internet of things, it is advisable to apply machine learning with fixing the results in the knowledge base of planning agents.

scholarly journals Research of long radius technology for implementation of solutions of the Іnternet of things

Connectivity ◽  
2020 ◽  
Vol 148 (6) ◽  
Author(s):  
S. A. Zhezhkun ◽  
◽  
L. B. Veksler ◽  
S. M. Brezitsʹkyy ◽  
B. O. Tarasyuk
Keyword(s):  
Internet Of Things ◽  
Third Party ◽  
Security And Privacy ◽  
The Internet ◽  
Daily Lives ◽  
Advantages And Disadvantages ◽  
Iot Applications ◽  
Security Properties ◽  
Technical Features ◽  
The Internet Of Things

This article focuses on the analysis of promising technologies for long-range traffic transmission for the implementation of the Internet of Things. The result of the review of technical features of technologies, their advantages and disadvantages is given. A comparative analysis was performed. An analysis is made that in the future heterogeneous structures based on the integration of many used radio technologies will play a crucial role in the implementation of fifth generation networks and systems. The Internet of Things (IoT) is heavily affecting our daily lives in many domains, ranging from tiny wearable devices to large industrial systems. Consequently, a wide variety of IoT applications have been developed and deployed using different IoT frameworks. An IoT framework is a set of guiding rules, protocols, and standards which simplify the implementation of IoT applications. The success of these applications mainly depends on the ecosystem characteristics of the IoT framework, with the emphasis on the security mechanisms employed in it, where issues related to security and privacy are pivotal. In this paper, we survey the security of the main IoT frameworks, a total of 8 frameworks are considered. For each framework, we clarify the proposed architecture, the essentials of developing third-party smart apps, the compatible hardware, and the security features. Comparing security architectures shows that the same standards used for securing communications, whereas different methodologies followed for providing other security properties.

scholarly journals An Agent-Based Model of Combination Oncolytic Viral Therapy and Anti-PD-1 Immunotherapy Reveals the Importance of Spatial Location When Treating Glioblastoma

Cancers ◽  
2021 ◽  
Vol 13 (21) ◽  
pp. 5314
Author(s):  
Kathleen M. Storey ◽  
Trachette L. Jackson
Keyword(s):  
Three Dimensional ◽  
Spatial Location ◽  
Adaptive Strategy ◽  
Treatment Modalities ◽  
Agent Based Model ◽  
Agent Based ◽  
Oncolytic Viral Therapy ◽  
Landscape Characteristics ◽  
Adaptive Immune ◽  
Viral Therapy

Oncolytic viral therapies and immunotherapies are of growing clinical interest due to their selectivity for tumor cells over healthy cells and their immunostimulatory properties. These treatment modalities provide promising alternatives to the standard of care, particularly for cancers with poor prognoses, such as the lethal brain tumor glioblastoma (GBM). However, uncertainty remains regarding optimal dosing strategies, including how the spatial location of viral doses impacts therapeutic efficacy and tumor landscape characteristics that are most conducive to producing an effective immune response. We develop a three-dimensional agent-based model (ABM) of GBM undergoing treatment with a combination of an oncolytic Herpes Simplex Virus and an anti-PD-1 immunotherapy. We use a mechanistic approach to model the interactions between distinct populations of immune cells, incorporating both innate and adaptive immune responses to oncolytic viral therapy and including a mechanism of adaptive immune suppression via the PD-1/PD-L1 checkpoint pathway. We utilize the spatially explicit nature of the ABM to determine optimal viral dosing in both the temporal and spatial contexts. After proposing an adaptive viral dosing strategy that chooses to dose sites at the location of highest tumor cell density, we find that, in most cases, this adaptive strategy produces a more effective treatment outcome than repeatedly dosing in the center of the tumor.

scholarly journals Adaptive Routing Strategy Based on Improved Double Q-Learning for Satellite Internet of Things

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Jian Zhou ◽  
Xiaotian Gong ◽  
Lijuan Sun ◽  
Yong Xie ◽  
Xiaoyong Yan
Keyword(s):  
Internet Of Things ◽  
Adaptive Routing ◽  
Mobile Internet ◽  
Secure Routing ◽  
Q Value ◽  
Average Delay ◽  
Q Learning ◽  
Data Packets ◽  
Routing Strategy ◽  
Node Status

Satellite Internet of Things (S-IoT), which integrates satellite networks with IoT, is a new mobile Internet to provide services for social networks. However, affected by the dynamic changes of topology structure and node status, the efficient and secure forwarding of data packets in S-IoT is challenging. In view of the abovementioned problem, this paper proposes an adaptive routing strategy based on improved double Q-learning for S-IoT. First, the whole S-IoT is regarded as a reinforcement learning environment, and satellite nodes and ground nodes in S-IoT are both regarded as intelligent agents. Each node in the S-IoT maintains two Q tables, which are used for selecting the forwarding node and for evaluating the forwarding value, respectively. In addition, the next hop node of data packets is determined depending on the mixed Q value. Second, in order to optimize the Q value, this paper makes improvements on the mixed Q value, the reward value, and the discount factor, respectively, based on the congestion degree, the hop count, and the node status. Finally, we perform extensive simulations to evaluate the performance of this adaptive routing strategy in terms of delivery rate, average delay, and overhead ratio. Evaluation results demonstrate that the proposed strategy can achieve more efficient and secure routing in the highly dynamic environment compared with the state-of-the-art strategies.

Sign in / Sign up

Export Citation Format

Share Document