Decentralized Multiagent Actor-Critic Algorithm Based on Message Diffusion

The exponential explosion of joint actions and massive data collection are two main challenges in multiagent reinforcement learning algorithms with centralized training. To overcome these problems, in this paper, we propose a model-free and fully decentralized actor-critic multiagent reinforcement learning algorithm based on message diffusion. To this end, the agents are assumed to be placed in a time-varying communication network. Each agent makes limited observations regarding the global state and joint actions; therefore, it needs to obtain and share information with others over the network. In the proposed algorithm, agents hold local estimations of the global state and joint actions and update them with local observations and the messages received from neighbors. Under the hypothesis of the global value decomposition, the gradient of the global objective function to an individual agent is derived. The convergence of the proposed algorithm with linear function approximation is guaranteed according to the stochastic approximation theory. In the experiments, the proposed algorithm was applied to a passive location task multiagent environment and achieved superior performance compared to state-of-the-art algorithms.

Treatment of Hypertriglyceridemia: A Review of Therapies in the Pipeline

Background The 2018 American College of Cardiology/American Heart Association (ACC/AHA) guidelines and 2021 ACC Expert Consensus Decision Pathway recommend nonpharmacological interventions and initiation of statin therapy for patients with moderate hypertriglyceridemia and addition of fibrates or omega-3 fatty acids in severe hypertriglyceridemia. Although the association between triglyceride (TG) lowering and atherosclerotic cardiovascular disease (ASCVD) risk reduction remains controversial, patients with hypertriglyceridemia may represent a subgroup that require additional therapy to further reduce residual ASCVD risk. Moreover, medications that target novel pathways could provide alternative options for patients who are intolerant of existing therapies or doses needed to provide adequate triglyceride lowering. Objective: Assess recent evidence for TG-lowering agents including omega-3 fatty acid-based therapies, PPARα modulators, apoC-III mRNA antisense inhibitors, angiopoietin-like 3 (ANGPTL3) antibodies, and herbal supplements. Methods: A literature search was performed using PubMed with hypertriglyceridemia specified as a MeSH term or included in the title or abstract of the article along with each individual agent. For inclusion, trials needed to have a primary or secondary outcome of TG levels or TG lowering. Conclusion: Currently, the only US Food and Drug Administration approved medication for CV risk reduction in patients with hypertriglyceridemia is icosapent ethyl. Results from phase 3 trials for CaPre, pemafibrate, and volanesorsen as well as additional evidence for pipeline pharmacotherapies with novel mechanisms of action (e.g., ApoC-III mRNA antisense inhibitors and ANGPTL3 antibodies) will help to guide future pharmacotherapy considerations for patients with hypertriglyceridemia.

Equality and Freedom as Uncertainty in Groups

In this paper, I investigate a connection between a common characterisation of freedom and how uncertainty is managed in a Bayesian hierarchical model. To do this, I consider a distributed factorization of a group’s optimization of free energy, in which each agent is attempting to align with the group and with its own model. I show how this can lead to equilibria for groups, defined by the capacity of the model being used, essentially how many different datasets it can handle. In particular, I show that there is a “sweet spot” in the capacity of a normal model in each agent’s decentralized optimization, and that this “sweet spot” corresponds to minimal free energy for the group. At the sweet spot, an agent can predict what the group will do and the group is not surprised by the agent. However, there is an asymmetry. A higher capacity model for an agent makes it harder for the individual to learn, as there are more parameters. Simultaneously, a higher capacity model for the group, implemented as a higher capacity model for each member agent, makes it easier for a group to integrate a new member. To optimize for a group of agents then requires one to make a trade-off in capacity, as each individual agent seeks to decrease capacity, but there is pressure from the group to increase capacity of all members. This pressure exists because as individual agent’s capacities are reduced, so too are their abilities to model other agents, and thereby to establish pro-social behavioural patterns. I then consider a basic two-level (dual process) Bayesian model of social reasoning and a set of three parameters of capacity that are required to implement such a model. Considering these three capacities as dependent elements in a free energy minimization for a group leads to a “sweet surface” in a three-dimensional space defining the triplet of parameters that each agent must use should they hope to minimize free energy as a group. Finally, I relate these three parameters to three notions of freedom and equality in human social organization, and postulate a correspondence between freedom and model capacity. That is, models with higher capacity, have more freedom as they can interact with more datasets.

A Novel Ship Collision Avoidance Awareness Approach for Cooperating Ships Using Multi-Agent Deep Reinforcement Learning

Ships are special machineries with large inertias and relatively weak driving forces. Simulating the manual operations of manipulating ships with artificial intelligence (AI) and machine learning techniques becomes more and more common, in which avoiding collisions in crowded waters may be the most challenging task. This research proposes a cooperative collision avoidance approach for multiple ships using a multi-agent deep reinforcement learning (MADRL) algorithm. Specifically, each ship is modeled as an individual agent, controlled by a Deep Q-Network (DQN) method and described by a dedicated ship motion model. Each agent observes the state of itself and other ships as well as the surrounding environment. Then, agents analyze the navigation situation and make motion decisions accordingly. In particular, specific reward function schemas are designed to simulate the degree of cooperation among agents. According to the International Regulations for Preventing Collisions at Sea (COLREGs), three typical scenarios of simulation, which are head-on, overtaking and crossing, are established to validate the proposed approach. With sufficient training of MADRL, the ship agents were capable of avoiding collisions through cooperation in narrow crowded waters. This method provides new insights for bionic modeling of ship operations, which is of important theoretical and practical significance.

Agent-based Modeling

Agent-based modeling (ABM) has become widely accepted as a methodological tool to model and simulate dynamic processes of geographical phenomena. A growing number of ABM studies across a variety of domains and disciplines is partially explained by the development of agent-modeling tools and platforms, the availability of micro-data, and the advancement in computer technology and cyberinfrastructure. In addition to these technical reasons, another key motivation underlying ABM research is to address challenges embedded in conventional modeling approaches being relatively coarse, aggregate, static, normative, and inflexible across scales with a reductionist viewpoint (Batty 2005 cited under Application: Urban Systems.” With complexity science, including complex systems, complex adaptive systems, and artificial life, providing theoretical foundations and rationales, ABM is a computational methodology for simulating dynamic processes of nature and human systems driven by disaggregated, heterogeneous, and autonomous entities, i.e., agents, that interact among themselves and their environments. A key fundamental concept of the ABM framework is that a system emerges from the dynamic individual-level interactions from bottom-up, where the simulated outcome is more than the sum of its components. This bottom-up approach enables ABM to exhibit complex system dynamics, properties of which could include feedback effect, path-dependence, phase shift, non-linearity, adaptation, self-organization, tipping points, and emergence. Three key components of ABM are agents, their environment, and their decision rules. Agents are the crucial component in ABM where each individual agent has its own characteristics and agenda, assesses its surrounded situation, and makes decisions. Agents reside in an environment, which can represent a geographic space in case for spatially explicit agent-based models. Agents’ behavioral decisions and interactions within their environment are defined based on a set of rules, which can alter their status and location over time. The purpose of ABM research can be classified into theoretical exploration and empirical investigation as well as the combination of two. In the latter case, ABM can be used as an artificial laboratory experiment to explore what-if scenarios and to investigate how changes in agents, environments, and/or rules affect the macro-level outcomes. ABM has been applied to represent a wide variety of geographic processes and behaviors including but not limited to urban system, land-use/land-cover change, ecology, transportation, animal/human movement, behavioral geography, spatial cognition, transportation, and disease epidemiology. While the growing interest in ABM as a modeling methodology to simulate complex systems is remarkable, there exist various conceptual, methodological, and technical challenges.

The prevalence of 30 HPV genotypes detected by EUROArray HPV in cervical samples among unvaccinated women from Vojvodina province, Serbia

This study evaluates the pre-vaccination prevalence of HPV infection in women from Vojvodina, Serbia, according to age and cytological status. A total of 1,495 women, ranging from 18 to 65 years of age, with different cytological results were enrolled. The HPV genotyping assay was performed using the EUROArray HPV test in order to detect thirty genitally relevant HPV subtypes. In our study, the most prevalent genotypeswere HPV 16, 31, 51, and 53. Among these, HPV 16 was consistently present in all cytological subgroups. Twelve HPV genotypes classified as carcinogenic to humans (Group 1) were detected in 77.8.0% of HSIL/ASCH and 55.0% of NILM with abnormal colposcopy findings. Six possible carcinogens—HRs (group 2B) were often found in women with normal cytology (14.8%) and mild abnormalities (ASCUS and LSIL), but with lower frequence in HSIL/ASCH lesions (7.1%). HPVs 6 and 11(Group 3) were not found in the cases of HSIL/ASCH. Unclassified HPV types were equally distributed in all cytology groups: 20.7%, 19.1%, 16.3% and 13% of NILM, ASCUS, LSIL and HSIL/ASCH, respectively. Our findings highlight that majority of abnormal Pap test results are caused by Group 1 HPVs among women from our region. Low frequency HPVs of group 2A/2B, especially HSIL/ASCH, supports the conclusion that individual genotypes require consideration of each type as an individual agent. We expect a positive impact of HPV vaccine in reducing HPV-associated cervical lesions among women from Vojvodina province, after establishing vaccination programs in our country.

Agent, Behaviour, Trace, Repeat: Understanding the Cognitive Processes Involved in Human Stigmergic Coordination

Stigmergy refers to the coordination of agents via artifacts of behaviours (behavioural traces) in the shared environment. Whilst primarily studied in biology and computer science/robotics, stigmergy underlies many human indirect interactions, both offline (e.g., trail building) and online (e.g., development of open-source software). In this review, we provide an introduction to stigmergy and emphasise how and where human stigmergy is distinct from animal or robot stigmergy, such as intentional communication via traces and causal inferences from the traces to the causing behaviour. Cognitive processes discussed on the agent level include attention, motivation, meaning and meta-cognition, as well as emergence/immergence, iterative learning and exploration/exploitation at the interface of individual agent and multi-agent systems. Characteristics of one-agent, two-agent and multi-agent systems are discussed and areas for future research highlighted.

Endostatin Genetically Engineered Placental Mesenchymal Stromal Cells Carrying Doxorubicin-Loaded Mesoporous Silica Nanoparticles for Combined Chemo- and Antiangiogenic Therapy

Combination therapies constitute a powerful tool for cancer treatment. By combining drugs with different mechanisms of action, the limitations of each individual agent can be overcome, while increasing therapeutic benefit. Here, we propose employing tumor-migrating decidua-derived mesenchymal stromal cells as therapeutic agents combining antiangiogenic therapy and chemotherapy. First, a plasmid encoding the antiangiogenic protein endostatin was transfected into these cells by nucleofection, confirming its expression by ELISA and its biological effect in an ex ovo chick embryo model. Second, doxorubicin-loaded mesoporous silica nanoparticles were introduced into the cells, which would act as vehicles for the drug being released. The effect of the drug was evaluated in a coculture in vitro model with mammary cancer cells. Third, the combination of endostatin transfection and doxorubicin-nanoparticle loading was carried out with the decidua mesenchymal stromal cells. This final cell platform was shown to retain its tumor-migration capacity in vitro, and the combined in vitro therapeutic efficacy was confirmed through a 3D spheroid coculture model using both cancer and endothelial cells. The results presented here show great potential for the development of combination therapies based on genetically-engineered cells that can simultaneously act as cellular vehicles for drug-loaded nanoparticles.

Carboplatin-Cyclophosphamide or Paclitaxel without or with Bevacizumab as First-Line Treatment for Metastatic Triple-Negative Breast Cancer (BOOG 2013-01)

<b><i>Background:</i></b> The addition of bevacizumab to chemotherapy conferred a modest progression-free survival (PFS) benefit in metastatic triple-negative breast cancer (mTNBC). However, no overall survival (OS) benefit has been reported. Also, its combination with carboplatin-cyclophosphamide (CC) has never been investigated. <b><i>Methods:</i></b> The Triple-B study is a multicenter, randomized phase IIb trial that aims to prospectively validate predictive biomarkers, including baseline plasma vascular endothelial growth factor receptor-2 (pVEGFR-2), for bevacizumab benefit. mTNBC patients were randomized between CC and paclitaxel (P) without or with bevacizumab (CC ± B or P ± B). Here we report on a preplanned safety and preliminary efficacy analysis after the first 12 patients had been treated with CC+B and on the predictive value of pVEGFR-2. <b><i>Results:</i></b> In 58 patients, the median follow-up was 22.1 months. Toxicity was manageable and consistent with what was known for each agent separately. There was a trend toward a prolonged PFS with bevacizumab compared to chemotherapy only (7.0 vs. 5.2 months; adjusted HR = 0.60; 95% CI 0.33–1.08; <i>p</i> = 0.09), but there was no effect on OS. In this small study, pVEGFR-2 concentration did not predict a bevacizumab PFS benefit. Both the intention-to-treat analysis and the per-protocol analysis did not yield a significant treatment-by-biomarker test for interaction (<i>p</i>_interaction = 0.69). <b><i>Conclusions:</i></b>CC and CC+B are safe first-line regimens for mTNBC and the side effects are consistent with those known for each individual agent. pVEGFR-2 concentration did not predict a bevacizumab PFS benefit.

Some Forms of Collectively Bringing About or ‘Seeing to it that’

One of the best known approaches to the logic of agency are the ‘stit’ (‘seeing to it that’) logics. Often, it is not the actions of an individual agent that bring about a certain outcome but the joint actions of a set of agents, collectively. Collective agency has received comparatively little attention in ‘stit’. The paper maps out several different forms, several different senses in which a particular set of agents, collectively, can be said to bring about a certain outcome, and examines how these forms can be expressed in ‘stit’ and stit-like logics. The outcome that is brought about may be unintentional, and perhaps even accidental; the account deliberately ignores aspects such as joint intention, communication between agents, awareness of other agents’ intentions and capabilities, even the awareness of another agent’s existence. The aim is to investigate what can be said about collective agency when all such considerations are ignored, besides mere consequences of joint actions. The account will be related to the ‘strictly stit’ of Belnap and Perloff (Annals of Mathematics and Artificial Intelligence 9(1–2), 25–48 1993) and their suggestions concerning ‘inessential members’ and ‘mere bystanders’. We will adjust some of those conjectures and distinguish further between ‘potentially contributing bystanders’ and ‘impotent bystanders’.