Approximating Stackelberg Equilibrium in Anti-UAV Jamming Markov Game with Hierarchical Multi-Agent Deep Reinforcement Learning Algorithm

In order to avoid the malicious jamming of the intelligent unmanned aerial vehicle (UAV) to ground users in the downlink communications, a new anti-UAV jamming strategy based on multi-agent deep reinforcement learning is studied in this paper. In this method, ground users aim to learn the best mobile strategies to avoid the jamming of UAV. The problem is modeled as a Stackelberg game to describe the competitive interaction between the UAV jammer (leader) and ground users (followers). To reduce the computational cost of equilibrium solution for the complex game with large state space, a hierarchical multi-agent proximal policy optimization (HMAPPO) algorithm is proposed to decouple the hybrid game into several sub-Markov games, which updates the actor and critic network of the UAV jammer and ground users at different time scales. Simulation results suggest that the hierarchical multi-agent proximal policy optimization -based anti-jamming strategy achieves comparable performance with lower time complexity than the benchmark strategies. The well-trained HMAPPO has the ability to obtain the optimal jamming strategy and the optimal anti-jamming strategies, which can approximate the Stackelberg equilibrium (SE).

The service pricing strategies and the strategic behavior of customers in an unobservable Markovian queue with unreliable server

A Stackelberg game is used to study the service pricing and the strategic behavior of customers in an unreliable and totally unobservable M/M/1 queue under a reward-cost structure. At the first stage, the server manager, acting as a leader, chooses a service price and, at the second stage, a customer, arriving at the system and acting as a follower, chooses to join the system or an outside opportunity, knowing only the service price imposed by the server manager and the system parameters. We show that the constructed game admits an equilibrium and we give explicit forms of server manager and customers equilibrium behavioral strategies. The results of the proposed model show that the assumption that customers are risk-neutral is fundamental for the standard approach usually used. Moreover, we determine the socially optimal price that maximizes the social welfare and we compare it to the Stackelberg equilibrium. We illustrate, by numerical examples, the effect of some system parameters on the equilibrium service price and the revenue of the server manager.

CrowdBox: Crowdsourced Network-in-Box Recruitment for Edge Computing-Enabled Industrial Internet of Things

The rapidly increasing number of smart devices deployed in the Industrial Internet of Things (IIoT) environment has been witnessed. To improve communication efficiency, edge computing-enabled Industrial Internet of Things (E-IIoT) has gained attention recently. Nevertheless, E-IIoT still cannot conquer the rapidly increasing communication demands when hundreds of millions of IIoT devices are connected at the same time. Considering the future 6G environment where smart network-in-box (NIB) nodes are everywhere (e.g., deployed in vehicles, buses, backpacks, etc.), we propose a crowdsourcing-based recruitment framework, leveraging the power of the crowd to provide extra communication resources and enhance the communication capabilities. We creatively treat NIB nodes as edge layer devices, and CrowdBox is devised using a Stackelberg game where the E-IIoT system is the leader, and the NIB nodes are the followers. CrowdBox can calculate the optimal reward to reach the unique Stackelberg equilibrium where the utility of E-IIoT can be maximized while none of the NIB nodes can improve its utility by deviating from its strategy. Finally, we evaluate the performance of CrowdBox with extensive simulations with various settings, and it shows that CrowdBox outperforms the compared algorithms in improving system utility and attracting more NIB nodes.

A Game Model for Medical Service Pricing Based on the Diagnosis Related Groups

Background: Currently there are various issues that exist in the medical institutions in China as a result of the price-setting in DRGs, which include the fact that medical institutions tend to choose patients and that the payment standard for complex cases cannot reasonably compensate the cost.Objective: The main objective is to prevent adverse selection problems in the operations of a diagnosis-related groups (DRGs) system with the game pricing model for scientific and reasonable pricing.Methods: The study proposes an improved bargaining game model over three stages, with the government and patients forming an alliance. The first stage assumes the alliance is the price maker in the Stackelberg game to maximize social welfare. Medical institutions are a price taker and decide the level of quality of medical service to maximize their revenue. A Stackelberg equilibrium solution is obtained. The second stage assumes medical institutions dominate the Stackelberg game and set an optimal service quality for maximizing their revenues. The alliance as the price taker decides the price to maximize the social welfare. Another Stackelberg equilibrium solution is achieved. The final stage establishes a Rubinstein bargaining game model to combine the Stackelberg equilibrium solutions in the first and second stage. A new equilibrium between the alliance and medical institutions is established.Results: The results show that if the price elasticity of demand increases, the ratio of cost compensation on medical institutions will increase, and the equilibrium price will increase. The equilibrium price is associated with the coefficient of patients' quality preference. The absolute risk aversion coefficient of patients affects government compensation and total social welfare.Conclusion: In a DRGs system, considering the demand elasticity and the quality preference of patients, medical service pricing can prevent an adverse selection problem. In the future, we plan to generalize these models to DRGs pricing systems with the effects of competition of medical institutions. In addition, we suggest considering the differential compensation for general hospitals and community hospitals in a DRGs system, in order to promote the goal of hierarchical diagnosis and treatment.

Stackelberg Population Dynamics: A Predictive-Sensitivity Approach

Hierarchical decision-making processes traditionally modeled as bilevel optimization problems are widespread in modern engineering and social systems. In this work, we deal with a leader with a population of followers in a hierarchical order of play. In general, this problem can be modeled as a leader–follower Stackelberg equilibrium problem using a mathematical program with equilibrium constraints. We propose two interconnected dynamical systems to dynamically solve a bilevel optimization problem between a leader and follower population in a single time scale by a predictive-sensitivity conditioning interconnection. For the leader’s optimization problem, we developed a gradient descent algorithm based on the total derivative, and for the followers’ optimization problem, we used the population dynamics framework to model a population of interacting strategic agents. We extended the concept of the Stackelberg population equilibrium to the differential Stackelberg population equilibrium for population dynamics. Theoretical guarantees for the stability of the proposed Stackelberg population learning dynamics are presented. Finally, a distributed energy resource coordination problem is solved via pricing dynamics based on the proposed approach. Some simulation experiments are presented to illustrate the effectiveness of the framework.

A Game-Based Scheme for Resource Purchasing and Pricing in MEC for Internet of Things

Mobile edge computing (MEC) is emerging as a promising paradigm to support the applications of Internet of Things (IoT). The edge servers bring computing resources to the edge of the network, so as to meet the delay requirements of the IoT devices’ service requests. At the same time, the edge servers can gain profit by leasing computing resources to IoT users and realize the allocation of computing resources. How to determine a reasonable resource leasing price for the edge servers and how to determine the number of resource purchased by users with different needs is a challenging problem. In order to solve the problem, this paper proposes a game-based scheme for resource purchasing and pricing aiming at maximizing user utility and server profit. The interaction between users and the edge servers is modeled based on Stackelberg game theory. The properties of incentive compatibility and envy freeness are theoretically proved, and the existence of Stackelberg equilibrium is also proved. A game-based user resource purchasing algorithm called GURP and a game-based server resource pricing algorithm called GSRP are proposed. It is theoretically proven that solutions of the proposed algorithms satisfy the individual rationality property. Finally, simulation experiments are carried out, and the experimental results show that the GURP algorithm and the GSRP algorithm can quickly converge to the optimal solutions. Comparison experiments with the benchmark algorithms are also carried out, and the experimental results show that the GURP algorithm and the GSRP algorithm can maximize user utility and server profit.

Optimally Deceiving a Learning Leader in Stackelberg Games

Recent results have shown that algorithms for learning the optimal commitment in a Stackelberg game are susceptible to manipulation by the follower. These learning algorithms operate by querying the best responses of the follower, who consequently can deceive the algorithm by using fake best responses, typically by responding according to fake payoffs that are different from the actual ones. For this strategic behavior to be successful, the main challenge faced by the follower is to pinpoint the fake payoffs that would make the learning algorithm output a commitment that benefits them the most. While this problem has been considered before, the related literature has only focused on a simple setting where the follower can only choose from a finite set of payoff matrices, thus leaving the general version of the problem unanswered. In this paper, we fill this gap by showing that it is always possible for the follower to efficiently compute (near-)optimal fake payoffs, for various scenarios of learning interaction between the leader and the follower. Our results also establish an interesting connection between the follower’s deception and the leader’s maximin utility: through deception, the follower can induce almost any (fake) Stackelberg equilibrium if and only if the leader obtains at least their maximin utility in this equilibrium.

Hierarchical games with feedback on the assumption of a lower-level player's benevolence

A new optimality principle is proposed that generalizes the Stackelberg equilibrium principle. Its connection with the classical definition is investigated. The technique of working with the new definition is discussed. As an example, solutions are found in two hierarchical games with feedback.

Optimal Game-Based Energy Management with Renewable Energy for Secure Electric Vehicles Charging in Internet of Things

The integration of smart grid and Internet of Things (IoT) has been facilitated with the proliferation of electric vehicles (EVs). However, due to EVs’ random mobility and different interests of energy demand, there exists a significant challenge to optimally schedule energy supply in IoT. In this paper, we propose a secure game theoretic scheme for charging EVs supplied by mobile charging stations (MCSs) in IoT, considering the dynamic renewable energy source. Firstly, the charging system composed of MCSs is developed to implement the charging service. Secondly, when the secure charging scheme of EV users is designed, the utility function of each entity in the charging system is formulated to express the trading relationship between EV users and MCSs. Moreover, with consideration of the competition and cooperation, we propose a Stackelberg game framework with sub-noncooperative optimization. Thirdly, the existence and uniqueness of both Stackelberg equilibrium (SE) and Nash equilibrium (NE) are theoretically analyzed and proved. Through the presented distributed energy scheduling algorithm, we can achieve the optimal solution. Finally, numerical results demonstrate the effectiveness and efficiency of our proposal through comparison with other existing schemes.

Dynamic Evolution of the Zinc-Nickel Battery Industry and Evidence from China

This paper analyzes the development prospects of zinc-nickel battery industry, further investigates the industry competition in existing markets by mathematical modeling, calculates the equilibrium price and profit of the oligarch competition by using the method of Stackelberg equilibrium and Nash equilibrium, and makes a comparison between them. Then, we study and model the case of renting and selling simultaneously. In addition, we also study the impact of futures prices on the zinc-nickel battery companies and carry out numerical simulation. At the end of this paper, we analyze the location of zinc-nickel battery enterprises and the industry development under the COVID-19 pandemic. The finding show that the reduction of raw material cost is of great help to the development of the zinc-nickel battery industry.