The Stackelberg Games of Water Extraction with Myopic Agents

We consider a discrete-time, infinite-horizon dynamic game of groundwater extraction. A Water Agency charges an extraction cost to water users and controls the marginal extraction cost so that it depends not only on the level of groundwater but also on total water extraction (through a parameter [Formula: see text] that represents the degree of strategic interactions between water users) and on rainfall (through parameter [Formula: see text]). The water users are selfish and myopic, and the goal of the agency is to give them incentives so as to improve their total discounted welfare. We look at this problem in several situations. In the first situation, the parameters [Formula: see text] and [Formula: see text] are considered to be fixed over time. The first result shows that when the Water Agency is patient (the discount factor tends to 1), the optimal marginal extraction cost asks for strategic interactions between agents. The contrary holds for a discount factor near 0. In a second situation, we look at the dynamic Stackelberg game where the Agency decides at each time what cost parameter they must announce. We study theoretically and numerically the solution to this problem. Simulations illustrate the possibility that threshold policies are good candidates for optimal policies.

Reinforcement Learning for Interference Coordination Stackelberg Games in Heterogeneous Cellular Networks

In future heterogeneous cellular networks with small cells, such as D2D and relay, interference coordination between macro cells and small cells should be addressed through effective resource allocation and power control. The two-step Stackelberg game is a widely used and feasible model for resource allocation and power control problem formulation. Both in the follower games for small cells and in the leader games for the macro cell, the cost parameters are a critical variable for the performance of Stackelberg game. Previous studies have failed to adequately address the optimization of cost parameters. This paper presents a reinforcement learning approach for effectively training cost parameters for better system performance. Furthermore, a two-stage pretraining plus ε -greedy algorithm is proposed to accelerate the convergence of reinforcement learning. The simulation results can demonstrate that compared with the three beachmarking algorithms, the proposed algorithm can enhance average throughput of all users and cellular users by up to 7% and 9.7%, respectively.

Optimally Deceiving a Learning Leader in Stackelberg Games

Recent results have shown that algorithms for learning the optimal commitment in a Stackelberg game are susceptible to manipulation by the follower. These learning algorithms operate by querying the best responses of the follower, who consequently can deceive the algorithm by using fake best responses, typically by responding according to fake payoffs that are different from the actual ones. For this strategic behavior to be successful, the main challenge faced by the follower is to pinpoint the fake payoffs that would make the learning algorithm output a commitment that benefits them the most. While this problem has been considered before, the related literature has only focused on a simple setting where the follower can only choose from a finite set of payoff matrices, thus leaving the general version of the problem unanswered. In this paper, we fill this gap by showing that it is always possible for the follower to efficiently compute (near-)optimal fake payoffs, for various scenarios of learning interaction between the leader and the follower. Our results also establish an interesting connection between the follower’s deception and the leader’s maximin utility: through deception, the follower can induce almost any (fake) Stackelberg equilibrium if and only if the leader obtains at least their maximin utility in this equilibrium.

Casting Light on the Hidden Bilevel Combinatorial Structure of the Capacitated Vertex Separator Problem

Exploiting Bilevel Optimization Techniques to Disconnect Graphs into Small Components In order to limit the spread of possible viral attacks in a communication or social network, it is necessary to identify critical nodes, the protection of which disconnects the remaining unprotected graph into a bounded number of shores (subsets of vertices) of limited cardinality. In the article “'Casting Light on the Hidden Bilevel Combinatorial Structure of the Capacitated Vertex Separator Problem”, Furini, Ljubic, Malaguti, and Paronuzzi provide a new bilevel interpretation of the associated capacitated vertex separator problem and model it as a two-player Stackelberg game in which the leader interdicts (protects) the vertices, and the follower solves a combinatorial optimization problem on the resulting graph. Thanks to this bilevel interpretation, the authors derive different families of strengthening inequalities and show that they can be separated in polynomial time. The ideas exploited in their framework can also be extended to other vertex/edge deletion/insertion problems or graph partitioning problems by modeling them as two-player Stackelberg games to be solved through bilevel optimization.