Toward Robust Monitoring of Malicious Outbreaks

Recently, diffusion processes in social networks have attracted increasing attention within computer science, marketing science, social sciences, and political science. Although the majority of existing works focus on maximizing the reach of desirable diffusion processes, we are interested in deploying a group of monitors to detect malicious diffusion processes such as the spread of computer worms. In this work, we introduce and study the [Formula: see text]-Monitoring Game} on networks. Our game is composed of two parties an attacker and a defender. The attacker can launch an attack by distributing a limited number of seeds (i.e., virus) to the network. Under our [Formula: see text]-Monitoring Game, we say an attack is successful if and only if the following two conditions are satisfied: (1) the outbreak/propagation reaches at least α individuals without intervention, and (2) it has not been detected before reaching β individuals. Typically, we require that β is no larger than α in order to compensate the reaction delays after the outbreak has been detected. On the other end, the defender’s ultimate goal is to deploy a set of monitors in the network that can minimize attacker’s success ratio in the worst-case. (We also extend the basic model by considering a noisy diffusion model, where the propagation probabilities on each edge could vary within an interval.) Our work is built upon recent work in security games, our adversarial setting provides robust solutions in practice. Summary of Contribution: Although the diffusion processes in social networks have been extensively studied, most existing works aim at maximizing the reach of desirable diffusion processes. We are interested in deploying a group of monitors to detect malicious diffusion processes, such as the spread of computer worms. To capture the impact of model uncertainty, we consider a noisy diffusion model in which the propagation probabilities on each edge could vary within an interval. Our work is built upon recent work in security games; our adversarial setting leads to robust solutions in practice.

Temporal Induced Self-Play for Stochastic Bayesian Games

One practical requirement in solving dynamic games is to ensure that the players play well from any decision point onward. To satisfy this requirement, existing efforts focus on equilibrium refinement, but the scalability and applicability of existing techniques are limited. In this paper, we propose Temporal-Induced Self-Play (TISP), a novel reinforcement learning-based framework to find strategies with decent performances from any decision point onward. TISP uses belief-space representation, backward induction, policy learning, and non-parametric approximation. Building upon TISP, we design a policy-gradient-based algorithm TISP-PG. We prove that TISP-based algorithms can find approximate Perfect Bayesian Equilibrium in zero-sum one-sided stochastic Bayesian games with finite horizon. We test TISP-based algorithms in various games, including finitely repeated security games and a grid-world game. The results show that TISP-PG is more scalable than existing mathematical programming-based methods and significantly outperforms other learning-based methods.

Solving Large-Scale Extensive-Form Network Security Games via Neural Fictitious Self-Play

Securing networked infrastructures is important in the real world. The problem of deploying security resources to protect against an attacker in networked domains can be modeled as Network Security Games (NSGs). Unfortunately, existing approaches, including the deep learning-based approaches, are inefficient to solve large-scale extensive-form NSGs. In this paper, we propose a novel learning paradigm, NSG-NFSP, to solve large-scale extensive-form NSGs based on Neural Fictitious Self-Play (NFSP). Our main contributions include: i) reforming the best response (BR) policy network in NFSP to be a mapping from action-state pair to action-value, to make the calculation of BR possible in NSGs; ii) converting the average policy network of an NFSP agent into a metric-based classifier, helping the agent to assign distributions only on legal actions rather than all actions; iii) enabling NFSP with high-level actions, which can benefit training efficiency and stability in NSGs; and iv) leveraging information contained in graphs of NSGs by learning efficient graph node embeddings. Our algorithm significantly outperforms state-of-the-art algorithms in both scalability and solution quality.