Temporal Induced Self-Play for Stochastic Bayesian Games

One practical requirement in solving dynamic games is to ensure that the players play well from any decision point onward. To satisfy this requirement, existing efforts focus on equilibrium refinement, but the scalability and applicability of existing techniques are limited. In this paper, we propose Temporal-Induced Self-Play (TISP), a novel reinforcement learning-based framework to find strategies with decent performances from any decision point onward. TISP uses belief-space representation, backward induction, policy learning, and non-parametric approximation. Building upon TISP, we design a policy-gradient-based algorithm TISP-PG. We prove that TISP-based algorithms can find approximate Perfect Bayesian Equilibrium in zero-sum one-sided stochastic Bayesian games with finite horizon. We test TISP-based algorithms in various games, including finitely repeated security games and a grid-world game. The results show that TISP-PG is more scalable than existing mathematical programming-based methods and significantly outperforms other learning-based methods.

A parametric approximation algorithm for spatial group keyword queries

With the application of big data, various queries arise for information retrieval. Spatial group keyword queries aim to find a set of spatial objects that cover the query keywords and minimize a goal function such as the total distance between the objects and the query point. This problem is widely found in database applications and is known to be NP-hard. Efficient algorithms for solving this problem can only provide approximate solutions, and most of these algorithms achieve a fixed approximation ratio (the upper bound of the ratio of an approximate goal value to the optimal goal value). Thus, to obtain a self-adjusting algorithm, we propose an approximation algorithm for achieving a parametric approximation ratio. The algorithm makes a trade-off between the approximation ratio and time consumption enabling the users to assign arbitrary query accuracy. Additionally, it runs in an on-the-fly manner, making it scalable to large-scale applications. The efficiency and scalability of the algorithm were further validated using benchmark datasets.

Mathematical models and soil fertility management software

The article presents the results of studies on parametric approximation in spaces R2 (functions of one variable), R3 (functions of two variables) and Rn(n>3) (functions of three or more variables). Various classes of functions satisfying a priori conditions were studied: f(0, 0, 0)=0, $\mathop {\lim 1}\limits_{{x_i} \to + \infty } \,\,({x_1},\, \ldots ,\,{x_n}) = {c_i}$, ci = cont. Working algorithms and C/C++ software functioning in Microsoft Visual Studio 2019 system in Microsoft Windows 10 environment were developed. The main studies of the authors were aimed at developing effective computational algorithms for constructing approximating functions of two variables from various given classes of three-dimensional data samples (three-dimensional interconnected time series). The article provides a detailed description of the problem statement, introduces classes of approximating functions, provides algorithms for estimating the parameters of approximating functions and a description of the software. The estimation algorithm considered in the article is constructed according to the scheme of the coordinate descent method with optimization of the step length (Gauss-Seidel method).

AUV Link Mobile Ad-Hoc Network Examination

Wireless optical communication systems allow reliable communication in the ultraviolet (UV) range, and protected from intentional suppression and interception, both in the presence and in the absence of direct visibility between the transmitter and the receiver. Currently, the actual and insufficiently explored direction is the development and research of mobile ad-hoc networks (MANET) with a UV communication channel. To analyze the capacity of a mobile network of UV communications with different spatial orientation of network nodes, it is necessary to take into account not only their coordinates and elevation angles but also azimuths. NLOS UV bandwidth models have been developed using three types of modulation: on-off keying (OOK), pulse-phase modulation (PPM), digital pulse interval modulation (DPIM) with a different number of guard intervals. The noted advantages of DPIM, which are most important for the implementation of a mobile self-organizing network with a UV channel, are: it does not require symbolic synchronization of the transmitter and receiver, it requires a smaller frequency band compared to PPM, it has a higher energy efficiency compared to OOK. A parametric approximation of losses in the NLOS UV channel has been performed, taking into account the azimuthal deviations of the transmitter and receiver for specific parameters of the communication system. Has been done a performance simulation of the MANET network with an ultraviolet communication channel Has been calculated values of the bit error rate (BER), and bit rate for a system with different types of modulation with low, medium, and high noise levels in the channel. With the use of parametric approximation of losses in the channel, are analyzed the BER and the bitrate of the UV communication system with an azimuthal deviation within 30 degrees. Based on the obtained simulation results, prospects for the further development of mobile self-organizing networks with a UV channel are indicated to improve their reliability and performance.

Multiagent reinforcement learning using Non-Parametric Approximation

This paper presents a hybrid control proposal for multi-agent systems, where the advantages of the reinforcement learning and nonparametric functions are exploited. A modified version of the Q-learning algorithm is used which will provide data training for a Kernel, this approach will provide a sub optimal set of actions to be used by the agents. The proposed algorithm is experimentally tested in a path generation task in an unknown environment for mobile robots.