E xploration E xploitation Problem in Policy Based Deep Reinforcement Learning for Episodic and Continuous Environments

Reinforcement learning is an artificial intelligence paradigm that enables intelligent agents to accrue environmental incentives to get superior results. It is concerned with sequential decision-making problems which offer limited feedback. Reinforcement learning has roots in cybernetics and research in statistics, psychology, neurology, and computer science. It has piqued the interest of the machine learning and artificial intelligence groups in the last five to ten years. It promises that it allows you to train agents using rewards and penalties without explaining how the task will be completed. The RL issue may be described as an agent that must make decisions in a given environment to maximize a specified concept of cumulative rewards. The learner is not taught which actions to perform but must experiment to determine which acts provide the greatest reward. Thus, the learner has to actively choose between exploring its environment or exploiting it based on its knowledge. The exploration-exploitation paradox is one of the most common issues encountered while dealing with Reinforcement Learning algorithms. Deep reinforcement learning is the combination of reinforcement learning (RL) and deep learning. We describe how to utilize several deep reinforcement learning (RL) algorithms for managing a Cartpole system used to represent episodic environments and Stock Market Trading, which is used to describe continuous environments in this study. We explain and demonstrate the effects of different RL ideas such as Deep Q Networks (DQN), Double DQN, and Dueling DQN on learning performance. We also look at the fundamental distinctions between episodic and continuous activities and how the exploration-exploitation issue is addressed in their context.

CHAT BOTS AS A TOOL FOR POPULARIZATION OF MEDIA MATERIALS

The development of social networks, personalized assistants, chatbots, and computational algorithms makes the media think about new prospects. Users have long departed from the format of receiving information through one channel. Today, audiences consume content through a variety of channels, from paper noses to receiving data through eyepiece lenses. Under these conditions, journalism must experiment, because no one can predict the trajectory of further transformations of the media system. Bot programs have become indispensable assistants for media outlets. Chatbots are specially programmed computer programs that work according to a certain algorithm of actions. Nowadays, there are new types of bot programs - simple and smart. After all, with their help, you can quickly publish posts on social networks and messengers, search for the most popular topics, quickly answer questions from the audience, track the reaction to posts, and more. As well as smart programs can write journalistic material. And they work 24/7 and process a lot of information in seconds. Such bot programs provide additional tools for the media, which facilitate and optimize the work of journalists. The first bot programs were created in the 60-the 70s of the XX century. And in messengers chatbots were in the late '90s of the XX century - ICQ. But a significant impetus for the use and development of chatbots was the messenger Telegram, which has a whole bot farm. Such programs have prospects, so the editors of the leading media invest in the creation and training of such bots because they optimize the editorial process. In the study, we analyze the prospects of using media chatbots to disseminate and promote their materials and optimize editorial processes.

Universal Basic Income as a Way of Redistribution of Experience between Individuals and Groups

This article reviews the possible role of the universal basic income in the transformation of experience in gender and age perspectives. The universal basic income has been particularly hotly debated in recent decades. Regardless of the position, the common tone of the debates is the imperative “we must experiment.” Such a close interest in the universal basic income derives from the fact that it threatens to change the “generic” for humans situation of finiteness of resources and the need to work. Thus, it is able to change the experience of what it means to be human. Since the universal basic income allows to separate labor from income, it is likely that its introduction will return value to the currently stigmatized or devalued types of labor, such as child care, elderly care or domestic work. It creates opportunities for experience redistribution in gender perspective: care and leaving (career break), affective connection, and sensitivity could become the business of both parents, not just mothers. Another experience redistribution is possible in age perspective. Along with automation of labor, population ageing is a universal process that will sooner or later affect all the countries. Alarmist narratives that present this process as a threat and a problem now prevail. They are largely based on outdated ideas about old age and what it means to be old. However, in fact, ageing is the maturation of the population as a whole. With a proper re-evaluation of the meaning and significance of old age, the introduction of the universal basic income could create material conditions for the transfer of experience from the elderly to the younger – for the first time since traditional societies.

An optimization for reducing the size of an existing urban-like monitoring network for retrieving an unknown point source emission

This study presents an optimization methodology for reducing the size of an existing monitoring network of the sensors measuring polluting substances in an urban-like environment in order to estimate an unknown emission source. The methodology is presented by coupling the simulated annealing (SA) algorithm with the renormalization inversion technique and the computational fluid dynamics (CFD) modeling approach. This study presents an application of the renormalization data-assimilation theory for optimally reducing the size of an existing monitoring network in an urban-like environment. The performance of the obtained reduced optimal sensor networks is analyzed by reconstructing the unknown continuous point emission using the concentration measurements from the sensors in that optimized network. This approach is successfully applied and validated with 20 trials of the Mock Urban Setting Test (MUST) tracer field experiment in an urban-like environment. The main results consist of reducing the size of a fixed network of 40 sensors deployed in the MUST experiment. The optimal networks in the MUST urban region are determined, which makes it possible to reduce the size of the original network (40 sensors) to ∼1/3 (13 sensors) and 1∕4 (10 sensors). Using measurements from the reduced optimal networks of 10 and 13 sensors, the averaged location errors are obtained as 19.20 and 17.42 m, respectively, which are comparable to the 14.62 m obtained from the original 40-sensor network. In 80 % of the trials with networks of 10 and 13 sensors, the emission rates are estimated within a factor of 2 of the actual release rates. These are also comparable to the performance of the original network, whereby in 75 % of the trials the releases were estimated within a factor of 2 of the actual emission rates.

Corrosion of Reinforce Bars in RC Constructions

In this article presented results of researching corrosion of steel bars in aggressive environment in time under loading. For researching were used special equipment. The experience and research works shown that steel bars in the crack cross-section area can be corrode. With increasing width of crack in re-bars and power of aggressive of environment increased the level of corrosion and decreased time of progress. The level of danger of corrosion in the crack in depend of specialty of steel bars. It is geometry parameters of steel bars and characteristic of corrosive behaviour. The general tendency of the influence of various defects on the strength of steels is widely studied experimentally and theoretically only for geometrically correct stress concentrators. For damages that are irregular in shape, such as corrosion ulcers, significantly less researching in each case must experiment to find their effect on the mechanical properties of steels. In this work the influence of simultaneous action of the aggressive environment and loading on strength of steel re-bars has been described.

A Novel adaptative Discrete Cuckoo Search Algorithm for parameter optimization in computer vision

Computer vision applications require choosing operators and their parameters, in order to provide the best outcomes. Often, the users quarry on expert knowledge and must experiment many combinations to find manually the best one. As performance, time and accuracy are important, it is necessary to automate parameter optimization at least for crucial operators. In this paper, a novel approach based on an adaptive discrete cuckoo search algorithm (ADCS) is proposed. It automates the process of algorithms’ setting and provides optimal parameters for vision applications. This work reconsiders a discretization problem to adapt the cuckoo search algorithm and presents the procedure of parameter optimization. Some experiments on real examples and comparisons to other metaheuristic-based approaches: particle swarm optimization (PSO), reinforcement learning (RL) and ant colony optimization (ACO) show the efficiency of this novel method.

The Present Visual Equipment System and Its Application in fMRI

The visual system is the part of the central nervous system that gives organisms the ability to process visual details and enables the formation of several non-image photo response functions. It detects and explains information from visible by the light to build a representation of the surrounding environment. One reason why the visual system is so important is that it enables us to perceive information at a distance. We need not be in immediate contact with a stimulus to process it. We must experiment with visual equipment to understand how we process visual information. This article summarizes current visual system equipment and how this equipment can be used to determine how the visual system functions.

Financing Experimentation

Entrepreneurs must experiment to learn how good they are at a new activity. What happens when the experimentation is financed by a lender? Under common scenarios, i.e., when there is the opportunity to learn by “starting small” or when “noncompete” clauses cannot be enforced ex post, we show that financing experimentation can become harder precisely when it is more profitable, i.e., for lower values of the known arm and for more optimistic priors. Endogenous collateral requirements (like those frequently observed in microcredit schemes) are shown to be part of the optimal contract. (JEL D82, G21, G32, L25, L26)