E xploration E xploitation Problem in Policy Based Deep Reinforcement Learning for Episodic and Continuous Environments

Reinforcement learning is an artificial intelligence paradigm that enables intelligent agents to accrue environmental incentives to get superior results. It is concerned with sequential decision-making problems which offer limited feedback. Reinforcement learning has roots in cybernetics and research in statistics, psychology, neurology, and computer science. It has piqued the interest of the machine learning and artificial intelligence groups in the last five to ten years. It promises that it allows you to train agents using rewards and penalties without explaining how the task will be completed. The RL issue may be described as an agent that must make decisions in a given environment to maximize a specified concept of cumulative rewards. The learner is not taught which actions to perform but must experiment to determine which acts provide the greatest reward. Thus, the learner has to actively choose between exploring its environment or exploiting it based on its knowledge. The exploration-exploitation paradox is one of the most common issues encountered while dealing with Reinforcement Learning algorithms. Deep reinforcement learning is the combination of reinforcement learning (RL) and deep learning. We describe how to utilize several deep reinforcement learning (RL) algorithms for managing a Cartpole system used to represent episodic environments and Stock Market Trading, which is used to describe continuous environments in this study. We explain and demonstrate the effects of different RL ideas such as Deep Q Networks (DQN), Double DQN, and Dueling DQN on learning performance. We also look at the fundamental distinctions between episodic and continuous activities and how the exploration-exploitation issue is addressed in their context.

Dynamics of Parasympathetic Activity in Violent Incarcerated Offenders Before, During, and in Recovery From An Emotional Inhibition Task

Dynamics of the autonomic nervous system (ANS) are hypothesized to play a role in the emergence of interpersonal violence. In the present study, we examined continuous activities of the inhibitory parasympathetic pathway of the ANS through the root mean square of successive differences between normal heartbeats (RMSSD) in 22 male offenders who committed interpersonal violence and 24 matched controls from the general population across three successive phases: resting baseline, while performing an emotional Go/No-Go task, and post-task recovery. Results showed that across the three phases, the offender group presented lower RMSSD at baseline (pFDR = .003; Cohen’s d = -1.11), but similar levels during the task, attributed to a significant increase in their RMSSD level (pFDR = .027, Cohen’s d = -1.26). During recovery, no distinction between the two groups was found, but although both groups showed signs of recovering toward baseline values. These findings suggest that violent incarcerated offenders can flexibly engage parasympathetic resources to meet environmental challenges. This underscores the necessity of considering parasympathetic dynamics and its respective mobilization/flexibility to better understand ANS profiles underlying interpersonal violence and designed more tailored intervention.

ADMINISTRATIVE REGULATIONS OF STATE FUNCTIONS AT THE PRESENT STAGE: DYNAMICS OF CHANGES IN THE APPROACH TO THE CONCEPT AND VARIETIES AT THE LEGAL REGULATORY

In this article the author analyses the development of administrative regulations in the system of modern public administration. The state of administrative regulations and their institutionalisation as a result of the administrative reform is noted. It has been identified and found that there has now been a change in the approach to the delineation of the administrative regulations of the executive authorities - before 2018, administrative regulations were divided into service regulations and function regulations, where service regulations are related to requests by private persons to the state represented by its bodies and officials - and function regulations are related to the implementation of continuous activities to perform assigned powers and exercise the established competence, after 2018, administrative regulations are divided into service regulations and control (supervision) regulations. According to the author, the established approach has allowed the above varieties of regulations to be merged into such a group of types of administrative regulations as administrative regulations for the implementation of state functions. Also the author concludes that it is necessary to adopt a special legislative act systemising the functions of the executive authorities and the administrative procedures they carry out in a single logical connection.

Continuous Human Activity Recognition through Parallelism LSTM with Multi-Frequency Spectrograms

According to the real-living environment, radar-based human activity recognition (HAR) is dedicated to recognizing and classifying a sequence of activities rather than individual activities, thereby drawing more attention in practical applications of security surveillance, health care and human–computer interactions. This paper proposes a parallelism long short-term memory (LSTM) framework with the input of multi-frequency spectrograms to implement continuous HAR. Specifically, frequency-division short-time Fourier transformation (STFT) is performed on the data stream of continuous activities collected by a stepped-frequency continuous-wave (SFCW) radar, generating spectrograms of multiple frequencies which introduce different scattering properties and frequency resolutions. In the designed parallelism LSTM framework, multiple parallel LSTM sub-networks are trained separately to extract different temporal features from the spectrogram of each frequency and produce corresponding classification probabilities. At the decision level, the probabilities of activity classification from these sub-networks are fused by addition as the recognition output. To validate the proposed method, an experimental data set is collected by using an SFCW radar to monitor 11 participants who continuously perform six activities in sequence with three different transitions and random durations. The validation results demonstrate that the average accuracies of the designed parallelism unidirectional LSTM (Uni-LSTM) and bidirectional LSTM (Bi-LSTM) based on five frequency spectrograms are 85.41% and 96.15%, respectively, outperforming traditional Uni-LSTM and Bi-LSTM networks with only a single-frequency spectrogram by 5.35% and 6.33% at least. Additionally, the recognition accuracy of the parallelism LSTM network reveals an upward trend as the number of multi-frequency spectrograms (namely the number of LSTM subnetworks) increases, and tends to be stable when the number reaches 4.

Pengaruh Lingkungan Keluarga Terhadap Pembentukan Tingkah Laku Anak

Education through activities outside of school is a learning activity that does not require tiered and continuous activities. The family is the main foundation that can influence the attitudes and behavior of a child. Children's behavior includes attitudes, speech, actions and perceptions of children in the process of development so that they become a character. The character appears through thoughts obtained based on experience and observations from where a person grows and develops, starting from the first meaning that is understood based on a person's words, gestures and behavior. The purpose of this study was to determine the influence of the family environment on the formation of children's character. The method used is descriptive quantitative, with a correlational study approach. After testing the hypothesis which reads: 'there is an influence of the family environment on the formation of children's behavior', it was found that the hypothesis is acceptable, meaning that the family environment has an effect of 13.8%. These results indicate that there is an influence of the family environment on the formation and changes in children's behavior, the family is the initial foundation for children to interact with the outside environment, both the living environment and the school environment.

Effective Customer Relationship Management: A Key to Success for Banks

Generally, consumers do not change their banks unless serious problems occur. The philosophy, culture and organization of financial institutions were grounded in this assumption and reflected in their marketing policies, which were product and transaction-oriented, reactionary, focused on discrete rather than continuous activities. With the advent of new technologies in the business of bank, such as Internet Banking and ATMs, now customers can freely choose any bank for their transactions. The pressures of competitive and dynamic markets have contributed to the growth of CRM in the Financial Services Sector. Also, before the Internet revolution, consumers largely selected their banks based on how convenient the location of bank's branches was to their homes or offices. With the advent of new technologies in the business of bank, such as Internet banking and ATMs, now customers can freely choose any bank for their transactions. Thus, the customer base of banks has increased, and so has the choices of customers for selecting the banks.

SECURITY CULTURE AS A FUNDAMENTAL HUMAN NEED

Modern society exists in conditions of evident contradictions and complex problems. In addition to chronic poverty, ethnic violence, regional war conflicts, the expansion of terrorism and organized crime, we have long faced climate change and environmental degradation. The destruction of ecosystems has become a kind of crisis multiplier, which has deep implications for international peace and stability. In the age of scientific and technological prosperity in the search for answers to old and new security threats, we are also facing a latent moral stumbling on a global level, which is reflected in the phenomenon of “alienation of man from man”, which is the cause of many evils. Addiction diseases, where drug addiction, smoking and alcoholism predominate, dominate among the young population and develop serious diseases with fatal outcomes. It is known that many important aspects of human development also relate to the security of people, who naturally strive for “freedom from fear and freedom from poverty.” Therefore, building and maintaining a security culture, on the one hand, is not a simple process, and on the other hand, it is like any other process that people manage and which involves continuous activities, planning, control and audit. The sentence, “what kind of security culture you have, such is the character of your society”, has gained real meaning in the current time.

Motion and Dance as a Type of Therapy With Working With Children/People With Developmental Difficulties

Besides the standard treatments that are nowadays used with working with children/people with developmental difficulties and the obligatory aspect of schooling, active free time is also very important for maintaining and improving psychological, motoric and social abilities of children/people with developmental difficulties. Active free time contributes to developing of creativity, satisfaction of one’s own life, personality development, and therefore it improves the quality of life itself. The main aim of this research is observation and assessment of motoric abilities of children/people with developmental difficulties, before and after conducting continuous activities in the form of traditional dance i.e. play games as well as the influence on the quality of life. The evaluation of motoric abilities has been done by defectologists before performing an adjusted work program and after completion, in which, using quantitative and qualitative analysis, the progress has been established in segments such as motoric body control, motion coordination of upper extremities, motion coordination of upper and lower extremities and motion coordination of upper and lower extremities by rhythm. Active participation and cooperation between participants, influenced improvement of social skills and the easier overcoming of problems in the group, creative and active free time and improvement of life quality. Based on obtained result, one should strive to adopt new creative ways of working with children/people with developmental difficulties in order to gain comprehensive holistic and rehabilitation approach.

Memory for Dynamic Events When Event Boundaries Are Accentuated With Emotional Stimuli

Event boundaries are important moments throughout an ongoing activity that influence perception and memory. They allow people to parse continuous activities into meaningful events, encode the temporal sequence of events and bind event information together in episodic memory (DuBrow & Davachi, 2013). Thus, drawing attention to event boundaries may facilitate these important perceptual and encoding processes. In the current study, we used emotionally arousing stimuli to guide attention to event boundaries because this type of stimulus has been shown to influence perception and attention. We evaluated whether accentuating event boundaries with commercials improves memory and whether emotional stimuli further enhance this effect. A total of 97 participants watched a television episode in which we manipulated commercial break locations (boundary, non-boundary, no commercial) and the type of commercial (emotional, neutral) and then completed memory tasks. Overall, placing emotionally arousing commercials at event boundaries increased memory for the temporal order of events, but no other effects of accentuating event boundaries were observed. Thus, drawing attention to event boundaries—via emotionally charged commercials—increases the likelihood that people will perceive the change in events, update their mental model accordingly and better integrate temporal information from the just-encoded event.

Method for Base Estimation of Construction Time for Linear Projects in Front-end Project Phases

Even though horizontally linear projects have low complexity schedules, they are still not successful in meeting planned time. The deadlines are mostly based on estimations done in front-end project development when limited data are available. Early time estimation models in literature rely on few variables and, almost in all cases, one of them is the estimated cost. Early cost estimations can significantly deviate from actual costs and thus lead to unreliable time estimation. Time estimation models based on neural network and other alternative methods require databases and software, which complicates the process of time estimation. The purpose of this paper is to bridge the gap of scarce time estimation models and unreliable time estimates by developing a new method for time estimation. This research has been done on one large sewer system project. The case study shows how to extract several continuous activities for a pipeline project chosen from a sewer system. Moreover, a new algorithm for the calculation of project duration is devised based on the existing equation related to the linear scheduling method, and this algorithm works with continuous activities. The new method for construction time estimation is based on the extraction of linear continuous activities, usage of the algorithm for identification of minimal buffer between activities, and calculation of the project duration. To verify the algorithm, this method is used on another pipeline project from a sewer system. The limitation is that this method can be used only for base estimation. Further research needs to be done to include uncertainties and risks in the method.