Proximal policy optimization with model-based methods

Model-free reinforcement learning methods have successfully been applied to practical applications such as decision-making problems in Atari games. However, these methods have inherent shortcomings, such as a high variance and low sample efficiency. To improve the policy performance and sample efficiency of model-free reinforcement learning, we propose proximal policy optimization with model-based methods (PPOMM), a fusion method of both model-based and model-free reinforcement learning. PPOMM not only considers the information of past experience but also the prediction information of the future state. PPOMM adds the information of the next state to the objective function of the proximal policy optimization (PPO) algorithm through a model-based method. This method uses two components to optimize the policy: the error of PPO and the error of model-based reinforcement learning. We use the latter to optimize a latent transition model and predict the information of the next state. For most games, this method outperforms the state-of-the-art PPO algorithm when we evaluate across 49 Atari games in the Arcade Learning Environment (ALE). The experimental results show that PPOMM performs better or the same as the original algorithm in 33 games.

Defining Post-Pandemic Work and Organizations: The Need for Team Belongingness and Trust

The COVID-19 pandemic has brought forth substantial unrest in the ways in which people work and organize. This had led to disconnection, rapid adaptation, work from home, emergence of a new digital industry, and an opportunity to create anew. This chapter provides a position for the future state of work and organizing, drawing on the belongingness hypothesis, to characterize a revised method of human connection that acknowledges unique differences in online connections. It also explores the role that flexibility and working from home have on organizational outcomes, through changing presenteeism, changes in how people develop trust, and how social resources are deployed. Advancing an understanding of this position creates a possible post-pandemic model of work that acknowledges the current climate and the learnings from before that pandemic. Through genuine acknowledgment of the current and past ways of working, it is possible to build a pathway to heighten employee’s sense of belonging and trust. This will support the return to, and evolution of, a form of normality post-pandemic.

Railway Infrastructure Asset Management Modelling

Railway infrastructure comprises a large collection of diverse and complex systems and structures. Many of them are aging assets and require a significant budget to deliver the maintenance to ensure the required safety and service performance standards. With such a large amount of money spent each year to maintain and renew the network assets, small percentage savings can amount to substantial sums of money. There is therefore a huge incentive to better understand the degradation that assets experience and how these can be effectively managed through the implementation of efficient maintenance strategies. Track and bridges are the most expensive assets to keep in the required condition, and this chapter provides a method to analyse their degradation and build Petri net models to predict the future state of the asset, the number of each maintenance activity which are expected to be undertaken, and their costs experienced by following a particular strategy. Utilising these models enables the quality of the asset management decisions made to be improved and optimised.

Current Advancements and Future Prospects of Alternative Digital Credentials (ADCs) in a Technology-Disrupted World

This chapter focuses on the current and future state of traditional and alternative digital credentials (ADCs) and discusses their importance for economies, societies, and individuals. It provides an overview of the unprecedented technology disruption within tertiary education and its impact on evaluating skills and competencies. Further, it discusses ADCs in greater detail (e.g., similarities, differences, challenges, and potential opportunities), and finally, it concludes the discussion by highlighting ADCs' prospects in the higher education sector.

The Post-COVID-19 Future: State Capability in Ensuring Shared Prosperity

The coronavirus (COVID-19) pandemic has caused profound upheavals in national communities, from humanitarian disasters to unprecedented economic downturns. All the consequences of COVID-19 have made it necessary to understand the reasons for state inefficiency and its traditional functions of ensuring economic balance and financial stability in the period before COVID-19. In fact, inefficiency is a fundamental problem of modern socioeconomic systems. Only a violation of societal integrity can explain why economic isolation and social distancing managed to instantly destroy economic structures, cause a loss in confidence in governments by citizens and increase the potential for protest against the extraordinary actions of nation-states in the fight against COVID-19. At the end of 2020, there was universal agreement about a fundamentally uncertain post-COVID-19 reality. Many progressive specialists have expressed the opinion that the degree of future socioeconomic progress directly depends on the abilities of policymakers to prioritise societal integrity in solving economic problems and achieving the goal of shared prosperity in the future

Multivariate Markov Chain Model for Sales Demand Estimation in a Company

Estimation of the number of demands for a product must be done correctly, so that the company can get maximum profit. Therefore, this study discusses how to estimate the amount of sales demand in a company correctly. The model that will be used to estimate sales demand is the Multivariate Markov Chain Model. This model can estimate the future state by observing the present state. The model requires parameter estimation values ​​first, namely the transition probability matrix and the weighted Markov chain, where in previous studies an estimation of the transition probability matrix has been carried out, so that in this study we will continue to estimate the weighted Markov chain parameters. This model is compatible with 5 data sequences (product types) defined as product 1, product 2, product 3, product 4, and product 5, with 6 conditions (no sales volume, very slow-moving, slow-moving, standard, fast moving, and very fast moving). As the result, the state probability for product 1, product 2 and product 3 in company 1 are stationary at state 6 (very fast moving), product 4 and product 5 are stationary at state 2 (very slow moving).

Vulgar Talk and Learned Reasoning in Berkeley’s Moral and Religious Thought

Berkeley “argues with the learned and speaks with the vulgar.” I use his double maxim to interpret his ethics. My approach is new. The Sermons and Guardian Essays mainly speak to the vulgar and Passive Obedience and Alciphron reason with the learned. The reward of ethics is eternal bliss in a future state: religion and ethics are connected. I study a set of problems: resurrection, eternal life, happiness, benevolence, the goodness of God, and self-love. Divine bliss is unlike any earthly happiness. The idea of law does not support benevolence, even if it is a Christian duty and virtue. God is good, but how to prove it? The learned must study the complex theodicy problem; the vulgar need assurance based on their sensuous experience and fervent hope of eternal bliss. Self-love may be a vital issue to the learned, although the vulgar may not realize their need to overcome it. The main questions concern Berkeley’s two approaches to ethical problems: first, how do their topics differ, and second, are they mutually consistent?

Fire now, fire later: alarm-based systems for prescriptive process monitoring

Predictive process monitoring is a family of techniques to analyze events produced during the execution of a business process in order to predict the future state or the final outcome of running process instances. Existing techniques in this field are able to predict, at each step of a process instance, the likelihood that it will lead to an undesired outcome. These techniques, however, focus on generating predictions and do not prescribe when and how process workers should intervene to decrease the cost of undesired outcomes. This paper proposes a framework for prescriptive process monitoring, which extends predictive monitoring with the ability to generate alarms that trigger interventions to prevent an undesired outcome or mitigate its effect. The framework incorporates a parameterized cost model to assess the cost–benefit trade-off of generating alarms. We show how to optimize the generation of alarms given an event log of past process executions and a set of cost model parameters. The proposed approaches are empirically evaluated using a range of real-life event logs. The experimental results show that the net cost of undesired outcomes can be minimized by changing the threshold for generating alarms, as the process instance progresses. Moreover, introducing delays for triggering alarms, instead of triggering them as soon as the probability of an undesired outcome exceeds a threshold, leads to lower net costs.

The Creation of the Nahal Settlement Enterprise

During the days of the British Mandate in Palestine, the leadership of the Hebrew Yishuv developed the concept of security settlements, i.e., settlements established on the frontier to provide security along the borders of the future state. The concept was put into practice with the Nahal (acronyms of Pioneer Youth Warrior) Brigade settlement enterprise which set up dozens of settlements from 1951 onwards. The first six settlements were founded by ‘lone’ soldiers: immigrants from Eastern Europe and Islamic countries, and natives who did not have a youth movement or pioneering background. The article offers an account of the creation of the Nahal settlement enterprise which adds to the existing research on the subject in two ways. Firstly, it identifies some of the stages in the historical process that have not as yet been adequately described. Secondly, contrary to existing research which claims that the attempt to integrate lone soldiers within the Nahal settlement enterprise failed and had no long-term effects, we argue that the integration achieved most of its goals.