scholarly journals Control-Stopping Games for Market Microstructure and Beyond

2020 ◽  
Vol 45 (4) ◽  
pp. 1289-1317
Author(s):  
Roman Gayduk ◽  
Sergey Nadtochiy

In this paper, we present a family of control-stopping games that arise naturally in equilibrium-based models of market microstructure as well as in other models with strategic buyers and sellers. A distinctive feature of this family of games is the fact that the agents do not have any exogenously given fundamental value for the asset, and they deduce the value of their position from the bid and ask prices posted by other agents (i.e., they are pure speculators). As a result, in such a game, the reward function of each agent at the time of stopping depends directly on the controls of other players. The equilibrium problem leads naturally to a system of coupled control-stopping problems (or, equivalently, reflected-backward stochastic differential equations), in which the individual reward functions (or reflecting barriers) depend on the value functions (or solution components) of other agents. The resulting system, in general, presents multiple mathematical challenges because of the nonstandard form of coupling (or reflection). In the present case, this system is also complicated by the fact that the continuous controls of the agents, describing their posted bid and ask prices, are constrained to take values in a discrete grid. The latter feature reflects the presence of a positive tick size in the market, and it creates additional discontinuities in the agents’ reward functions (or reflecting barriers). Herein we prove the existence of a solution to the associated system in a special Markovian framework, provide numerical examples, and discuss the potential applications.

Author(s):  
Zhen Yu ◽  
Yimin Feng ◽  
Lijun Liu

In general reinforcement learning tasks, the formulation of reward functions is a very important step in reinforcement learning. The reward function is not easy to formulate in a large number of systems. The network training effect is sensitive to the reward function, and different reward value functions will get different results. For a class of systems that meet specific conditions, the traditional reinforcement learning method is improved. A state quantity function is designed to replace the reward function, which is more efficient than the traditional reward function. At the same time, the predictive network link is designed so that the network can learn the value of the general state by using the special state. The overall structure of the network will be improved based on the Deep Deterministic Policy Gradient (DDPG) algorithm. Finally, the algorithm was successfully applied in the environment of FrozenLake, and achieved good performance. The experiment proves the effectiveness of the algorithm and realizes rewardless reinforcement learning in a class of systems.


2020 ◽  
Vol 13 (11) ◽  
pp. 250
Author(s):  
Iwao Maeda ◽  
David deGraw ◽  
Michiharu Kitano ◽  
Hiroyasu Matsushima ◽  
Kiyoshi Izumi ◽  
...  

While exchanges and regulators are able to observe and analyze the individual behavior of financial market participants through access to labeled data, this information is not accessible by other market participants nor by the general public. A key question, then, is whether it is possible to model individual market participants’ behaviors through observation of publicly available unlabeled market data alone. Several methods have been suggested in the literature using classification methods based on summary trading statistics, as well as using inverse reinforcement learning methods to infer the reward function underlying trader behavior. Our primary contribution is to propose an alternative neural network based multi-modal imitation learning model which performs latent segmentation of stock trading strategies. As a result that the segmentation in the latent space is optimized according to individual reward functions underlying the order submission behaviors across each segment, our results provide interpretable classifications and accurate predictions that outperform other methods in major classification indicators as verified on historical orderbook data from January 2018 to August 2019 obtained from the Tokyo Stock Exchange. By further analyzing the behavior of various trader segments, we confirmed that our proposed segments behaves in line with real-market investor sentiments.


2018 ◽  
Author(s):  
Karel Kleisner ◽  
Šimon Pokorný ◽  
Selahattin Adil Saribay

In present research, we took advantage of geometric morphometrics to propose a data-driven method for estimating the individual degree of facial typicality/distinctiveness for cross-cultural (and other cross-group) comparisons. Looking like a stranger in one’s home culture may be somewhat stressful. The same facial appearance, however, might become advantageous within an outgroup population. To address this fit between facial appearance and cultural setting, we propose a simple measure of distinctiveness/typicality based on position of an individual along the axis connecting the facial averages of two populations under comparison. The more distant a face is from its ingroup population mean towards the outgroup mean the more distinct it is (vis-à-vis the ingroup) and the more it resembles the outgroup standards. We compared this new measure with an alternative measure based on distance from outgroup mean. The new measure showed stronger association with rated facial distinctiveness than distance from outgroup mean. Subsequently, we manipulated facial stimuli to reflect different levels of ingroup-outgroup distinctiveness and tested them in one of the target cultures. Perceivers were able to successfully distinguish outgroup from ingroup faces in a two-alternative forced-choice task. There was also some evidence that this task was harder when the two faces were closer along the axis connecting the facial averages from the two cultures. Future directions and potential applications of our proposed approach are discussed.


2004 ◽  
Vol 12 (1) ◽  
pp. 99-135 ◽  
Author(s):  
Tim Kovacs

It has long been known that in some relatively simple reinforcement learning tasks traditional strength-based classifier systems will adapt poorly and show poor generalisation. In contrast, the more recent accuracy-based XCS, appears both to adapt and generalise well. In this work, we attribute the difference to what we call strong over general and fit over general rules. We begin by developing a taxonomy of rule types and considering the conditions under which they may occur. In order to do so an extreme simplification of the classifier system is made, which forces us toward qualitative rather than quantitative analysis. We begin with the basics, considering definitions for correct and incorrect actions, and then correct, incorrect, and overgeneral rules for both strength and accuracy-based fitness. The concept of strong overgeneral rules, which we claim are the Achilles' heel of strength-based classifier systems, are then analysed. It is shown that strong overgenerals depend on what we call biases in the reward function (or, in sequential tasks, the value function). We distinguish between strong and fit overgeneral rules, and show that although strong overgenerals are fit in a strength-based system called SB-XCS, they are not in XCS. Next we show how to design fit overgeneral rules for XCS (but not SB-XCS), by introducing biases in the variance of the reward function, and thus that each system has its own weakness. Finally, we give some consideration to the prevalence of reward and variance function bias, and note that non-trivial sequential tasks have highly biased value functions.


2021 ◽  
Vol 11 ◽  
Author(s):  
Rui Ding ◽  
Laipeng Luo ◽  
Ruixiang Han ◽  
Meiling Zhang ◽  
Tingting Li ◽  
...  

An efficient bioflocculant-producing strain, Raoultella ornithinolytica 160-1, was identified by 16S rRNA and mass spectrometry analyses. Rapid production of bioflocculant EPS-160 was obtained with 10.01 g/(L⋅d) after optimized by response surface methodology. With the aid of Al(III), more than 90% flocculation activity of EPS-160 at 8 mg/L dosage was achieved in 5 min. Thus, this novel Al(III) dependent bioflocculant was used in combined with chemical coagulants AlCl3 to remove kaolin suspensions and wastewater treatment. The results indicated that the addition of EPS-160 in aggregation system not only largely improved the flocculation ability than the individual use of chemical flocculant (over 30 percent), but also overcome the decrease of flocculation activity due to the overdose of AlCl3 and maintained the optimum dosage of AlCl3 in a wide range (11–23 mg/L). The zeta potentials and EPS-160 structure indicated that both charge neutralization and bridging were the flocculation mechanism with kaolin. During the wastewater treatment, this composite flocculants consisted of EPS-160 and AlCl3 also had great performance for turbidity elimination. Moreover, with the properties of high flocculation activity, hyperthermal stability, pH tolerance and non-toxicity, EPS-160 shows great potential applications.


Food Research ◽  
2019 ◽  
Vol 4 (2) ◽  
pp. 557-562
Author(s):  
M.D. Lieu ◽  
T.T.H. Hoang ◽  
H.N.T. Nguyen ◽  
T.K.T. Dang

Anthocyanin is a water-soluble color compound of the flavonoid which was successfully encapsulation in Saccharomyces cerevisiae by plasmolysis, ethanol, and ultrasound treatments using alone or in combination in the first time. Treatment agents significantly enhanced the encapsulation efficiency of anthocyanin fluid. The encapsulation yield (EY) of the combined factors was higher than the individual impact factors. Ethanol 10% (v/v) and ultrasound 180 seconds for the highest EY 40.22±0.67%, then ethanol 10% (v/v) and NaCl 10% (w/v) for EY 36.45±0.35%, NaCl 10% and ultrasound for EY 32.14±0.98% lowest. The color stability evaluation of the capsules was carried out at 80°C for 30 minutes. The color lost rate was determined by the spectrometer. The color loss of samples with the un-treatment yeast was 20.45±1.21%, higher than the treated sample. This suggests that anthocyanin encapsulation by yeast cell is efficient in overcoming the effects of high temperatures and having potential applications in food processing.


Author(s):  
Hao Ji ◽  
Yan Jin

Abstract Self-organizing systems (SOS) are developed to perform complex tasks in unforeseen situations with adaptability. Predefining rules for self-organizing agents can be challenging, especially in tasks with high complexity and changing environments. Our previous work has introduced a multiagent reinforcement learning (RL) model as a design approach to solving the rule generation problem of SOS. A deep multiagent RL algorithm was devised to train agents to acquire the task and self-organizing knowledge. However, the simulation was based on one specific task environment. Sensitivity of SOS to reward functions and systematic evaluation of SOS designed with multiagent RL remain an issue. In this paper, we introduced a rotation reward function to regulate agent behaviors during training and tested different weights of such reward on SOS performance in two case studies: box-pushing and T-shape assembly. Additionally, we proposed three metrics to evaluate the SOS: learning stability, quality of learned knowledge, and scalability. Results show that depending on the type of tasks; designers may choose appropriate weights of rotation reward to obtain the full potential of agents’ learning capability. Good learning stability and quality of knowledge can be achieved with an optimal range of team sizes. Scaling up to larger team sizes has better performance than scaling downwards.


Nanophotonics ◽  
2019 ◽  
Vol 8 (4) ◽  
pp. 517-547 ◽  
Author(s):  
Brian Szychowski ◽  
Matthew Pelton ◽  
Marie-Christine Daniel

AbstractThe assembly of inorganic nanoparticles often leads to collective properties that are different from the combined properties of the individual components. In particular, coupling plasmonic and excitonic nanoparticles has been shown to modify their optical properties, including absorption, emission, and scattering. Because of this, these coupled assemblies have potential applications in a wide range of areas, including sensing, light harvesting, and photocatalysis. More recently, unique properties, including Fano interference and Rabi splitting, have been observed by increasing the coupling strength. However, the behavior of coupled nanoparticles is highly dependent on the exact organization of the components, including the number of particles coupled, the distance separating them, and their spatial orientation. This is especially true in the case of strongly coupled particles. Because of this, it is important to achieve synthetic techniques that not only can link particles together but also offer good control over how the particles are connected. In this review, assemblies of plasmonic and excitonic nanoparticles are reviewed, including the various methods that have been used for their construction, the properties that these systems have been predicted to possess as well as the ones that have been observed, and their current applications along with current challenges in the field and potential future applications.


2009 ◽  
Vol 46 (04) ◽  
pp. 1130-1145 ◽  
Author(s):  
G. Deligiannidis ◽  
H. Le ◽  
S. Utev

In this paper we present an explicit solution to the infinite-horizon optimal stopping problem for processes with stationary independent increments, where reward functions admit a certain representation in terms of the process at a random time. It is shown that it is optimal to stop at the first time the process crosses a level defined as the root of an equation obtained from the representation of the reward function. We obtain an explicit formula for the value function in terms of the infimum and supremum of the process, by making use of the Wiener–Hopf factorization. The main results are applied to several problems considered in the literature, to give a unified approach, and to new optimization problems from the finance industry.


Author(s):  
Enrique Del Percio

In 1976, a terrible dictatorship was established in Argentina, even before Foucault claimed with crystal clarity that the fundamental difference between classical liberalism and neoliberalism was the substitution of the homo economicus −related to the exchange− by the homo economicus as entrepreneur of himself (lecture delivered on 14 March 1979); and also before Margaret Thatcher (in Ronald Butt’s interview, Sunday Times, 3 May 1981) confirmed Foucault´s analysis stating that: “Economics are the method; the object is to change the heart and soul”. In the same year, Milton Friedman received the Nobel Prize in Economics. The explicit purpose of the Military Junta was to promote a profound cultural transformation, based on the premise that the causes of the alleged “underdevelopment” were not so much economical but cultural and political. Nevertheless, as García Delgado and Molina (2006) pointed out, the problem is not related to a sort of inevitable structural poverty, due to the culture of our people. It is a matter of a decline in society, produced by the policy orientation of the dictatorship. Until then, the income distribution was similar to that of the countries from the Southern Europe with an almost frictional unemployment. Until the coup d’état, Argentina had a poverty rate of 8% and the best distributive structure of income in Latin America. However, 1976 was a turning point; the surge of the neoliberal model promoted a process of over-indebtedness, wealth concentration, unrestricted opening of markets with an unfavourable exchange rate for national industry, labour flexibilization, with the insertion in a competitive globalization of “savage capitalism” that “strengthened the asymmetries and transfers of resources from the periphery to the centre. This concept differs from thinking about inequality as a problem related to culture, corruption and poor institutional quality” (García Delgado, 2006).Despite the overwhelming adverse evidence, it is still a commonplace to blame all the ills of our society on that culture, the maximum expression of which would be Peronism. In fact, the great majority of disappeared people during the dictatorship were Peronist political, trade union and social leaders. The motto of the Ministry of Economics during the dictatorship was “towards a change of mentality”. The current Argentine situation, in terms of advances of neoliberalism as well as resistances to it, cannot be understood without referring to the dictatorship. In Poratti words, “the coup d’état of 1976 does not only put an end to a government, a political system and project, but also to a 'world' in which Argentinians were living at least from the independence project of 1810. In those days, there was not an abrupt differentiation between generations and, in many aspects, people could identify themselves, diachronically, with a historical line beyond the particular generational characteristics” (Various Authors, 2009).These aspects go along with others that appeared in other areas, such as the implementation of new computer and communication technologies and, as a consequence, individual and social fragmentation. The impact of these technologies on daily life was decisive to the emergence of what some authors, like Sloterdijk (2002), called “mass individualism.” No doubt, this is a necessary aspect to explain the rise of the neoliberal subjectivity in developed countries. Yet, in Argentina, the existence of political, social, trade-union and ecclesiastical movements based on popular roots, with solidarity as a fundamental value, hampered the conquest of the “heart and soul” in 1976; and they are still now an obstacle to be overcome by sectors interested in imposing a neoliberal model. It is impossible to explain any isolated phenomenon of popular resistance to the hegemonic attempts from neoliberalism without analysing the common conceptions and understandings found in Argentina. Indeed, the popular culture substrate in Argentina is made up, mainly, by the confluence of different cultures: Andean, Guaraní Indians, Afro and Criollo (native). All of them are characterized by their relational and solidarity conceptions, intrinsically opposed to a subjectivity that conceives the individual as an entrepreneur of himself/herself.


Sign in / Sign up

Export Citation Format

Share Document