Development of a Reinforcement Learning based Agent Model and People Flow Data to Mega Metropolitan Area

This paper proposes a new multi-objective bi-level programming model for the ring road bus lines and fare design problems. The proposed model consists of two layers: the traffic management operator and travelers. In the upper level, we propose a multi-objective bus lines and fares optimization model in which the operator’s profit and travelers’ utility are set as objective functions. In the lower level, evolutionary multi agent model of travelers’ bounded rational reinforcement learning with social interaction is introduced. A solution algorithm for the multi-objective bi-level programming is developed on the basis of the equalization algorithm of OD matrix. A numerical example based on a real case was conducted to verify the proposed models and solution algorithm. The computational results indicated that travel choice models with different degrees of rationality significantly changed the optimization results of bus lines and the differentiated fares; furthermore, the multi-objective bi-level programming in this paper can generate the solution to reduce the maximum section flow, increase the profit, and reduce travelers’ generalized travel cost.

Download Full-text

Robust Market Making via Adversarial Reinforcement Learning

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/633 ◽

2020 ◽

Cited By ~ 1

Author(s):

Thomas Spooner ◽

Rahul Savani

Keyword(s):

Reinforcement Learning ◽

Nash Equilibria ◽

Single Agent ◽

Market Maker ◽

Test Environment ◽

Agent Model ◽

Market Participants ◽

Domain Specific ◽

Special Cases ◽

Zero Sum

We show that adversarial reinforcement learning (ARL) can be used to produce market marking agents that are robust to adversarial and adaptively-chosen market conditions. To apply ARL, we turn the well-studied single-agent model of Avellaneda and Stoikov [2008] into a discrete-time zero-sum game between a market maker and adversary. The adversary acts as a proxy for other market participants that would like to profit at the market maker's expense. We empirically compare two conventional single-agent RL agents with ARL, and show that our ARL approach leads to: 1) the emergence of risk-averse behaviour without constraints or domain-specific penalties; 2) significant improvements in performance across a set of standard metrics, evaluated with or without an adversary in the test environment, and; 3) improved robustness to model uncertainty. We empirically demonstrate that our ARL method consistently converges, and we prove for several special cases that the profiles that we converge to correspond to Nash equilibria in a simplified single-stage game.

Download Full-text

A Biologically-Inspired Cognitive Agent Model Integrating Declarative Knowledge and Reinforcement Learning

2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology ◽

10.1109/wi-iat.2010.210 ◽

2010 ◽

Cited By ~ 2

Author(s):

Ah-Hwee Tan ◽

Gee-Wah Ng

Keyword(s):

Reinforcement Learning ◽

Declarative Knowledge ◽

Cognitive Agent ◽

Biologically Inspired ◽

Agent Model

Download Full-text

Short-Form Application of the Screening Test of Spanish Grammar and the Northwestern Syntax Screening Test to Latino Children

Language Speech and Hearing Services in Schools ◽

10.1044/0161-1461.0904.220 ◽

1978 ◽

Vol 9 (4) ◽

pp. 220-235

Author(s):

David L. Ratusnik ◽

Carol Melnick Ratusnik ◽

Karen Sattinger

Keyword(s):

New York ◽

Case Studies ◽

Metropolitan Area ◽

Screening Test ◽

Normative Data ◽

Short Form ◽

Latino Children ◽

Data Application ◽

York Metropolitan Area ◽

Multiple Regression Technique

Short-form versions of the Screening Test of Spanish Grammar (Toronto, 1973) and the Northwestern Syntax Screening Test (Lee, 1971) were devised for use with bilingual Latino children while preserving the original normative data. Application of a multiple regression technique to data collected on 60 lower social status Latino children (four years and six months to seven years and one month) from Spanish Harlem and Yonkers, New York, yielded a small but powerful set of predictor items from the Spanish and English tests. Clinicians may make rapid and accurate predictions of STSG or NSST total screening scores from administration of substantially shortened versions of the instruments. Case studies of Latino children from Chicago and Miami serve to cross-validate the procedure outside the New York metropolitan area.

Download Full-text

Emergency Department Admission and Mortality Rate for Suicidal Behavior

Crisis ◽

10.1027/0227-5910/a000282 ◽

2014 ◽

Vol 35 (6) ◽

pp. 406-414 ◽

Cited By ~ 10

Author(s):

Raimondo Maria Pavarin ◽

Angelo Fioritti ◽

Francesca Fontana ◽

Silvia Marani ◽

Alessandra Paparelli ◽

...

Keyword(s):

Emergency Department ◽

Metropolitan Area ◽

Suicidal Behavior ◽

Mortality Risk ◽

Illicit Drugs ◽

Attempted Suicide ◽

Cox Model ◽

Emergency Department Admission ◽

Completed Suicide

Background: The international literature reports that for every completed suicide there are between 8 and 22 visits to an Emergency Department (ED) for attempted suicide/suicidal behavior. Aims: To describe the characteristics of admission to emergency departments (EDs) for suicide-related presenting complaints in the metropolitan area of Bologna; to estimate the risk for all-cause mortality and for suicide; to identify the profiles of subjects most at risk. Method: Follow-up of patients admitted to the EDs of the metropolitan area of Bologna between January 2004 and December 2010 for attempted suicide. A Cox model was used to evaluate the association between sociodemographic variables and the general mortality risk. Results: We identified 505 cases of attempted suicide, which were more frequent for female subjects, over the weekend, and at night (8:00 p.m./8:00 a.m.). The most used suicide methods were psychotropic drugs, sharp or blunt objects, and jumping from high places. In this cohort, 3.6% of subjects completed suicide (4.5% of males vs. 2.9% of females), 2.3% within 1 year of the start of follow-up. The most common causes of death were drug use and hanging. In the multivariate analysis, those who used illicit drugs 24 hr prior to admission to the ED (hazard ratio [HR] = 3.46, 95% CI = 1.23–9.73) and patients who refused the treatment (HR = 6.74, 95% CI = 1.86–24.40) showed an increased mortality risk for suicide. Conclusion: Deliberate self-harm patients presenting to the ED who refuse treatment represent a specific target group for setting up dedicated prevention schemes.

Download Full-text