A Dynamic Holding Approach to Stabilizing a Bus Line Based on the Q-Learning Algorithm with Multistage Look-Ahead

The unreliable service and the unstable operation of a high-frequency bus line are shown as bus bunching and the uneven distribution of headways along the bus line. Although many control strategies, such as the static and dynamic holding strategies, have been proposed to solve the above problems, many of them take on some oversimplified assumptions about the real bus line operation. So it is hard for them to continuously adapt to the evolving complex system. In view of this dynamic setting, we present an adaptive holding method that combines the classic approximate dynamic programming (ADP) with the multistage look-ahead mechanism. The holding time, the only control means used in this study, will be determined by estimating its impact on the operation stability of the bus line system in the remaining observation period. The multistage look-ahead mechanism introduced into the classic Q-learning algorithm of the ADP model makes it easy that the algorithm gets through its earlier unstable phase more quickly and easily. During the implementation of the new holding approach, the past experiences of holding operations can be cumulated effectively into an artificial neural network used to approximate the unavailable Q-factor. The use of a detailed simulation system in the new approach makes it possible to take into account most of the possible causes of instability. The numerical experiments show that the new holding approach can stabilize the system by producing evenly distributed headway and removing bus bunching thoroughly. Compared with the terminal station holding strategies, the new method brings a more reliable bus line with shorter waiting times for passengers.

A Short-Turning Strategy for the Management of Bus Bunching Considering Variable Spatial-Temporal Running Time

Bus bunching could seriously damage the stability of transit system. This resultant instability always causes a dissatisfying performance of transit system. Traditional bus bunching control methods (e.g., holding control strategy) add slack to schedules or adapt cruising speed. The control methods can alleviate bus bunching in theory, but it is difficult to apply to actual operation, especially in busy traffic. The short-turning strategy only deals with spatial concentration of demand in the existing literatures. We find that the short-turning strategy is also very effective in alleviating bus bunching. In this study, based on the passenger arrival rate of each stop and the spatial-temporal running time, a short-turning model with bunching penalty is developed, and the waiting time of passengers and the operation cost are also considered. Based on data from Beijing Transportation Information Center, we take the Yuntong 111 bus line of Beijing as an example. Compared with the currently used timetable, it is found that a 46.78% reduction in bus bunching is achieved by using the optimal timetable, and there is no increase in operating costs.

Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning

The bus system is a critical component of sustainable urban transportation. However, due to the significant uncertainties in passenger demand and traffic conditions, bus operation is unstable in nature and bus bunching has become a common phenomenon that undermines the reliability and efficiency of bus services. Despite recent advances in multi-agent reinforcement learning (MARL) on traffic control, little research has focused on bus fleet control due to the tricky asynchronous characteristic---control actions only happen when a bus arrives at a bus stop and thus agents do not act simultaneously. In this study, we formulate route-level bus fleet control as an asynchronous multi-agent reinforcement learning (ASMR) problem and extend the classical actor-critic architecture to handle the asynchronous issue. Specifically, we design a novel critic network to effectively approximate the marginal contribution for other agents, in which graph attention neural network is used to conduct inductive learning for policy evaluation. The critic structure also helps the ego agent optimize its policy more efficiently. We evaluate the proposed framework on real-world bus services and actual passenger demand derived from smart card data. Our results show that the proposed model outperforms both traditional headway-based control methods and existing MARL methods.

An Integrated Bus Holding and Speed Adjusting Strategy Considering Passenger’s Waiting Time Perceptions

To solve the problems of bus bunching and large gaps, this study combines bus holding and speed adjusting to alleviate them respectively considering the characteristics of passenger’s perceived waiting time. The difference between passenger’s perceived waiting time at stops and actual time is described quantitatively through the expected waiting time of passengers. Bus holding based on a threshold method is implemented at any stops for bunching buses, and speed adjusting based on a Markovian decision model is implemented at limited stops for lagging buses. Simulations based on real data of a bus route show that the integrated control strategy is able to improve the service reliability and to decrease passengers’ perceived waiting time at stops. Several insights have been uncovered through performance analysis: (1) The increase of holding control strength results in improvement of the headway regularity, and leads to a greater perceived waiting time though; (2) Compared to traveling freely, suitable speed guidance will not slow down the average cruising speed in the trip; (3) The scale of passenger demand and through passengers are the two key factors influencing whether a stop should be selected as a speed-adjusting control point.

Modeling bus bunching and anti-bunching control accounting for signal control and passenger swapping behavior

The conventional bus propagation process has two main shortcomings: one is bus bunching, the other is extra energy consumption by idling at signalized intersection and unexpected speed variation along the route. To overcome these problems simultaneously, an extended bus propagation model and an anti-bunching control are proposed. To extend the time-based bus propagation model, we employ finite state machine and intelligent driver model to establish a spatial-temporal based bus propagation model accounting for bus dynamic motion and passenger swapping behavior between bunching buses. To mitigate bus bunching and improve bus fuel economy in a cyber-traffic environment, an anticipated average speed planning is employed to improve headway regularity and reduce the chance of encountering red light, and then model predictive control accounting for state and control constraints is used to generate a smooth speed trajectory for connected bus to follow the commands given by anticipated average speed planning, which will in turn ensure that all the connected buses traverse signalized intersection and approach downstream stop in ecological driving behaviors. Numerical simulations show that the proposed model can imitate passenger swapping behavior when bus bunching occurs, and the anti-bunching control is suitable to mitigate bus bunching and guide connected bus to traverse signalized intersection and reach downstream stop with fewer delays.

Analysis and Simulation of Intervention Strategies against Bus Bunching by means of an Empirical Agent-Based Model

In this paper, we propose an empirically based Monte Carlo bus-network (EMB) model as a test bed to simulate intervention strategies to overcome the inefficiencies of bus bunching. The EMB model is an agent-based model which utilizes the positional and temporal data of the buses obtained from the Global Positioning System (GPS) to constitute (1) a set of empirical velocity distributions of the buses and (2) a set of exponential distributions of interarrival time of passengers at the bus stops. Monte Carlo sampling is then performed on these two derived probability distributions to yield the stochastic dynamics of both the buses’ motion and passengers’ arrival. Our EMB model is generic and can be applied to any real-world bus network system. In particular, we have validated the model against the Nanyang Technological University’s Shuttle Bus System by demonstrating its accuracy in capturing the bunching dynamics of the shuttle buses. Furthermore, we have analyzed the efficacy of three intervention strategies: holding, no-boarding, and centralized-pulsing, against bus bunching by incorporating the rule set of these strategies into the model. Under the scenario where the buses have the same velocity, we found that all three strategies improve both the waiting and travelling times of the commuters. However, when the buses have different velocities, only the centralized-pulsing scheme consistently outperforms the control scenario where the buses periodically bunch together.