PressLite: A Max-Pressure-Rewarded Deep Reinforcement Learning Controller for Coordinated Arterial Traffic Signals

Abstract

Signalized intersections are the points where most urban delay accumulates, and the qual ity of the timing decisions taken there decides whether a road network behaves smoothly or grinds to a halt. The conventional answer has been fixed-time plans, sometimes augmented with actuation: useful, low-cost, but rigid in the face of demand that swings hour by hour. Re inforcement learning, on the other hand, lets an agent figure out timing strategies by acting in a simulated environment and watching what happens. Most published RL controllers, though, lean on rewards that the designer cooks up by hand—queue length, cumulative wait, sometimes a weighted blend of both. The trouble with such rewards is twofold: they offer no theoretical assurance that the underlying control loop will stay stable, and the choice of weights tends to be a matter of experimentation rather than principled design. This thesis develops and evaluates PressLite, a deep-RL controller that is designed to dis solve precisely that tension. The idea is to borrow the reward signal from a result in trans portation theory—the Max Pressure controller of Varaiya (2013), which is proved throughput optimal: as long as the demand fits inside the network’s capacity, balancing pressure between connected lanes keeps queues bounded. PressLite takes the negative of the per-intersection pressure and feeds it to a Deep Q-Network as the scalar the agent should maximise. In doing so, the controller picks up the adaptability that comes with deep RL and, at the same time, leans on the stability properties that Max Pressure brings with it. Each agent observes a com pact state—which phase is currently running, how many vehicles sit on the outgoing lanes, and a segment-by-segment count of vehicles approaching from upstream—enough to reason about both the immediate intersection and the downstream consequences of its action. For evaluation, the system was trained and tested inside the CityFlow simulator on a mix ture of synthetic flow patterns (light/heavy, flat/peak) and traffic recordings drawn from arterials in Jinan, State College, and four corridors in Manhattan. PressLite comes out ahead of every baseline tried—fixed-time, GreenWave, the greedy Max Pressure rule, and two earlier RL con trollers (LIT and GRL)—measured by average travel time. Two further observations are worth noting: the controller scales without modification to networks of up to twenty intersections and to a 3×3 grid topology, and even though each agent acts independently with only local obser vations, coordinated behaviour resembling a green wave emerges in the learned policies on its own.