Anastasia Psarou

List of main publications and preprints

1. arXiv

   URB - Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles

   Akman, Ahmet Onur, Psarou, Anastasia, Hoffmann, Michał, Gorczyca, Łukasz, Kowalski, Łukasz, Gora, Paweł, Jamróz, Grzegorz, and Kucharski, Rafał

   arXiv preprint arXiv:2505.17734 2025

   Abs HTML

   Connected Autonomous Vehicles (CAVs) promise to reduce congestion in future urban networks, potentially by optimizing their routing decisions. Unlike for human drivers, these decisions can be made with collective, data-driven policies, developed by machine learning algorithms. Reinforcement learning (RL) can facilitate the development of such collective routing strategies, yet standardized and realistic benchmarks are missing. To that end, we present \our: Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles. \our is a comprehensive benchmarking environment that unifies evaluation across 29 real-world traffic networks paired with realistic demand patterns. \our comes with a catalog of predefined tasks, four state-of-the-art multi-agent RL (MARL) algorithm implementations, three baseline methods, domain-specific performance metrics, and a modular configuration scheme. Our results suggest that, despite the lengthy and costly training, state-of-the-art MARL algorithms rarely outperformed humans. Experimental results reported in this paper initiate the first leaderboard for MARL in large-scale urban routing optimization and reveal that current approaches struggle to scale, emphasizing the urgent need for advancements in this domain.

2. SoftwareX

   RouteRL: Multi-agent reinforcement learning framework for urban route choice with autonomous vehicles

   Akman, Ahmet Onur, Psarou, Anastasia, Gorczyca, Łukasz, Varga, Zoltán György, Jamróz, Grzegorz, and Kucharski, Rafał

   SoftwareX 2025

   Abs HTML

   RouteRL is a novel framework that integrates multi-agent reinforcement learning (MARL) with a microscopic traffic simulation for the development of efficient collective route choice strategies for autonomous vehicles (AVs). The proposed framework models the daily urban route choices of driver agents of two types: human drivers, emulated using behavioral route choice models, and AVs, modeled as MARL agents optimizing their policies for a predefined objective. RouteRL aims to advance research in MARL, transport modeling, and human–AI interaction for transportation applications. This study presents a technical report on RouteRL, outlines its potential research contributions, and showcases its impact via illustrative examples. Initial findings show that human travel times could increase following the introduction of AVs in urban environments, highlighting the importance of future studies to support efficient urban mobility.

3. Sci. Rep.

   Social implications of coexistence of CAVs and human drivers in the context of route choice

   Jamróz, Grzegorz, Akman, Ahmet Onur, Psarou, Anastasia, Varga, Zoltán György, and Kucharski, Rafał

   Scientific Reports 2025

   Abs HTML

   Suppose in a stable urban traffic system populated only by human driven vehicles (HDVs), a given proportion (e.g. 10 %) is replaced by a fleet of Connected and Autonomous Vehicles (CAVs), which share information and pursue a collective goal. Suppose these vehicles are centrally coordinated and differ from HDVs only by their collective capacities allowing them to make more efficient routing decisions before the travel on a given day begins. Suppose there is a choice between two routes and every day each driver makes a decision which route to take. Human drivers maximize their utility. CAVs might optimize different goals, such as the total travel time of the fleet. We show that in this plausible futuristic setting, the strategy CAVs are allowed to adopt may result in human drivers either benefitting or being systematically disadvantaged and urban networks becoming more or less optimal. Consequently, some regulatory measures might become indispensable.

4. arXiv

   Impact of collective behaviors of autonomous vehicles on urban traffic dynamics: A multi-agent reinforcement learning approach

   Akman, Ahmet Onur, Psarou, Anastasia, Varga, Zoltán György, Jamróz, Grzegorz, and Kucharski, Rafał

   arXiv preprint arXiv:2509.22216 2025

   Abs HTML

   This study examines the potential impact of reinforcement learning (RL)-enabled autonomous vehicles (AV) on urban traffic flow in a mixed traffic environment. We focus on a simplified day-to-day route choice problem in a multi-agent setting. We consider a city network where human drivers travel through their chosen routes to reach their destinations in minimum travel time. Then, we convert one-third of the population into AVs, which are RL agents employing Deep Q-learning algorithm. We define a set of optimization targets, or as we call them behaviors, namely selfish, collaborative, competitive, social, altruistic, and malicious. We impose a selected behavior on AVs through their rewards. We run our simulations using our in-house developed RL framework PARCOUR. Our simulations reveal that AVs optimize their travel times by up to 5%, with varying impacts on human drivers’ travel times depending on the AV behavior. In all cases where AVs adopt a self-serving behavior, they achieve shorter travel times than human drivers. Our findings highlight the complexity differences in learning tasks of each target behavior. We demonstrate that the multi-agent RL setting is applicable for collective routing on traffic networks, though their impact on coexisting parties greatly varies with the behaviors adopted.

5. arXiv

   Autonomous vehicles need social awareness to find optima in multi-agent reinforcement learning routing games

   Psarou, Anastasia, Gorczyca, Łukasz, Gaweł, Dominik, and Kucharski, Rafał

   arXiv preprint arXiv:2510.11410 2025

   Abs HTML

   Previous work has shown that when multiple selfish Autonomous Vehicles (AVs) are introduced to future cities and start learning optimal routing strategies using Multi-Agent Reinforcement Learning (MARL), they may destabilize traffic systems, as they would require a significant amount of time to converge to the optimal solution, equivalent to years of real-world commuting. We demonstrate that moving beyond the selfish component in the reward significantly relieves this issue. If each AV, apart from minimizing its own travel time, aims to reduce its impact on the system, this will be beneficial not only for the system-wide performance but also for each individual player in this routing game. By introducing an intrinsic reward signal based on the marginal cost matrix, we significantly reduce training time and achieve convergence more reliably. Marginal cost quantifies the impact of each individual action (route-choice) on the system (total travel time). Including it as one of the components of the reward can reduce the degree of non-stationarity by aligning agents’ objectives. Notably, the proposed counterfactual formulation preserves the system’s equilibria and avoids oscillations. Our experiments show that training MARL algorithms with our novel reward formulation enables the agents to converge to the optimal solution, whereas the baseline algorithms fail to do so. We show these effects in both a toy network and the real-world network of Saint-Arnoult. Our results optimistically indicate that social awareness (i.e., including marginal costs in routing decisions) improves both the system-wide and individual performance of future urban systems with AVs.

6. arXiv

   Equilibria in routing games with connected autonomous vehicles will not be strong, as exclusive clubs may form

   Kucharski, Rafał, Psarou, Anastasia, and Descormier, Natello

   arXiv preprint arXiv:2510.12862 2025

   Abs HTML

   User Equilibrium is the standard representation of the so-called routing game in which drivers adjust their route choices to arrive at their destinations as fast as possible. Asking whether this Equilibrium is strong or not was meaningless for human drivers who did not form coalitions due to technical and behavioral constraints. This is no longer the case for connected autonomous vehicles (CAVs), which will be able to communicate and collaborate to jointly form routing coalitions. We demonstrate this for the first time on a carefully designed toy-network example, where a ‘club‘ of three autonomous vehicles jointly decides to deviate from the user equilibrium and benefit (arrive faster). The formation of such a club has negative consequences for other users, who are not invited to join it and now travel longer, and for the system, making it suboptimal and disequilibrated, which triggers adaptation dynamics. This discovery has profound implications for the future of our cities. We demonstrate that, if not prevented, CAV operators may intentionally disequilibrate traffic systems from their classic Nash equilibria, benefiting their own users and imposing costs on others. These findings suggest the possible emergence of an exclusive CAV elite, from which human-driven vehicles and non-coalition members may be excluded, potentially leading to systematically longer travel times for those outside the coalition, which would be harmful for the equity of public road networks.

7. arXiv

   Collaboration Between the City and Machine Learning Community is Crucial to Efficient Autonomous Vehicles Routing

   Psarou, Anastasia, Akman, Ahmet Onur, Gorczyca, Łukasz, Hoffmann, Michał, Jamróz, Grzegorz, and Kucharski, Rafał

   arXiv preprint arXiv:2502.13188 2025

   Abs HTML

   Autonomous vehicles (AVs), possibly using Multi-Agent Reinforcement Learning (MARL) for simultaneous route optimization, may destabilize traffic networks, with human drivers potentially experiencing longer travel times. We study this interaction by simulating human drivers and AVs. Our experiments with standard MARL algorithms reveal that, both in simplified and complex networks, policies often fail to converge to an optimal solution or require long training periods. This problem is amplified by the fact that we cannot rely entirely on simulated training, as there are no accurate models of human routing behavior. At the same time, real-world training in cities risks destabilizing urban traffic systems, increasing externalities, such as CO2 emissions, and introducing non-stationarity as human drivers will adapt unpredictably to AV behaviors. In this position paper, we argue that city authorities must collaborate with the ML community to monitor and critically evaluate the routing algorithms proposed by car companies toward fair and system-efficient routing algorithms and regulatory standards.