Łukasz Gorczyca

List of main publications and preprints

1. arXiv

   URB - Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles

   Akman, Ahmet Onur, Psarou, Anastasia, Hoffmann, Michał, Gorczyca, Łukasz, Kowalski, Łukasz, Gora, Paweł, Jamróz, Grzegorz, and Kucharski, Rafał

   arXiv preprint arXiv:2505.17734 2025

   Abs HTML

   Connected Autonomous Vehicles (CAVs) promise to reduce congestion in future urban networks, potentially by optimizing their routing decisions. Unlike for human drivers, these decisions can be made with collective, data-driven policies, developed by machine learning algorithms. Reinforcement learning (RL) can facilitate the development of such collective routing strategies, yet standardized and realistic benchmarks are missing. To that end, we present \our: Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles. \our is a comprehensive benchmarking environment that unifies evaluation across 29 real-world traffic networks paired with realistic demand patterns. \our comes with a catalog of predefined tasks, four state-of-the-art multi-agent RL (MARL) algorithm implementations, three baseline methods, domain-specific performance metrics, and a modular configuration scheme. Our results suggest that, despite the lengthy and costly training, state-of-the-art MARL algorithms rarely outperformed humans. Experimental results reported in this paper initiate the first leaderboard for MARL in large-scale urban routing optimization and reveal that current approaches struggle to scale, emphasizing the urgent need for advancements in this domain.

2. SoftwareX

   RouteRL: Multi-agent reinforcement learning framework for urban route choice with autonomous vehicles

   Akman, Ahmet Onur, Psarou, Anastasia, Gorczyca, Łukasz, Varga, Zoltán György, Jamróz, Grzegorz, and Kucharski, Rafał

   SoftwareX 2025

   Abs HTML

   RouteRL is a novel framework that integrates multi-agent reinforcement learning (MARL) with a microscopic traffic simulation for the development of efficient collective route choice strategies for autonomous vehicles (AVs). The proposed framework models the daily urban route choices of driver agents of two types: human drivers, emulated using behavioral route choice models, and AVs, modeled as MARL agents optimizing their policies for a predefined objective. RouteRL aims to advance research in MARL, transport modeling, and human–AI interaction for transportation applications. This study presents a technical report on RouteRL, outlines its potential research contributions, and showcases its impact via illustrative examples. Initial findings show that human travel times could increase following the introduction of AVs in urban environments, highlighting the importance of future studies to support efficient urban mobility.

3. arXiv

   Autonomous vehicles need social awareness to find optima in multi-agent reinforcement learning routing games

   Psarou, Anastasia, Gorczyca, Łukasz, Gaweł, Dominik, and Kucharski, Rafał

   arXiv preprint arXiv:2510.11410 2025

   Abs HTML

   Previous work has shown that when multiple selfish Autonomous Vehicles (AVs) are introduced to future cities and start learning optimal routing strategies using Multi-Agent Reinforcement Learning (MARL), they may destabilize traffic systems, as they would require a significant amount of time to converge to the optimal solution, equivalent to years of real-world commuting. We demonstrate that moving beyond the selfish component in the reward significantly relieves this issue. If each AV, apart from minimizing its own travel time, aims to reduce its impact on the system, this will be beneficial not only for the system-wide performance but also for each individual player in this routing game. By introducing an intrinsic reward signal based on the marginal cost matrix, we significantly reduce training time and achieve convergence more reliably. Marginal cost quantifies the impact of each individual action (route-choice) on the system (total travel time). Including it as one of the components of the reward can reduce the degree of non-stationarity by aligning agents’ objectives. Notably, the proposed counterfactual formulation preserves the system’s equilibria and avoids oscillations. Our experiments show that training MARL algorithms with our novel reward formulation enables the agents to converge to the optimal solution, whereas the baseline algorithms fail to do so. We show these effects in both a toy network and the real-world network of Saint-Arnoult. Our results optimistically indicate that social awareness (i.e., including marginal costs in routing decisions) improves both the system-wide and individual performance of future urban systems with AVs.

4. arXiv

   Autonomous Vehicles Using Multi-Agent Reinforcement Learning for Routing Decisions Can Harm Urban Traffic

   Psarou, Anastasia, Akman, Ahmet Onur, Gorczyca, Łukasz, Hoffmann, Michał, Varga, Zoltán György, Jamróz, Grzegorz, and Kucharski, Rafał

   arXiv preprint arXiv:2502.13188 2025

   Abs HTML

   Autonomous vehicles (AVs) using Multi-Agent Reinforcement Learning (MARL) for simultaneous route optimization may destabilize traffic environments, with human drivers possibly experiencing longer travel times. We study this interaction by simulating human drivers and AVs. Our experiments with standard MARL algorithms reveal that, even in trivial cases, policies often fail to converge to an optimal solution or require long training periods. The problem is amplified by the fact that we cannot rely entirely on simulated training, as there are no accurate models of human routing behavior. At the same time, real-world training in cities risks destabilizing urban traffic systems, increasing externalities, such as CO2 emissions, and introducing non-stationarity as human drivers adapt unpredictably to AV behaviors. Centralization can improve convergence in some cases, however, it raises privacy concerns for the travelers’ destination data. In this position paper, we argue that future research must prioritize realistic benchmarks, cautious deployment strategies, and tools for monitoring and regulating AV routing behaviors to ensure sustainable and equitable urban mobility systems.