CoMAL: Collaborative Multi-Agent Large Language Models for

Mixed-Autonomy Traffic
Huaiyuan Yao1 , Longchao Da1 , Vishnu Nandam1 , Justin Turnau1 ,
Zhiwei Liu2 , Linsey Pang2 , Hua Wei1
1
Arizona State University, 2 Salesforce
arXiv:2410.14368v1 [cs.AI] 18 Oct 2024

Abstract Traffic dynamics are extremely complex and chaotic

The integration of autonomous vehicles into urban traffic has dynamical systems [8]. Pioneering researchers [9, 10]
great potential to improve efficiency by reducing congestion typically trade away the complexity of the model for de-
and optimizing traffic flow systematically. In this paper, we sirable provable properties. To this end, they propose
introduce CoMAL (Collaborative Multi-Agent LLMs), a a series of hand-designed control rules [11, 12] to direct
framework designed to address the mixed-autonomy traffic a fleet of vehicles to form a desired stable motion pat-
problem by collaboration among autonomous vehicles to op- tern. The experiments [13, 14] suggest that autonomous
timize traffic flow. CoMAL is built upon large language vehicles can enhance traffic throughput, which high-
models, operating in an interactive traffic simulation envi- lights the potential of mixed-autonomy systems. To
ronment. It utilizes a Perception Module to observe sur- model the complex interactions between autonomous
rounding agents and a Memory Module to store strategies and human-driven vehicles in mixed-autonomy traffic
for each agent. The overall workflow includes a Collabo- systems, simulation-based numerical methods like the
ration Module that encourages autonomous vehicles to dis- Flow benchmark [7, 15] have proven to be effective tools.
cuss the effective strategy and allocate roles, a reasoning Flow, a deep reinforcement learning (RL) framework,
engine to determine optimal behaviors based on assigned enables the systematic design of RL tasks to improve
roles, and an Execution Module that controls vehicle ac- overall traffic flow, including the control of autonomous
tions using a hybrid approach combining rule-based mod- vehicles and traffic signals. With its ability to simu-
els. Experimental results demonstrate that CoMAL achieves late dynamic multi-agent environments, Flow has fa-
superior performance on the Flow benchmark. Addition- cilitated the rise of multi-agent RL, which has gained
ally, we evaluate the impact of different language models popularity for modeling emergent behaviors in mixed-
and compare our framework with reinforcement learning ap- autonomy traffic systems [16]. By allowing autonomous
proaches. It highlights the strong cooperative capability vehicles to learn cooperative strategies, multi-agent RL
of LLM agents and presents a promising solution to the enables interaction with both human-driven and other
mixed-autonomy traffic challenge. The code is available at autonomous vehicles, optimizing traffic flow. These RL-
https://github.com/Hyan-Yao/CoMAL based models demonstrate strong performance in spe-
cific traffic scenarios, effectively learning control policies
1 Introduction for coordinating mixed-autonomy traffic [17, 18]. How-
ever, human behaviors in real-world settings are highly
Recently, there has been significant growth in end-to-
varied and unpredictable, while road networks are more
end autonomous driving systems [1]. The integration
complex and diverse. Deep RL performs well in specific
of large language models (LLMs) [2, 3] enhances the
and controlled environments but struggles to generalize
ability to generalize to unseen traffic scenarios with
across different scenarios [19] due to its reliance on large
embedded common-sense knowledge. These models [4,
amounts of specific training data. Meanwhile, it is dif-
5, 6] primarily focus on optimizing the performance of
ficult to understand the decision-making process of RL,
individual ego vehicles. However, it also matters to
which limits its interpretability.
study the problem of mixed-autonomy traffic to better
Rethinking human behavioral patterns, adolescents
deploy autonomous vehicles in society [7]. In this article,
can learn to drive in just 20 hours and handle unfa-
we enable connected autonomous vehicles (CAVs) to
miliar situations [20], while young children can sponta-
collaborate with human-driven vehicles across the traffic
neously cooperate to enhance work efficiency [21]. Hu-
network, aiming to optimize overall traffic flow and
man decision-making and cooperation are inherently
system-wide efficiency.
 Environment Multiple Agents on CAVs

CAV-3 .....
CAV CAV-2
CAV-4
.
Human Shared Message Pool Ring
CAV-1 CAV-n

Obs Actions
Scenario Task Planner Collaboration Module
Description Allocation Generation

Shared Messages
Figure Eight
LLM Agent Reason Engine
System prompt

Few-shot
Memory Module
Experiences
Execution Module
Prompt Generator
Merge
Updates
(a) Single-Agent Pipeline (b) Multi-Agent Workflow (c) Scenarios

Figure 1: The overall framework of CoMAL. (a) Single-Agent Pipeline: The prompt generator integrates scenario
descriptions, few-shot experiences, and shared messages, which are then fed into the LLM. The LLM subsequently
allocates tasks and generates planners. (b) Multi-Agent Workflow comprises three modules: the Collaboration
Module, the Reason Engine, and the Execution Module. (c) The three Benchmarks Scenarios for CoMAL Ring:
The ring road network consists of a closed-loop road where vehicles continuously travel in a circular fashion.
Figure Eight (FE): is an extension of the ring road, consisting of two circular loops connected by an intersection.
Merge: The merged network simulates how vehicles entering from an on-ramp cause disturbances.

knowledge-driven, relying on common sense, verbal the traffic system. Leveraging the stored experiences
communication, and reasoning [22]. This is in con- within the Memory Module, the agents participate in
trast to data-driven methods that require vast amounts a brainstorming session in the Collaboration Module,
of training data and often struggle with generaliza- where they allocate tasks and establish their specific
tion. Recent advancements in LLMs [23] offer promising roles. Subsequently, each agent develops driving plans
knowledge-driven solutions for addressing the mixed- according to its designated role through the Reason En-
autonomy traffic problem. LLMs can emulate human- gine. Then Reason Engine generates a rule-based driv-
like knowledge, including the ability to make decisions, ing planner grounded in the Intelligent Driver Model
form agreements, analyze road situations, and collabo- (IDM), which is then implemented in the Execution
rate in real-time. By leveraging this embodied human Module to compensate for the inherent limitations of
knowledge, multiple intelligent LLM agents with dis- LLMs in control performance.
tinct roles and attributes can work together to handle Our primary contributions are as follows:
complex tasks more efficiently [24], offering new avenues
for improving traffic systems through cooperation and 1) To the best of our knowledge, we are the first to in-
common-sense reasoning. tegrate the collaborative capability of multi-agent
Building upon these insights and recognizing the language models in autonomous driving. The Col-
limitations of RL in generalizing to new traffic situa- laboration Module effectively establishes a session
tions, we propose leveraging LLMs for their common- for multiple agents to engage in brainstorming and
sense reasoning and adaptability. LLMs offer better task allocation.
generalization by drawing on embedded knowledge, al-
lowing them to manage complex and unpredictable traf- 2) We propose CoMAL, a multi-agent framework de-
fic dynamics more effectively. We developed an innova- signed to address mixed-autonomy traffic chal-
tive framework named CoMAL specifically designed to lenges. CoMAL integrates rule-based planners to
address mixed-autonomy traffic challenges, as depicted enhance control mechanisms while leveraging the
in Figure 1. CoMAL comprises a simulation environ- cooperative and reasoning ability of LLM.
ment that enables the interaction of individual LLM
3) We evaluate CoMAL on Flow benchmark in three
agents and allows agents to collect perceptual data from
classical traffic scenarios (Figure 1 (c)) and com-
 pare its performance against RL methods. Exten- multi-agent systems. In traffic control, LLMs facili-
sive experimental results demonstrate significant tate human-machine interaction and improve decision-
performance improvements in terms of average ve- making processes. The survey [23] discusses how LLMs
locity and driving smoothness across various LLMs. can be integrated into multi-agent systems to enhance
communication and coordination among agents in traf-
4) We conduct experiments using the latest LLM fic environments. And the study DiLu [22] explores
models, including GPT-4o-mini and Qwen- a knowledge-driven approach using LLMs, while LLM-
72B/32B/7B, showcasing the adaptability of Light [35] utilizes LLMs as decision-making agents for
CoMAL across a diverse range of LLMs and traffic signal control. A pioneering model Open-TI [36]
demonstrating its superior performance over that integrates LLMs with external traffic analysis tools
state-of-the-art RL approaches in mixed-autonomy to perform comprehensive traffic simulations and task-
traffic scenarios. specific operations. Their findings suggest that incorpo-
rating domain-specific knowledge into LLMs can signifi-
2 Related Work cantly enhance the performance and reliability of multi-
agent systems in complex scenarios. This involves not
2.1 Mixed-Autonomy Traffic
only improving the performance of individual agents but
Mixed-autonomy traffic, where connected autonomous
also fostering effective communication and coordination
vehicles (CAVs) along with human-driven vehicles exist
among the agents.
in a system [7], presents a significant challenge in
traffic dynamics modeling and control. A control
3 Methodology
strategy named the “slow-in, fast-out” approach [13]
has demonstrated improvements in traffic throughput We introduce CoMAL, a framework designed for LLM
with a minimal percentage of autonomous vehicles. agents integrated into connected autonomous vehicles
Reinforcement learning (RL) offers a more dynamic (CAVs) to collaborate and enhance the overall velocity
and adaptable solution. Benchmarks in RL [25] like and driving smoothness of traffic flow. As illustrated
Mujoco and the Arcade Learning Environment [26] in Figure 1, we delineate CoMAL at two distinct levels:
provide systematic evaluation and comparison of algo- the single-agent pipeline and the multi-agent workflow.
rithms. And especially for Mixed-Autonomy, bench- At the single-agent level, the LLM-based agents
mark Flow [7] proposes four traffic scenarios to il- make decisions based on prompts that include few-shot
lustrate distinct RL problems including shockwave experiences stored in memory, as well as scenario de-
minimization, inflow management, efficient merging, scriptions derived from environmental perception. The
and intersection control. It evaluates and compares agents operate in two modes: task allocation within the
RL algorithms like Trust Region Policy Optimiza- Collaboration Module and planner generation within
tion (TRPO) [15, 27], Proximal Policy Optimization the Reason Engine. The multi-agent workflow consists
(PPO) [28], Evolutionary Strategies (ES) [29], and Aug- of three modules: the Collaboration Module, the Rea-
mented Random Search (ARS) [30] in traffic scenar- son Engine, and the Execution Module. In the Col-
ios. [31] also explores the mixed-autonomy scenario in a laboration Module, CoMAL establishes a shared mes-
multi-agent traffic signal control system [32]. sage pool that facilitates brainstorming and collabora-
tive decision-making among agents. Within this shared
2.2 Large Language Model-based Multi-Agents space, agents collectively allocate tasks, define their re-
Large Language models (LLMs) have become integral spective roles, and formulate individual driving plans.
to multi-agent systems [23] due to their capabilities Each agent then generates a rule-based driving plan-
in generalization and common-sense reasoning. LLM- ner in Reason Engine, which is subsequently executed
based multi-agent systems leverage these strengths to within the Execution Module to ensure coordinated
enhance decision-making and communication among driving behavior and smooth traffic flow.
agents [33]. This approach is particularly beneficial in
complex scenarios like mixed-autonomy traffic, where it 3.1 Single-Agent Pipeline
is essential to have effective interaction between human The quality of prompts significantly influences the out-
drivers and autonomous systems. put quality of LLM. CoMAL utilizes a prompt gener-
The communication structure of LLM-based multi- ator that integrates all essential information for effec-
agent systems varies across different studies to ad- tive decision-making. The workflow for each individ-
dress specific challenges [23, 34, 24]. For example, ual agent involves several steps: (1) encode the sce-
research [24] has explored both centralized and de- nario into a textual description within the Perception
centralized communication structures for LLM-based Module; (2) recall relevant driving experiences from the
 Driving Task
You are the brain of an autonomous vehicle in the road. Your
vehicle ID is CAV_01. You can connect all the autonomous
vehicles in the scenario. Please make the decision to optimize the
average velocity of all the vehicles. Try your best to avoid collisions Collaboration Module
with other objects. Thank you all for your cooperation. Since everyone has confirmed their
positions and roles, I will lead the queue, setting a consistent speed to
maintain safe distances. Let's proceed safely and ensure we avoid
Map description collisions while maintaining optimal speed. Drive safely, everyone!
The merge map consists of two straight single lanes. One is the My Role Decision: "Leader"
main highway where all vehicles drive, and another is the merge
branch to intersect with main highy way. The perturbations caused
by the vehicles entering through the ramp leads to the formation of Reason Engine
congested waves downstream in the main highway. Step 1. Role Clarification
After reviewing the positions the consensus shown by other vehicles, I
am ready to take on the role of the leader.
Motion state
Your speed is 3.80 m/s, and lane position is 11.75 m. Step 2. Scene Understanding
There are other vehicles driving around you, and below is their My position at 27.26m provides a good overview ahead and control over
basic information: initiating movement through the intersection.
- Vehicle CAV_02 is driving on the same lane as you. The speed of Step 3. Motion Instruction
it is 4.51 m/s, and lane position is 15.64 m. ......... I propose we start moving slowly and steadily to ensure all vehicles can
follow safely.
Step 4. Planner Generation
Let’s use the planner: driving_planner(a_max = 0.2, v_target = 6.0,
min_headway = 2.0)

Figure 2: (a) Left: A detailed prompt example for CoMAL, consisting of a system prompt that specifies the
driving task, along with map description and motion state provided by the Perception Module. (b) Right: A case
of the collaboration and reasoning process. Following task allocation during brainstorming, a hierarchical chain
of thought breaks down the driving plan into incremental steps, ensuring consistency in decision-making. This
process includes role clarification, scene understanding, motion instruction, and planner generation.

Memory Module; (3) receive shared messages from other hicle and surrounding agents, which directly influences
agents through the Collaboration Module; (4) gener- the planning of vehicles’ movement.
ate prompts and feed it into the LLM; (5) decode the
LLM response for task allocation or planning purposes. 3.1.2 Memory Module Similar to human drivers,
In this section, we detail the Environment Perception the agent must make decisions based on reasoning
Module and Memory Module. processes that are informed by past driving experiences.
To achieve this, we employ a Memory Module that
3.1.1 Environment Perception Module To effi- stores experiences from previous driving scenarios and
ciently extract prompts from complex environmental handmade instructions. Initially, the agent is provided
data and enhance the scene understanding of LLMs, we with a set of predefined experiences, which the LLM
design an Environment Perception Module. This mod- then updates continuously as it engages in reasoning
ule extracts key information from the simulation envi- during new situations. This approach allows the agent
ronment and constructs a textual scenario description. to refine its decision-making over time, improving its
The description follows a set of standard rules to gener- performance in diverse driving contexts.
ate a thorough representation in natural language. The
scene information is divided into two parts: static map 3.2 Multi-Agent Workflow In a mixed-autonomy
and dynamic agents, as shown in Figure 2. traffic setting, where CAVs operate alongside human
The static map information represents the scenario drivers, the main objective is to optimize overall traffic
type, providing semantic priors for vehicle motion plan- flow. To achieve this, we propose a three-stage decision-
ning. The description of the map helps the LLM in- making workflow. In the Collaboration Module, agents
tuitively understand the scenario’s geometry. The dy- first discuss and allocate tasks within a public message
namic information describes the motion of the ego ve- pool. In Reason Engine, each agent then independently
 Good day, fellow vehicles! This is Vehicle 01. In order to form an efficient queue that allows us all to pass quickly
and safely, I propose the following initial order according to our current positions: Vehicle 1, Vehicle 2, Vehicle 3,
Vehicle 4 ...... Each vehicle should maintain a safe following distance from the vehicle in front. Please confirm your
agreement or suggest adjustments. Let's ensure we avoid collisions and maintain optimal speed!
CAV-01 My Role Decision: "Not decided yet"

Hello everyone, this is Vehicle 02. I agree with Vehicle 01's proposed order since it reflects our current positions
accurately. If no one has any objections, I suggest we move forward with this plan and start adjusting our speeds to
maintain safe distances. Waiting for confirmation from the rest. Thank you!
My Role Decision: "Follower"
CAV-02
......
Thank you all for your cooperation. Since everyone has confirmed their positions and roles, I will lead the queue,
setting a consistent speed to maintain safe distances. Here's the final order and role assignment: 1. Vehicle 1
(Leader) 2. Vehicle 2 (Leading Follower) 3. Vehicle 3 (Follower).... Let's proceed safely and ensure we avoid
collisions while maintaining optimal speed. Drive safely, everyone!
CAV-01 My Role Decision: "Leader"

Figure 3: Demonstration of the interaction process of agents in the scenario Figure Eight 1. The agents decide
to form a queue and subsequently allocate the roles of leader and follower.

determines its driving plan based on its assigned role System Prompt The system prompt defines the
and generates a driving planner. Finally, the driving planning task and associated driving knowledge. Its pri-
planner is executed in the Execution Module. mary goal is to standardize the format of both input and
output data, as well as clarify the objectives of planner
3.2.1 Collaboration Module Collaborative agents generation. Specifically, it ensures a clear understanding
work together towards a shared objective, typically ex- of the physical meaning of each parameter in the IDM
changing information to enhance the collective solution. planner, such as speed limit (v0 ), maximum accelera-
In the Collaboration Module, all CAVs participate in- tion (am ), and minimum headway (s0 ). This provides a
teractively by forming a queue for brainstorming and structured foundation for the decision-making process.
communication. In the brainstorming session, the vehi- Hierarchical Chain-of-thougts The hierarchical
cles take turns subsequently speaking in a public chan- chain-of-thought process involves four critical compo-
nel to propose strategies and assign tasks among them- nents: role clarification, scene understanding, motion
selves. This collaborative effort ensures that each CAV instruction, and planner generation. Initially, it is cru-
understands its specific role in the traffic system. cial to clarify the role and task of the ego vehicle within
Communication Structure Here, we introduce a a collaborative context. Then the LLM is directed to
shared message pool to boost communication efficiency, focus on key information in the scenario, such as head-
as shown in Figure 3. This communication structure way distance and lead vehicles. Based on the scenario
maintains a shared message pool where agents can analysis, the LLM is then prompted to provide motion
publish messages and subscribe to the latest messages instructions for the ego vehicle. Finally, each agent uti-
from one another. Agents will take turns to speak one lizes scenario analysis and motion instructions to gen-
at a time until the strategy is fully developed and each erate a driving planner, parameterized by IDM model.
agent’s role is clearly defined.
3.2.3 Execution Module We utilize the rule-based
3.2.2 Reason Engine During team brainstorming, IDM model as a planner to execute driving strategies by
each agent determines its role and formulates a strat- adjusting its parameters. IDM is a car-following model
egy to collaborate with other vehicles. Based on these to compute longitudinal dynamics. In this model, the
defined roles, the Reason Engine generates an appro- acceleration ak for vehicle k is defined by its bumper-
priate driving planner to effectively control the vehi- to-bumper headway sk (distance to preceding vehicle),
cle. The Reason Engine takes scenario description and velocity vk , and relative velocity ∆vk , via the following
predefined system prompts as inputs. Subsequently, equation:
the LLM generates the driving planner based on IDM
through reasoning guided by a hierarchical chain-of-
thought prompt.
 Figure Eight (FE) The FE network builds on
the ring road by connecting two circular loops via an
dvk vk s∗ (vk , ∆vk ) 2
(3.1) ak = = amax [1 − ( )δ − ( ) ] intersection. In mixed-autonomy scenarios, CAVs are
dt v0 sk introduced to smooth traffic and prevent stop-and-go
where s∗ is the desired headway of the vehicle, waves.
denoted by: Merge The merged network simulates highway
disturbances caused by vehicles entering from an on-
ramp, which creates stop-and-go waves. In mixed-
vk ∆vk
(3.2) s∗ (vk , ∆vk ) = s0 + max(0, vk T + √ ) autonomy scenarios, CAVs are tasked with mitigating
2 amax b these waves based on local observations and adjusting
where s0 , v0 , T, δ, amax , b are given parameters. We to fluctuating vehicle numbers in the open network.
set the desired time headway T , the comfortable braking We investigate different levels of difficulty for each
deceleration b, and the acceleration exponent δ as proposed benchmark by adjusting their scenario-specific
constants while adjusting the desired velocity v0 , the meta-parameters. Table 1 provides detailed descriptions
minimum spacing s0 , and the maximum acceleration of the selected meta-parameters for each benchmark.
amax to tailor the driving planners. Thus Reason
Engine generates a driving planner by customizing Table 1: Configurations of Benchmarks
IDM’s parameters (v0 , amax , s0 ).
Scenario Name Time(s) Vehicles Distribution
4 Experiments FE 0 150 13 humans, 1 CAV
In a mixed-autonomy setting, a subset of vehicles are FE 1 150 7 humans, 7 CAVs
tasked with the objective of improving overall traffic FE 2 150 0 humans, 14 CAVs
flow and mitigating the formation and propagation of Ring 0 150 21 humans, 1 CAV
stop-and-go waves. Thus, in our experiments, we aim Ring 1 150 19 humans, 3 CAVs
to address several key questions: Ring 2 150 11 humans, 11 CAVs
Merge 0 75 10.0% CAV penetration rate
• How can CAVs enhance traffic flow and eliminate Merge 1 75 25.0% CAV penetration rate
stop-and-go shockwaves? Merge 2 75 33.3% CAV penetration rate
Merge 3 75 50.0% CAV penetration rate
• How do multiple LLM-based agents collaborate to Merge 4 75 90.0% CAV penetration rate
achieve this goal?
• Do different LLM models influence the results? 4.3 Metrics To provide a comprehensive assessment
of traffic flow and mitigate the occurrence of shock-
4.1 Implementation Details The experiments are waves, we utilize two metrics:
conducted in Flow [15] with SUMO [37], a microscopic • Average vehicle speed in the network (m/s). Higher
simulator for traffic and vehicle dynamics. For details values indicate better overall traffic flow.
on the architecture and on training autonomous vehicles
to maximize system-level velocity, we refer the readers • Standard deviation of vehicle speed (m/s). The
to [15]. The environment offers several driving models smaller is more stable. Lower values reflect greater
to simulate human driver and a realistic interaction stability and consistency in traffic movement.
between vehicles. We adopt OpenAI GPT-4o-mini,
Qwen-72B/32B/7B in this paper. 4.4 Specification on Communication In this sec-
tion, we focus on the interactive process among agents
4.2 Scenarios We evaluate our model on the Figure as they work to solve the mixed-traffic problem. In the
Eight (FE), Ring, and Merge scenarios from the Flow FE scenario, the agents recognize the need to form a
benchmark. Further details are provided below and queue, identify a leader, and designate the remaining
illustrated in Figure 1 (c). agents as followers. The process of task allocation and
Ring The ring road network consists of a circu- leader selection is illustrated in Figure 3. Additionally,
lar lane where vehicles continuously travel in a loop. in the ring and merge scenarios, agents aim to elimi-
It is commonly used to study traffic dynamics, as dis- nate shockwaves. Their reasoning is as follows: if there
turbances can cause stop-and-go waves. In mixed- is relative traffic congestion ahead of the ego vehicle, the
autonomy scenarios, CAVs are deployed to reduce these agent approaches the lead vehicle slowly; otherwise, it
waves and enhance traffic flow stability. accelerates to follow the lead vehicle closely.
 Table 2: Quantitative Evaluation of CoMAL on Flow Benchmarks
Metric Model FE 0 FE 1 FE 2 Ring 0 Ring 1 Ring 2 Merge 0 Merge 1 Merge 2 Merge 3 Merge 4
Human Driver 5.61 5.61 5.61 2.88 2.88 2.88 6.40 6.40 6.40 6.40 6.40
Avg
CoMAL 6.40 6.47 6.29 2.86 2.85 2.87 6.59 7.40 7.42 7.86 8.83
Human Driver 4.55 4.55 4.55 0.79 0.79 0.79 3.12 3.12 3.12 3.12 3.12
Std
CoMAL 1.74 1.77 2.24 0.29 0.26 0.31 2.88 2.91 2.61 2.47 2.70

Figure 4: Visualization of vehicle trajectories in Ring 0 setting. The ring road has a total length of 230 meters
and contains 22 vehicles. Each line in the space-time diagrams shows the position of a specific vehicle over time.
When a vehicle completes a full lap of the ring, its position resets to zero. Left: In the absence of automated
vehicles, human-driven vehicles exhibit stop-and-go shockwaves due to inherent instability. Right: With three
connected autonomous vehicles using the CoMAL framework, the unstable vehicles are stabilized.

4.6 Ablation Studies We conducted a detailed

Table 3: Ablation Study
analysis of the effectiveness of each component of Co-
No. Perception Memory Collaboration FE 1 Merge 1 MAL in the FE 1 and Merge 1 scenarios, as presented
1 × × × 5.61 6.40 in Table 3.
2 × ✓ ✓ 5.81 6.51 Ablation on Perception The comparisons in the
3 ✓ × ✓ 5.17 6.72 second and fifth rows of Table 3 demonstrate the
4 ✓ ✓ × 5.18 6.88 effectiveness of incorporating textual descriptions of
5 ✓ ✓ ✓ 6.47 7.40 the map and agents’ motion states in the Perception
Module. Once perception information is lost, agents are
no longer able to comprehend the spatial relationships
between the ego vehicle and surrounding agents. As
4.5 Quantitative Results We evaluated our model a result, their capacity for effective collaboration and
on the aforementioned benchmarks, varying the number reasoning is significantly impaired.
or percentage of CAVs across different settings. Ablation on Memory The comparisons in the
As shown in Table 2, we compared the performance third and fifth rows of Table 3 illustrate the impact
of CoMAL with that of human drivers. The results in- of the Memory Module, in which specific experiences
dicate that as the number of CAVs increases, CoMAL’s are allocated for each scenario. In the absence of
performance generally improves and surpasses that of high-quality experiences, agents are more susceptible to
human drivers. This highlights the strong capability of errors in both discussion and reasoning.
LLM agents in achieving effective cooperation. A visu- Ablation on Collaboration The comparison pre-
alization of the vehicle trajectories in Ring 0 setting is sented in the fourth and fifth rows of Table 3, as well as
shown in Figure 4. We can see that the proposed Co- in the first row, highlights the effectiveness of the Col-
MAL framework can stabilize the unstable vehicle flow. laboration Module. The absence of a collaboration mod-
 Table 4: Quantitative Experiment of Different LLMs: Average Velocity and Standard Deviation Analysis
Average Velocity (m/s)
Model
FE 0 FE 1 FE 2 Ring 0 Ring 1 Ring 2 Merge 0 Merge 1 Merge 2 Merge 3 Merge 4
Human Driver 5.61 5.61 5.61 2.88 2.88 2.88 6.40 6.40 6.40 6.40 6.40
GPT-4o-mini 6.40 6.47 6.29 2.86 2.85 2.87 6.59 7.40 7.42 7.86 8.83
Qwen-72B 6.37 5.96 6.07 2.86 2.83 2.85 6.66 7.38 7.46 8.07 8.75
Qwen-32B 6.39 5.73 6.11 2.84 2.86 2.81 6.58 7.12 7.39 7.73 8.54
Qwen-7B 5.60 5.17 4.80 2.82 2.84 2.83 6.55 6.87 7.05 7.47 8.53
Standard Deviation (m/s)
Model
FE 0 FE 1 FE 2 Ring 0 Ring 1 Ring 2 Merge 0 Merge 1 Merge 2 Merge 3 Merge 4
Human Driver 4.55 4.55 4.55 0.79 0.79 0.79 3.12 3.12 3.12 3.12 3.12
GPT-4o-mini 1.74 1.77 2.24 0.29 0.26 0.31 2.88 2.91 2.61 2.47 2.70
Qwen-72B 1.73 2.76 1.92 0.32 0.32 0.30 2.82 2.71 2.54 2.61 2.78
Qwen-32B 1.74 2.37 2.64 0.33 0.37 0.33 2.81 2.99 2.62 2.57 2.50
Qwen-7B 4.51 1.89 1.76 0.49 0.61 0.48 3.02 2.88 2.74 2.71 2.64

Comparison of various LLM models We eval-

Table 5: Comparison to RL Benchmark
uate performance across LLM models of varying sizes
Model FE 0 FE 1 FE 2 Merge 0 Merge 1 Merge 2 (see Table 4). The GPT-4o-mini achieves the highest
IDM 5.61 5.61 5.61 6.40 6.40 6.40 performance among these. Among open-source models,
ARS 7.31 6.43 5.70 11.30 11.06 11.50 the Qwen 72B has a similar level with the GPT-4o-mini,
ES 6.87 - 5.96 13.31 17.29 17.36 while the Qwen 32B shows slightly lower performance,
TRPO 8.26 5.61 5.03 14.95 13.74 14.14 and the Qwen 7B performs significantly worse. Notably,
PPO 8.26 5.61 5.03 13.66 14.61 14.54 we observe that in scenarios requiring extensive collab-
CoMAL 6.40 6.47 6.29 6.59 7.40 7.42 oration, the performance of smaller models deteriorates
more rapidly. This finding suggests that collaboration
is a more challenging task than reasoning within the
CoMAL framework.
ule causes all agents to adopt nearly identical strategies,
which in turn leads to conflicts and duplicated efforts. 5 Conclusion
The absence of collaboration results in performance that In this paper, we present CoMAL, an effective
can be even worse than that of the simple rule-based LLM-based multi-agent framework to address mixed-
model shown in the first row. autonomy traffic challenges. By prompt-tuning LLMs
with a hierarchical LLM-based planner, CoMAL is able
4.7 Discussion to handle complex vehicle driving tasks towards a col-
Compared to RL method We conducted exper- laborative goal under mixed-autonomy traffic. The
iments on FE and the Merge scenarios by comparing LLM agent does so by serving primarily as a high-level
them to RL methods developed in [7]. The results commander, coordinating with lower-level controllers
are shown in Table 5. The Ring is not included in to execute detailed operations. Extensive experiments
this experiment because the adopted RL benchmark demonstrate the significant potential of multi-agent sys-
doesn’t have the Ring scenario embedded. In the FE tems driven by LLMs to make informed decisions and
scenario, CoMAL demonstrates robust global collabo- collaborate effectively in driving scenarios.
ration, whereas multi-agent RL models struggle to dif- We also acknowledge the limitations of our current
ferentiate roles, hindering effective cooperation. Con- work and would like to point out several important fu-
sequently, CoMAL outperforms RL-based approaches. ture directions. First, we can extend the experimental
However, in the Merge scenario, CoMAL performs less current settings to more agents and improve their col-
effectively than RL, indicating that the collaboration is laboration to see if there would be emerging behaviors
not global. This finding highlights the critical impor- formed like in RL methods. Second, our paper addresses
tance of cooperation in enhancing performance. a simplified use of LLMs alone, whereas the combina-
tion of RL with LLM might be helpful in improving vol. 91, no. 12, pp. 2043-2067, Dec. 2003, doi:
the performance of LLMs. In the future, more sophis- 10.1109/JPROC.2003.819610.
ticated scenarios will be explored to test the possibility [11] B. Besselink and K. H. Johansson, ”String Stability
of LLM’s behavior in complex tasks. and a Delay-Based Spacing Policy for Vehicle Platoons
Subject to Disturbances,” in IEEE Transactions on
Automatic Control, vol. 62, no. 9, pp. 4376-4391, Sept.
Acknowledgements
2017, doi: 10.1109/TAC.2017.2682421.
We thank OpenAI for providing us with API credits [12] SHLADOVER, STEVEN E. 1995. “Review of the State
under the Researcher Access program. of Development of Advanced Vehicle Control Systems
(AVCS).” Vehicle System Dynamics 24 (6–7): 551–95.
doi:10.1080/00423119508969108.
References
[13] Taniguchi, Y., Nishi, R., Tomoeda, A., Shimura, K.,
Ezaki, T., Nishinari, K. (2015). A Demonstration Ex-
[1] P. S. Chib and P. Singh, ”Recent Advancements in periment of a Theory of Jam-Absorption Driving. In:
End-to-End Autonomous Driving Using Deep Learn- Chraibi, M., Boltes, M., Schadschneider, A., Seyfried,
ing: A Survey,” in IEEE Transactions on Intelligent A. (eds) Traffic and Granular Flow ’13. Springer,
Vehicles, vol. 9, no. 1, pp. 103-118, Jan. 2024, doi: Cham. https://doi.org/10.1007/978-3-319-10629-853
10.1109/TIV.2023.3318070. [14] S. Mosharafian and J. M. Velni, ”Cooperative Adaptive
[2] Hallgarten, M., Zapata, J., Stoll, M., Renz, K., Zell, Cruise Control in a Mixed-Autonomy Traffic System:
A. (2024). Can Vehicle Motion Planning General- A Hybrid Stochastic Predictive Approach Incorporat-
ize to Realistic Long-tail Scenarios?. arXiv preprint ing Lane Change,” in IEEE Transactions on Vehicular
arXiv:2404.07569. Technology, vol. 72, no. 1, pp. 136-148, Jan. 2023, doi:
[3] Da, L., Gao, M., Mei, H., Wei, H. (2024). Prompt 10.1109/TVT.2022.3202084.
to Transfer: Sim-to-Real Transfer for Traffic Signal [15] C. Wu, A. R. Kreidieh, K. Parvate, E. Vinitsky and
Control with Prompt Learning. Proceedings of the A. M. Bayen, ”Flow: A Modular Learning Framework
AAAI Conference on Artificial Intelligence, 38(1), 82- for Mixed Autonomy Traffic,” in IEEE Transactions on
90. https://doi.org/10.1609/aaai.v38i1.27758 Robotics, vol. 38, no. 2, pp. 1270-1286, April 2022, doi:
[4] Y. Hu et al., ”Planning-oriented Autonomous Driv- 10.1109/TRO.2021.3087314.
ing,” 2023 IEEE/CVF Conference on Computer [16] Wu, C., Kreidieh, A., Vinitsky, E. Bayen, A.M.
Vision and Pattern Recognition (CVPR), Vancou- (2017). Emergent Behaviors in Mixed-Autonomy
ver, BC, Canada, 2023, pp. 17853-17862, doi: Traffic. Proceedings of the 1st Annual Confer-
10.1109/CVPR52729.2023.01712. ence on Robot Learning, in Proceedings of Ma-
[5] Dauner, D., Hallgarten, M., Geiger, A., Chitta, K. chine Learning Research 78:398-407 Available from
(2023, December). Parting with misconceptions about https://proceedings.mlr.press/v78/wu17a.html.
learning-based vehicle motion planning. In Conference [17] R. Valiente, B. Toghi, R. Pedarsani and Y. P. Fal-
on Robot Learning (pp. 1268-1281). PMLR. lah, ”Robustness and Adaptability of Reinforcement
[6] Mao, J., Qian, Y., Ye, J., Zhao, H., Wang, Y. (2023). Learning-Based Cooperative Autonomous Driving in
Gpt-driver: Learning to drive with gpt. arXiv preprint Mixed-Autonomy Traffic,” in IEEE Open Journal of
arXiv:2310.01415. Intelligent Transportation Systems, vol. 3, pp. 397-410,
[7] Vinitsky, E., Kreidieh, A., Flem, L.L., Kheterpal, 2022, doi: 10.1109/OJITS.2022.3172981.
N., Jang, K., Wu, C., Wu, F., Liaw, R., Liang, E. [18] Liu, Lu, Maonan Wang, Man-On Pun and Xi Xiong.
Bayen, A.M.. (2018). Benchmarks for reinforcement “A Multi-Agent Rollout Approach for Highway Bot-
learning in mixed-autonomy traffic. Proceedings of The tleneck Decongenston in Mixed Autonomy.” ArXiv
2nd Conference on Robot Learning, in Proceedings of abs/2405.03132 (2024): n. pag.
Machine Learning Research 87:399-409 Available from [19] Glanois, C., Weng, P., Zimmer, M. et al. A survey on
https://proceedings.mlr.press/v87/vinitsky18a.html. interpretable reinforcement learning. Mach Learn 113,
[8] Chang, T., Lai, I. (1997). Analysis of characteristics of 5847–5890 (2024). https://doi.org/10.1007/s10994-
mixed traffic flow of autopilot vehicles and manual ve- 024-06543-w
hicles. Transportation Research Part C-emerging Tech- [20] LeCun, Yann and Courant. “A Path Towards Au-
nologies, 5, 333-348. tonomous Machine Intelligence Version 0.9.2, 2022-06-
[9] C. Wu, A. M. Bayen and A. Mehta, ”Stabilizing 27.” (2022).
Traffic with Autonomous Vehicles,” 2018 IEEE In- [21] M.A. Veldman, S. Doolaard, R.J. Bosker, T.A.B.
ternational Conference on Robotics and Automation Snijders, Young children working together. Cooper-
(ICRA), Brisbane, QLD, Australia, 2018, pp. 6012- ative learning effects on group work of children in
6018, doi: 10.1109/ICRA.2018.8460567. Grade 1 of primary education, Learning and In-
[10] M. Papageorgiou, C. Diakaki, V. Dinopoulou, A. struction, Volume 67, 2020, 101308, ISSN 0959-4752,
Kotsialos and Yibing Wang, ”Review of road traf- https://doi.org/10.1016/j.learninstruc.2020.101308.
fic control strategies,” in Proceedings of the IEEE, [22] Wen, L., D. Fu, X. Li, X. Cai, T. Ma, P. Cai, M. Dou,
 B. Shi, L. He, and Y. Qiao. “DILU: A KNOWLEDGE- arXiv:2404.11584, 2024.
DRIVEN APPROACH TO AUTONOMOUS DRIV- [35] Lai S, Xu Z, Zhang W, et al. Large language models
ING WITH LARGE LANGUAGE MODELS.” In 12th as traffic signal control agents: Capacity and opportu-
International Conference on Learning Representations, nity[J]. arXiv preprint arXiv:2312.16044, 2023.
ICLR 2024, 2024. [36] Da L, Liou K, Chen T, et al. Open-ti: Open traffic
[23] Guo, Taicheng, Xiuying Chen, Yaqi Wang, Ruidi intelligence with augmented language model[J]. Inter-
Chang, Shichao Pei, N. Chawla, Olaf Wiest and Xi- national Journal of Machine Learning and Cybernetics,
angliang Zhang. “Large Language Model based Multi- 2024: 1-26.
Agents: A Survey of Progress and Challenges.” ArXiv [37] Lopez P A, Behrisch M, Bieker-Walz L, et al. Mi-
abs/2402.01680 (2024): n. pag. croscopic traffic simulation using sumo[C]//2018 21st
[24] Talebirad, Yashar and Amirhossein Nadiri. “Multi- international conference on intelligent transportation
Agent Collaboration: Harnessing the Power of Intel- systems (ITSC). IEEE, 2018: 2575-2582.
ligent LLM Agents.” ArXiv abs/2306.03314 (2023): n.
pag.
[25] Duan, Y., Chen, X., Houthooft, R., Schulman,
J. Abbeel, P. (2016). Benchmarking Deep Re-
inforcement Learning for Continuous Control.
Proceedings of The 33rd International Conference
on Machine Learning, in Proceedings of Machine
Learning Research 48:1329-1338 Available from
https://proceedings.mlr.press/v48/duan16.html.
[26] Bellemare M G, Naddaf Y, Veness J, et al. The
arcade learning environment: An evaluation platform
for general agents[J]. Journal of Artificial Intelligence
Research, 2013, 47: 253-279.
[27] John Schulman, Sergey Levine, Philipp Moritz,
Michael Jordan, and Pieter Abbeel. 2015. Trust re-
gion policy optimization. In Proceedings of the 32nd In-
ternational Conference on International Conference on
Machine Learning - Volume 37 (ICML’15). JMLR.org,
1889–1897.
[28] Schulman J, Wolski F, Dhariwal P, et al. Proxi-
mal policy optimization algorithms[J]. arXiv preprint
arXiv:1707.06347, 2017.
[29] Salimans T, Ho J, Chen X, et al. Evolution strategies
as a scalable alternative to reinforcement learning[J].
arXiv preprint arXiv:1703.03864, 2017.
[30] Mania H, Guy A, Recht B. Simple random search pro-
vides a competitive approach to reinforcement learn-
ing[J]. arXiv preprint arXiv:1803.07055, 2018.
[31] Hao Mei, Junxian Li, Bin Shi, and Hua Wei, Reinforce-
ment learning approaches for traffic signal control un-
der missing data, Proceedings of the Thirty-Second In-
ternational Joint Conference on Artificial Intelligence,
pp. 2261–2269, 2023.
[32] Mei, H., Lei, X., Da, L. et al. Libsignal: an open library
for traffic signal control. Mach Learn 113, 5235–5271
(2024). https://doi.org/10.1007/s10994-023-06412-y
[33] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang,
Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang,
Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei,
and Jirong Wen. 2024. A survey on large language
model based autonomous agents. Front. Comput. Sci.
18, 6 (Dec 2024). https://doi.org/10.1007/s11704-024-
40231-1
[34] Masterman T, Besen S, Sawtell M, et al. The land-
scape of emerging ai agent architectures for reasoning,
planning, and tool calling: A survey[J]. arXiv preprint