Autonomous driving works reasonably well when traffic is orderly and the environment is structured. It works far less well when these assumptions do not hold. CRΔL's work targets the latter case where traffic is dense, heterogeneous, and unstructured. This setting enhances the long-tail problem in autonomous driving.
Each stage below is a section on this page. The dashed path is the newer question we are testing: how much of the pipeline can a large model replace, and at what cost?
Everything downstream depends on knowing where the other agents are, and dense traffic is close to the worst case for a tracker. Agents are small, they occlude each other constantly, and a scooter and a bus do not move alike. A tracker that treats every detection as an independent box loses agents exactly when the scene is most crowded.
Our answer is to put the interaction model inside the tracker, so that the behavior causing the occlusion also helps resolve it.
Tracks pedestrians in dense crowds using a motion model of how people actually avoid each other (Front-RVO) rather than generic constant-velocity assumptions. The same collision avoidance that creates the occlusions becomes the prior that recovers from them.
Extends the idea to road agents of very different sizes and dynamics sharing the same space, and runs in real time on video from a moving vehicle.
Once you can see the agents, you need to know where they are going. In dense traffic that answer depends far more on the neighbors than on the road geometry, which inverts the usual assumption of lane-based predictors. A scooter two meters to the right constrains you more than the entire lane ahead.
Weights each neighbor by how much it actually constrains the ego agent instead of treating everything within a radius equally, and weights heterogeneous agents differently because a bus and a bicycle impose different costs.
Predicts end to end from raw sensor input, so the predictor absorbs the tracker's noise rather than assuming it was handed clean trajectories. Accuracy measured against ground-truth trajectories overstates what a predictor actually delivers inside a real stack.
Predicts where an agent will go and what kind of driver it is at the same time. The two help each other: knowing a driver is aggressive sharpens the trajectory, and the trajectory is the only evidence available for the label.
A planner in dense traffic cannot wait for a gap, because in this kind of traffic the gap is produced by the merge rather than preceding it. It has to reason about what the other drivers will concede, which makes planning here close to the social mini-games work on the other research page.
Each agent infers its neighbors' intent from their observed trajectories and plans against that estimate, instead of assuming that everyone shares a policy or a set of rules.
Most driving simulators generate well-behaved traffic, so policies trained in them are only ever tested against drivers who yield. B-GAP generates behavior-rich traffic with a controllable mix of aggressive and conservative drivers, and trains navigation policies in it.
"Aggressive" and "conservative" are useful words, but a planner needs a number, computed in real time, from data it already has. We build driver behavior measures out of the geometry of the interaction itself rather than out of hand-labeled video.
The common thread is a graph: agents are nodes, and the fact that one agent has to react to another is an edge. Most of what we want to say about behavior turns out to be a statement about that graph.
Defines driver behavior through centrality on the traffic interaction graph. A driver that many others must react to is, by that measure, aggressive, and the measure is computable from trajectories alone.
Classifies driver behavior from the spectrum of that same graph, which gives a classifier fast enough to run alongside the planner rather than offline.
The journal treatment that pulls the graph-theoretic line together and tests it across datasets, with the analysis the conference papers had no room for.
The behaviors that matter most are the ones you have least data for. This work recognizes heavy-tail driving behavior across diverse traffic scenarios by transferring from the common cases into the rare ones.
Dense, unstructured traffic was under-represented in public driving data for a long time, which meant that methods built for it could not be compared. We record and release the traffic we actually want to solve, and pay particular attention to the rare events, because those are the ones that decide whether a system is deployable.
A dense, heterogeneous, and unstructured traffic dataset annotated specifically for rare behaviors: the pedestrian crossing mid-block, the animal on the road, the scooter coming the wrong way down the lane.
A video dataset built around vulnerable road users, with atomic visual element annotations, in environments where their behavior is genuinely hard to anticipate.
Listed above under planning, and just as much a simulation contribution: it is how we generate the aggressive drivers that the datasets only capture occasionally.
The pipeline above is modular and you can inspect every stage, which is the argument for it. The argument against it is that errors compound across four stages, and each stage is tuned on a different objective than the one that matters. Large models offer to collapse the whole thing into one policy.
We are testing that claim rather than assuming it, and reporting the cases where it does not hold.
A case study rather than a proposal. We put an LLM in the loop of a decentralized driving policy and report where it helps and where it only adds cost.
Adapts an end-to-end driving policy to a new domain through prompting and in-context reinforcement learning instead of retraining, which is the property that would make deployment across regions practical.