DualViewDistill - Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Tracking
This repository is the official implementation of the paper:
Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Tracking
Markus Käppeler, Özgün Cicek, Daniele Cattaneo, Claudius Gläser, Yakov Miron, and Abhinav Valada.
If you find our work useful in your research or applications, please consider giving us a star or citing our paper: TODO
13 October, 2025: DualViewDistill sets the state-of-the-art on the nuScenes test set for camera-only online 3D detection and multi-object tracking (0.695 NDS, 0.621 mAP, 0.669 AMOTA, 407 IDS) and also achieves state-of-the-art results on Argoverse 2.
Camera-based 3D object detection and tracking are essential for perception in autonomous driving. Current state-of-the-art approaches often rely exclusively on either perspective-view (PV) or bird’s-eye-view (BEV) features, limiting their ability to leverage both fine-grained object details and spatially structured scene representations. In this work, we propose DualViewDistill, a hybrid detection and tracking framework that incorporates both PV and BEV camera image features to leverage their complementary strengths. Our approach introduces BEV maps guided by foundation models, leveraging descriptive DINOv2 features that are distilled into BEV representations through a novel distillation process. By integrating PV features with BEV maps enriched with semantic and geometric features from DINOv2, our model leverages this hybrid representation via deformable aggregation to enhance 3D object detection and tracking. Extensive experiments on the nuScenes and Argoverse 2 benchmarks demonstrate that DualViewDistill achieves state-of-the-art performance. The results showcase the potential of foundation model BEV maps to enable more reliable perception for autonomous driving.
We will release the code upon the acceptance of our paper.
This research was funded by Bosch Research as part of a collaboration between Bosch Research and the University of Freiburg on AI-based automated driving.