AR Drone
A mixed-reality prototype made with Lund University School of Aviation: track a physical drone with a Meta Quest 3 and let it collide with virtual obstacles in Unity.
The rig
Why mix the two
Lund University School of Aviation wanted to explore whether drone training could use augmented reality instead of relying only on physical courses or simulation. The prototype kept the drone real and moved the course into mixed reality.
The technical problem was tracking one physical object well enough for virtual collision. The project iterated through drone detection approaches before using a HuggingFace model with Meta's object-detection building blocks on Quest 3. The obstacle course was made in Blender and Unity: one ring to fly through, plus a sphere and three rectangular obstacles to avoid. On collision, the obstacle turns red and a crash sound plays.