cat README.md
# Vision-Based Drone Flight
A quadrotor that **follows another drone using only its onboard camera**: a
learned detector finds the target in the image, and a reinforcement-learning
policy turns those detections into flight commands. Team project for the
*Vision-Based Drone Flight* course at the Robotics and Perception Group, UZH.
## Why
Chasing a moving target from a single camera couples perception and control:
the detector is noisy, late, and sometimes blind, and the controller has to stay
smooth and safe anyway. On top of that the policy is trained purely in
simulation, so everything it learns has to survive the jump to real hardware.
## How
- **Detection** — camera and target poses are sampled and rendered into a
synthetic dataset, augmented with the real camera's artifacts (transmission
noise, motion blur, contrast), and used to train a **YOLOv5** drone detector
that outputs bounding boxes in real time
- **Hover** — a first **PPO** policy learns a smooth hover, with action-rate and
body-rate costs replacing the bang-bang thrust of a naive reward
- **State-based tracking** — the policy follows a target flying random splines,
rewarded for holding its distance and keeping the target near the optical axis
- **Vision-based tracking** — the target is observed only through a short
history of bounding boxes, with a detector model in the loop that simulates
latency, jitter, missed frames, and false detections
- **Sim-to-real** — randomized mass, drag, thrust, and command delay during
training; candidate policies are stress-tested under extra lag and model
mismatch before one is picked to fly
- **Deployment** — the policy and detector run as **ROS 2** nodes on the real
quadrotor, and the flown trajectories are replayed in simulation to compare
## Results
Replaying a real flight in the simulator showed the two agree on tracking
accuracy — a median error of roughly 11 cm to a moving reference on both sides.
What the simulator missed was a small vertical oscillation on the real drone,
which we traced to a longer effective command delay than the model assumed.
## Stack
- **Simulation & RL** — Isaac Lab, PPO (Stable-Baselines3), PyTorch
- **Perception** — YOLOv5, synthetic rendering, image augmentation
- **Deployment** — ROS 2, Docker
ls -l
Python, PyTorch, Isaac Lab, Stable-Baselines3, YOLOv5, ROS 2, Docker