cat README.md
# Vision-Based Drone Flight A quadrotor that **follows another drone using only its onboard camera**: a learned detector finds the target in the image, and a reinforcement-learning policy turns those detections into flight commands. Team project for the *Vision-Based Drone Flight* course at the Robotics and Perception Group, UZH. ## Why Chasing a moving target from a single camera couples perception and control: the detector is noisy, late, and sometimes blind, and the controller has to stay smooth and safe anyway. On top of that the policy is trained purely in simulation, so everything it learns has to survive the jump to real hardware. ## How - **Detection** — camera and target poses are sampled and rendered into a synthetic dataset, augmented with the real camera's artifacts (transmission noise, motion blur, contrast), and used to train a **YOLOv5** drone detector that outputs bounding boxes in real time - **Hover** — a first **PPO** policy learns a smooth hover, with action-rate and body-rate costs replacing the bang-bang thrust of a naive reward - **State-based tracking** — the policy follows a target flying random splines, rewarded for holding its distance and keeping the target near the optical axis - **Vision-based tracking** — the target is observed only through a short history of bounding boxes, with a detector model in the loop that simulates latency, jitter, missed frames, and false detections - **Sim-to-real** — randomized mass, drag, thrust, and command delay during training; candidate policies are stress-tested under extra lag and model mismatch before one is picked to fly - **Deployment** — the policy and detector run as **ROS 2** nodes on the real quadrotor, and the flown trajectories are replayed in simulation to compare ## Results Replaying a real flight in the simulator showed the two agree on tracking accuracy — a median error of roughly 11 cm to a moving reference on both sides. What the simulator missed was a small vertical oscillation on the real drone, which we traced to a longer effective command delay than the model assumed. ## Stack - **Simulation & RL** — Isaac Lab, PPO (Stable-Baselines3), PyTorch - **Perception** — YOLOv5, synthetic rendering, image augmentation - **Deployment** — ROS 2, Docker

ls -l
tech Python, PyTorch, Isaac Lab, Stable-Baselines3, YOLOv5, ROS 2, Docker