02 / Vision & control
Pan/Tilt Camera Tracking Simulation
A two-axis camera tracking moving drone geometry in MuJoCo. Rendered camera frames feed day/night vision models, an object tracker, and a controller that moves both simulated joints.
Python / PyTorch / MuJoCo / C
Source code →- 44.40 → 2.04 px
- nominal mean error
- 16
- evaluation scenarios
- Simulation
- validation domain
Implementation
I connected segmentation and crop-classification models with a Kalman tracker and camera-rotation compensation. The simulated controller moves both axes to center a selected target. Scenarios include distractors, occlusion, lost frames, low light, and motion blur. A host build also exercises the C command parser and watchdog.
Results
In the recorded nominal daytime scenario, mean centering error fell from 44.40 pixels with a fixed camera to 2.04 pixels with tracking, a 95.4% reduction. The evaluation covers 16 scenarios; this result describes the nominal comparison, not every condition.
Observations
Dim and defocused scenes can lose tracking. Fast night motion reaches the pan-speed limit, and the combined spotlight/short-exposure scenario did not improve mean error over its comparison. Those failures matter as much as the nominal result.
Limitations
This is a simulation project using synthetic training data. Real-camera detection and physical hardware operation are unverified. Joint placement, mass, friction, and motor response are estimates. ESP32 firmware is included for a future physical build.
Code & reports
Simulation geometry adapted from Turret Mechanism by PJohnso1, licensed under CC BY 4.0. Original STL geometry was prepared for simulation.