Automating Low-Cost FPV Drones: Embedded AI, Computer Vision, and Real-Time Systems for Resource-Constrained UAV Platforms
Organizers and Lecturers
Sergii Baidachnyi, School of Engineering, The University of British Columbia,
Dr. Rakiba Rayhana, School of Engineering, The University of British Columbia,
Hao Liu (by Zoom), School of Engineering, The University of British Columbia,
Summary
Unmanned Aerial Vehicles (UAVs), especially low-cost first-person-view (FPV) multirotors, have become pivotal in modern conflicts, with millions of missions conducted annually in Ukraine alone. A typical front-line platform is a 10-inch quadcopter with a payload capacity of up to 3 kg, a cruise speed of 120–240 km/h, and a unit cost of approximately USD 600–700. These drones operate without GPS, often rely on analog video systems, and are simple enough to be assembled by individuals with minimal technical training after completing introductory online courses. Despite this simplicity, researchers and engineers are actively working to enable automated missions by integrating inexpensive compute modules into existing FPV designs.
While high-end military drones benefit from sophisticated autonomy, the low-cost FPV platforms that dominate real-world operations remain largely manual. This workshop provides a theoretical and practical foundation for developing mission-ready automated software on these resource-constrained platforms, and addresses a critical capability gap. To bridge this gap, solutions must combine classical computer vision, lightweight deep learning, real-time embedded systems design, and careful ethical reasoning.
Part 1 — Operational Context and Mission Analysis
This section presents the current state of the art in FPV drone design, flight control software, and communication protocols. Based on feedback from conflict zones, we identify a taxonomy of critical missions and their associated challenges, including environmental constraints, communication degradation, and human-factors issues. A representative example is a last-mile targeting system that enables an FPV pilot to designate a target and initiate an automated copilot mission, which is particularly valuable in complex terrain or under heavy electronic warfare interference. Ethical considerations regarding automated targeting and levels of autonomy are explicitly addressed throughout.
Part 2 — Hardware Platforms and Real-Time System Design
This part surveys System-on-Chip (SoC) and Neural Processing Unit (NPU) accelerators suitable for integration into FPV platforms, including Raspberry Pi with Hailo AI modules, Qualcomm Dragonwing SoC, and Toradex modules. Common pitfalls in designing best-effort versus real-time embedded systems are highlighted, and Linux-based approaches to asymmetric multiprocessing emulation using standard Debian images are demonstrated. Using one of the surveyed platforms, this part presents a software architecture for the missions identified in Part 1 and applies Rate Monotonic Analysis (RMA) to assess schedulable time and resource utilization.
Part 3 — Computer Vision for Resource-Constrained Platforms
Limited onboard computation makes advanced deep learning models impractical as standalone solutions. This part, therefore, focuses on classical computer vision methods, including Optical Flow, MOSSE, KCF, and CSRT trackers, and demonstrates their practical deployment. For instance, Optical Flow serves as the basis for navigation and last-mile targeting for stationary targets, while CSRT enables tracking of fast-moving targets, albeit at reduced frame rates (a few FPS on a Raspberry Pi Compute Module 4 with Full HD input). Hybrid approaches that combine classical methods with lightweight deep learning models at low frame rates to handle complex backgrounds are analyzed. Special attention is given to night missions using thermal cameras, where lower resolution can be mitigated by narrow-field-of-view optics, and where trackers like MOSSE and CSRT exhibit different performance characteristics.
Part 4 — Simulation, Design Validation, and Field Readiness
This part presents tools and workflows for drone design prototyping and mission simulation, demonstrates several simulation workloads and platform configurations, and discusses realistic development timeframes. Best practices for tuning feedback control systems and validating designs before field deployment are covered, acknowledging the limitations of simulation-only approaches.
Workshop Objectives
The central objective is to equip participants with the theoretical knowledge and practical skills needed to develop automated software packages for low-cost FPV copters. Specific objectives include:
- Understanding the operational context and mission requirements for low-cost FPV drones in real-world scenarios.
- Evaluating hardware platforms (SoCs, NPUs) and their suitability for real-time embedded AI workloads.
- Applying classical computer vision algorithms and hybrid CV/DL pipelines on computationally constrained hardware.
- Using simulation tools to prototype, validate, and iterate on drone designs and mission software.
- Engaging critically with ethical considerations surrounding autonomous targeting, human-in-the-loop control thresholds, and proportionality in mission automation.
Key Topics
- FPV drone design, flight control software, and communication protocols
- Mission taxonomy and operational challenges derived from conflict-zone feedback
- SoC and NPU accelerator platforms for edge AI (Raspberry Pi + Hailo, Qualcomm Dragonwing, Toradex)
- Real-time versus best-effort embedded system design and Rate Monotonic Analysis
- Linux-based asymmetric multiprocessing for resource-constrained systems
- Classical computer vision: Optical Flow, MOSSE, KCF, CSRT trackers
- Fusion of classical CV methods with lightweight deep learning models
- Thermal imaging considerations for night missions
- Simulation environments for drone design and mission validation
- Ethical dimensions of autonomous UAV operations: targeting, proportionality, and human oversight
Workshop Format
The workshop will include a series of detailed presentations with multimedia modules.
Target Audience
This workshop is suitable for:
- Graduate students conducting research in UAV systems, embedded AI, or computer vision
- Scientists and engineers working on autonomous systems and edge computing
- Developers and manufacturers of small UAVs
- Defence and security researchers interested in low-cost drone automation
- Policy researchers and ethics scholars engaged with autonomous weapons governance
No prior experience with FPV drones is required, though familiarity with basic computer vision and embedded systems concepts is beneficial.
Tentative Outcome
By the end of the workshop, participants will be able to:
- Identify and categorize automated missions for low-cost FPV drones and their associated technical constraints.
- Select appropriate hardware platforms and evaluate their real-time processing capabilities using RMA.
- Implement and benchmark classical computer vision trackers on resource-constrained SoCs.
- Design hybrid CV/DL pipelines that balance accuracy and computational cost.
- Use simulation tools to prototype and validate mission software before field deployment.
Material
Participants will receive:
- Comprehensive lecture notes and presentation slides
- A curated set of research papers and technical references
- Links to video tutorials and demonstration recordings
- Links to simulation environments and configuration files used in Part 4