A Vehicular-Edge Federated, Quantized YOLOv12 System for Real-Time 3D Hand-Gesture–Based UAV Control
This repository is actively maintained. Check the GitHub Releases for the latest updates, and consider starring the repository to stay informed!
DroneWave is an open-source project that enables intuitive control of unmanned aerial vehicles (UAVs) through human gestures, using advanced object detection and privacy-preserving federated learning. Built around the UAV-Gesture dataset and the YOLOv12 model, this project provides a clear pipeline to acquire, annotate, and process gesture data, then train an efficient, quantized model optimized for resource-constrained devices. Using the Flower framework, it supports distributed training across multiple devices while keeping data private. Designed for researchers, students, and developers, DroneWave offers a reproducible framework for exploring UAV navigation, computer vision, and distributed machine learning.
We also leverage Microsoft AirSim to simulate UAV flight and gesture-based control scenarios, allowing safe, flexible experimentation before real-world deployment.
The project is divided into three phases:
- Data Retrieval and Auto-Annotation: Acquiring the UAV-Gesture dataset and generating initial annotations using YOLOv12.
- Data Processing and Manual Correction: Uploading data to Roboflow for manual quality checking, cleaning, augmentation, and exporting in YOLOv12-compatible format.
- Federated Learning Setup: Partitioning data across clients and training a federated, quantized model using Flower.
- Gesture Simulation in AirSim: Using Microsoft AirSim to simulate UAVs, validate gesture-based control in a virtual environment, and bridge the gap between dataset-driven training and real-world UAV applications
Each phase is accompanied by a dedicated README file with detailed instructions, scripts, and dependencies.
The repository is organized into directories corresponding to each phase of the project, ensuring modularity and ease of navigation. Below is the directory structure:
DroneWave/
├── data/
│ ├── raw/
│ └── annotations/
├── scripts/
│ ├── data_retrieval/
│ │ ├── request_dataset.py
│ │ ├── require_dataset.py
│ │ ├── annotate_dataset.py
│ │ └── README.md ← (Phase 1 instructions)
│ └── data_processing/
│ ├── upload_to_roboflow.py
│ └── README.md ← (Phase 2 instructions)
├── federated-learning/
│ ├── utils
│ │ ├── load_model.sh
│ │ └── load_dataset.py
│ ├── federated_learning/
│ │ ├── client_app.py
│ │ ├── server_app.py
│ │ ├── task.py
│ │ └── __init__.py
│ ├── pyproject.toml
│ └── README.md ← (Phase 3 instructions)
├── simulation/
│ └── README.md ← (Phase 4 instructions)
├── environments/
│ ├── requirements.txt
│ ├── setup_env.sh
│ └── setup_env.bat
└── README.md ← (Main overview)
-
data/: Stores dataset files at different stages.
- raw/: Contains the unzipped UAV-Gesture dataset (videos).
- annotations/: Holds extracted frames, processed images, and annotations.
-
scripts/: Contains Python scripts for early phases, organized by task.
- data_retrieval/: Scripts for dataset acquisition, extraction, and auto-annotation with YOLOv12.
- data_processing/: Script for preparing data (uploading to Roboflow, cleaning, augmenting, and exporting in YOLOv12 format).
-
federated-learning/: Contains the full federated learning setup.
- utils/load_model.sh, utils/load_dataset.py: Utilities for loading models and datasets.
- federated_learning/: Flower apps for simulating federated learning (client, server, task management).
- pyproject.toml: Python project configuration (dependencies, build system).
-
environments/: Environment setup scripts and requirements.
- requirements.txt: Python dependencies.
- setup_env.sh / setup_env.bat: Scripts to create and configure the Python environment (Linux/Mac & Windows).
-
README.md: Main project overview and quickstart guide.
To run this project, ensure you have:
- Python 3.10 or higher
- A Roboflow account for dataset management
- A machine with a GPU (recommended for YOLOv12 training)
- Git for cloning the repository
- Basic familiarity with command-line interfaces and Python scripting
-
Clone the Repository:
git clone https://github.com/AnarchyNinetyNine/DroneWave.git cd DroneWave -
Set Up the Virtual Environment:
bash environments/setup_env.sh
(Windows users can run environments/setup_env.bat instead)
-
Follow Phase-Specific Instructions:
Contributions are always welcome! To contribute:
- Fork the repository
- Create a feature branch
- Submit a Pull Request (PR)
Please follow Conventional Commits for commit messages.
If you use this project or the UAV-Gesture dataset in your research, please cite the original dataset paper:
Perera, A. G., Law, Y. W., & Chahl, J. (2019). UAV-GESTURE: A Dataset for UAV Control and Gesture Recognition. Paper link
Additionally, please cite this repository:
[Lamaakal, I. Elgarrab, I. Alouach, A]. et al. (2025). A Vehicular-Edge Federated, Quantized YOLOv12 System for Real-Time 3D Hand-Gesture–Based UAV Control. GitHub Repository
For questions or contributions, please open an issue on GitHub or contact [ismail.lamaakal@ieee.org].