Automize LogoFloLogixAI
Back to Blog

Real-Time Drone Detection and Tracking Using YOLO11

Real-Time Drone Detection and Tracking Using YOLO11

Project Overview

This project aimed to develop a real-time drone detection and tracking system using the YOLO11 deep learning framework. By combining object detection and tracking capabilities, the system effectively identifies and follows drones in images, videos, and live camera feeds. This dual functionality is particularly valuable for military, surveillance, and monitoring applications.

The primary objectives included:

  • Training a custom YOLO11 model on a robust dataset.
  • Implementing tracking functionality using DeepSORT for maintaining object continuity.
  • Optimizing performance using GPU acceleration with NVIDIA GTX 1660 Super and CUDA.
  • Testing the trained model and tracker on video data and real-time camera feeds.

Execution Process

1. Dataset Preparation

  • Dataset Source: The dataset was sourced from Roboflow and consisted of high-quality drone images annotated for object detection.
  • Dataset Structure:
    • Split into training, validation, and testing sets.
    • Organized into images and labels folders, adhering to YOLO's expected format.
  • Annotation Format: Bounding boxes with class IDs for drones, following the YOLO format.

2. Model Selection and Training

  • Model Used: YOLO11 Nano (yolov11n.pt), chosen for its lightweight architecture, balancing accuracy and speed.
  • Training Configuration:
    • Hardware: NVIDIA GTX 1660 Super with CUDA and cuDNN enabled.
    • Epochs: 50
    • Batch Size: Configured based on GPU memory.
    • Optimizer: Default optimizer and learning rate scheduler provided by YOLO11.

3. Integration with Object Tracking

  • Tracking Algorithm: DeepSORT was integrated with YOLO11 to enable tracking functionality.
    • Features Used: Bounding box coordinates, object class, and confidence scores from YOLO11 outputs.
    • Tracker: DeepSORT’s Kalman Filter and Hungarian Algorithm ensured robust tracking.
  • Implementation Steps:
    • Process YOLO11 detections as inputs for DeepSORT.
    • Assign unique IDs to detected objects and maintain continuity across frames.

4. Testing and Evaluation

  • Validation Metrics: The detection model was evaluated on validation and test datasets using metrics like mAP (Mean Average Precision) and precision-recall curves.
  • Video Testing: The combined detection and tracking system was tested on drone footage to identify and track drones frame-by-frame.
  • Live Camera Testing: The system was deployed to process live video feeds from a webcam, demonstrating real-time detection and tracking.

Results

Detection Performance

The YOLO11 Nano model exhibited consistent improvements in both training and validation metrics over 50 epochs:

  • Losses: Figure 1: Training Results
    • Training and validation losses (box, class, DFL) steadily decreased, indicating convergence (see Figure 1: Training Results).
  • Performance Metrics:
    • mAP50: Exceeded 90%, demonstrating high precision in detecting drones.
    • mAP50-95: Reached approximately 70%, showcasing the model's ability to generalize across IoU thresholds.
    • Precision: 0.97
    • Recall: 0.9

Confusion Matrix

The confusion matrix (see Figure 2) highlights:

  • High True Positive Rate: The model accurately classified drones with a 97% true positive rate.
  • Low False Positives: Minimal confusion between the drone and background classes.

Figure 2: Confusion Matrix

Tracking Performance

  • Accuracy: The DeepSORT integration maintained over 95% accuracy in tracking drones across video frames.
  • ID Switching: Fewer than 2% ID switches occurred, ensuring reliable tracking.
  • Real-Time Performance: The system achieved ~18 FPS for detection and tracking on the NVIDIA GTX 1660 Super.

Challenges and Solutions

Challenge 1: Dataset Formatting

  • Problem: Ensuring the dataset was correctly structured and annotated for YOLO.
  • Solution: Used Roboflow to export the dataset in YOLO format and verified the directory structure before training.

Challenge 2: Dependency Issues

  • Problem: Installing dependencies like PyTorch, CUDA, and cuDNN caused conflicts.
  • Solution: Installed precompiled binaries for PyTorch and CUDA, matched to the GPU, and used Python 3.9 for compatibility.

Challenge 3: Real-Time Performance

  • Problem: Achieving real-time performance on the NVIDIA GTX 1660 Super.
  • Solution:
    • Used the lightweight YOLO11 Nano model.
    • Reduced input resolution to 640x640 for faster inference.
    • Leveraged GPU acceleration with CUDA and cuDNN.

Instructions for Running the Trained Model

If you have the trained model (best.pt) and want to test it on a video or camera feed:

Test on a Video

Use the following script to test the model on a video file:

import os
import cv2
from ultralytics import YOLO

# Define paths
ROOT_DIR = os.path.dirname(__file__) # Current script's directory
video_path = os.path.join(ROOT_DIR, 'drone2.mp4')
video_path_out = os.path.join(ROOT_DIR, 'drone2_out.mp4')

# Open the video
cap = cv2.VideoCapture(video_path)

if not cap.isOpened():
print(f"Error: Could not open video file {video_path}")
exit()

ret, frame = cap.read()
if not ret or frame is None:
print(f"Error: Could not read the video file {video_path}")
exit()

H, W, _ = frame.shape
out = cv2.VideoWriter(video_path_out, cv2.VideoWriter_fourcc(*'MP4V'), int(cap.get(cv2.CAP_PROP_FPS)), (W, H))

# Load YOLO model
model = YOLO(os.path.join(ROOT_DIR, 'runs', 'detect', 'train3', 'weights', 'best.pt'))

threshold = 0.5

while ret:
results = model.predict(frame, conf=threshold)

for result in results[0].boxes.data.tolist():
x1, y1, x2, y2, score, class_id = result

if score > threshold:
# Draw bounding boxes
cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 4)
cv2.putText(frame, model.names[int(class_id)].upper(), (int(x1), int(y1 - 10)),
cv2.FONT_HERSHEY_SIMPLEX, 1.3, (0, 255, 0), 3, cv2.LINE_AA)

out.write(frame)
ret, frame = cap.read()

# Release resources
cap.release()
out.release()
cv2.destroyAllWindows()

print(f"Video with predictions saved to {video_path_out}")

Test on a Camera

Use the following script to test the model on a live camera feed:

import cv2
from ultralytics import YOLO

# Load the YOLO model
model = YOLO("runs/detect/train3/weights/best.pt") # Replace with the path to your trained model

# Open the default camera (use 0 for the default webcam)
camera_index = 0 # Change to 1, 2, etc., if you have multiple cameras
cap = cv2.VideoCapture(camera_index)

if not cap.isOpened():
print(f"Error: Could not access the camera at index {camera_index}")
exit()

# Set up camera properties (optional)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
cap.set(cv2.CAP_PROP_FPS, 30)

threshold = 0.5 # Confidence threshold for detection

print("Starting camera feed... Press 'q' to quit.")

while True:
ret, frame = cap.read()
if not ret:
print("Failed to grab frame from camera. Exiting...")
break

# Perform inference with YOLO
results = model.predict(frame, conf=threshold)

# Draw the results on the frame
for result in results[0].boxes.data.tolist():
x1, y1, x2, y2, score, class_id = result

if score > threshold:
# Draw bounding box
cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)
# Add label text
cv2.putText(frame, model.names[int(class_id)].upper(), (int(x1), int(y1) - 10),
cv2.FONT_HERSHEY_SIMPLEX, 0.9, (0, 255, 0), 2)

# Show the frame
cv2.imshow("YOLO Real-Time Detection", frame)

# Exit if 'q' is pressed
if cv2.waitKey(1) & 0xFF == ord('q'):
break

# Release resources
cap.release()
cv2.destroyAllWindows()

print("Camera feed stopped.")

Future Work

  1. Enhanced Detection:
    • Train on larger, more diverse datasets to improve robustness.
    • Experiment with larger YOLO11 models (e.g., YOLO11m) for higher accuracy.
  2. Edge Deployment:
    • Optimize the model for edge devices like NVIDIA Jetson for field applications.
  3. Multiclass Detection and Tracking:
    • Extend the system to detect and track other aerial objects or classify drones into subcategories.
  4. User Interface:
    • Build a user-friendly interface for live detection, tracking, and result visualization.

Conclusion

This project successfully implemented a high-performing, real-time drone detection and tracking system using YOLO11 Nano and DeepSORT. By leveraging annotated datasets and GPU acceleration, the system demonstrated strong accuracy and reliability, showcasing its potential for surveillance and monitoring applications.