YOLO: Complete Guide — Real-Time Object Detection

Summary

YOLO (You Only Look Once) is one of the most popular and fastest object detection algorithms ever designed. First proposed by Joseph Redmon and his collaborators in 2015, YOLO revolutionized the field of computer vision by reformulating object detection as a single regression problem, rather than as a succession of distinct steps. The result? A neural network capable of simultaneously detecting multiple objects in an image, in real time, with remarkable accuracy.

Since its initial version, the YOLO algorithm has undergone numerous iterations — from YOLOv1 to YOLOv11 — each bringing significant improvements in accuracy, speed, and versatility. Today, YOLO is used in fields as diverse as autonomous driving, video surveillance, industrial robotics, precision agriculture, and medical imaging analysis. This complete guide explores in depth the mathematical principle of YOLO, its underlying intuition, its practical implementation with Ultralytics, as well as its advantages, limitations, and concrete use cases.

Mathematical Principle of YOLO

The S×S Detection Grid

The core of YOLO’s principle relies on an elegant and powerful idea: the input image is divided into a grid of size S × S. Each cell in this grid is responsible for detecting objects whose center falls within its boundaries. For example, if the image is divided into a 7 × 7 grid, each of these 49 cells examines its area and attempts to predict the presence of objects.

For each cell, YOLO predicts B bounding boxes. Each bounding box is characterized by five values:

• x: horizontal coordinate of the box center, normalized relative to the cell width.
• y: vertical coordinate of the box center, normalized relative to the cell height.
• w: width of the box, normalized relative to the total image width.
• h: height of the box, normalized relative to the total image height.
• Confidence score: a value indicating how sure the model is that an object is present in this box.

This confidence score reflects two aspects simultaneously: the probability that an object is actually present in the box, and the quality of the prediction compared to the ground truth. Mathematically, it is expressed as:

confidence score = P(object) × IoU(pred, ground truth)

Intersection over Union (IoU)

IoU (Intersection over Union) is a fundamental measure in object detection. It quantifies the overlap between the box predicted by the model and the manually annotated box (ground truth). Formally:

IoU = area(intersection) / area(union)

An IoU of 1 means the prediction is perfect (total overlap), while an IoU of 0 means there is no overlap. In practice, a prediction is generally considered valid if the IoU exceeds a threshold of 0.5.

Class Prediction

In addition to bounding boxes, each cell also predicts conditional probabilities for C object classes. For example, in the COCO dataset, there are 80 different classes (person, car, cat, dog, etc.). These probabilities are shared among all boxes predicted by the same cell, meaning each cell predicts only one type of object, even if it proposes multiple boxes.

Non-Maximum Suppression (NMS)

After the network has generated all its predictions, a problem naturally arises: many boxes overlap for the same object. This is where Non-Maximum Suppression (NMS) comes in, a crucial post-processing step.

The principle of NMS is as follows:

1. For each class, sort all detected boxes by decreasing confidence score.
2. Select the box with the highest score.
3. Remove all other boxes whose IoU with the selected box exceeds a defined threshold (usually 0.5).
4. Repeat the process until there are no more boxes to process.

This method ensures that a single object is detected only once, eliminating unwanted duplicates.

Anchor Boxes

Later versions of YOLO (starting from YOLOv2) introduced anchor boxes (or prior boxes). Rather than predicting absolute box dimensions, the model predicts offsets relative to predefined reference shapes. These shapes are determined automatically by clustering (usually k-means) on the training set annotations.

Using anchor boxes offers two major advantages:

• Better generalization for objects of varying sizes and proportions.
• Easier learning: the network only needs to predict relative adjustments rather than absolute coordinates, which stabilizes convergence.

Loss Function

The YOLO loss function combines several components to guide the network’s learning. It is broken down into four main terms:

Loss = λ_coord × L_coord + L_conf_obj + L_conf_noobj + L_class

• L_coord (coordinate loss): penalizes errors in the coordinates (x, y, w, h) of boxes that actually contain an object. For dimensions w and h, the square root is used to reduce the impact of errors on large boxes relative to small ones.
• L_conf_obj (confidence loss): penalizes confidence errors for boxes containing an object.
• L_conf_noobj (negative confidence loss): penalizes overly high confidences for boxes containing no object. This term is weighted by a factor λ_noobj (usually 0.5) to compensate for the imbalance between cells with and without objects.
• L_class (class loss): penalizes classification errors for cells containing an object.

This formulation allows the model to simultaneously learn to precisely localize objects, correctly estimate prediction confidence, and classify detected objects into the right category.

Intuition: Why YOLO is Revolutionary

To understand the power of YOLO, you need to compare it to the methods that existed before its invention. Traditional approaches, such as R-CNNs (Regions with CNN Features), worked in multiple distinct stages. First, a region proposal algorithm (like Selective Search) generated about 2,000 candidate regions in the image. Then, a convolutional neural network extracted features from each region individually. Finally, an SVM classifier determined the class of each region. This approach was extremely slow since the network had to process each region separately, making real time practically impossible.

YOLO radically changes this paradigm: instead of searching for objects in multiple places like the old methods, YOLO only looks at the image once — hence its name, “You Only Look Once.” It’s comparable to the difference between a professional photographer and a beginner:

• The beginner scans area by area, examines every corner of the scene methodically, and takes a long time to compose their shot.
• The professional, on the other hand, captures the entire scene in an instant, intuitively perceives where the important elements are, and captures the perfect moment.

In the same way, YOLO looks at the image in its entirety, understands the scene globally, and detects all objects simultaneously. This unified approach gives YOLO several decisive advantages:

• Exceptional speed: a single pass through the network is enough to detect all objects. This enables frame rates of 30 to 144 FPS depending on the version used.
• Contextual understanding: by processing the entire image, YOLO reduces background detection errors, unlike region-based methods that treat each area in isolation.
• Improved generalization: YOLO generalizes better to new domains because it learns global representations rather than local features specific to a dataset.

Python Implementation with Ultralytics YOLOv8

The Ultralytics library offers a modern and user-friendly implementation of the latest versions of YOLO, notably YOLOv8. Here’s how to use it concretely.

Installation

pip install ultralytics

Detection on an Image

from ultralytics import YOLO

# Load the pretrained YOLOv8n model (nano - the lightest)
model = YOLO("yolov8n.pt")

# Run detection on an image
results = model("path/to/image.jpg")

# Display results with bounding boxes
results[0].show()

# Save annotated results
results[0].save(filename="detected_image.jpg")

Detection on a Video

from ultralytics import YOLO

# Load the YOLOv8s model (good speed/accuracy trade-off)
model = YOLO("yolov8s.pt")

# Run detection on a video
results = model(
    source="path/to/video.mp4",
    show=True,
    save=True,
    conf=0.25,
    iou=0.45
)

# Results are automatically saved in the runs/detect/ folder

Object Counting

from ultralytics import YOLO

model = YOLO("yolov8n.pt")
results = model("street_photo.jpg")

# Iterate through results and count objects per class
result = results[0]
class_names = result.names

counts = {}
for box in result.boxes:
    class_id = int(box.cls[0])
    class_name = class_names[class_id]
    counts[class_name] = counts.get(class_name, 0) + 1

print("--- Detected Object Counts ---")
for cls, count in counts.items():
    print(f"{cls}: {count}")

Real-Time Object Tracking in a Video

from ultralytics import YOLO

# Load the YOLOv8 model with COCO pretraining
model = YOLO("yolov8n.pt")

# Launch tracking on a video sequence
results = model.track(
    source="camera_stream.mp4",
    show=True,
    tracker="bytetrack.yaml",
    persist=True,
    conf=0.3,
    iou=0.5
)

# Each tracked object retains a unique identifier across frames

Custom Training on Your Own Data

from ultralytics import YOLO

# Create a model from scratch (YOLOv8m - medium)
model = YOLO("yolov8m.pt")

# Train on a custom dataset
model.train(
    data="my_dataset.yaml",
    epochs=100,
    imgsz=640,
    batch=16,
    name="my_yolo_training"
)

# Evaluate the trained model
metrics = model.val()
print(f"mAP@50: {metrics.box.map50:.4f}")
print(f"mAP@50-95: {metrics.box.map:.4f}")

Key Hyperparameters of YOLO

To get the best performance with YOLO, it is essential to understand and properly adjust the following hyperparameters:

imgsz (image size)

Determines the dimension to which images are resized before being processed by the model. The default value is 640 pixels.

• Common values: 320, 416, 512, 640, 1280
• Impact: a larger size improves detection of small objects but significantly slows down inference. Conversely, a smaller size speeds up processing but may miss small objects.

conf_threshold (confidence threshold)

Sets the minimum confidence level required for a detection to be retained. The default value is 0.25.

• Low value (0.1-0.2): more detections, but more false positives.
• High value (0.5-0.8): fewer detections, but greater precision.

iou_threshold (IoU threshold for NMS)

Determines the Intersection over Union threshold used by Non-Maximum Suppression to eliminate redundant boxes. The default value is 0.45.

• Low value (0.3-0.4): more aggressively eliminates overlapping boxes, useful when objects are far apart.
• High value (0.6-0.7): retains more boxes, useful in very dense scenes where multiple objects touch.

device (execution hardware)

Specifies the hardware used for inference or training.

• “cpu”: uses the processor (slower, but universally available).
• “0” or “cuda:0”: uses the first NVIDIA GPU (recommended for speed).
• “mps”: uses the Apple Silicon GPU on recent Macs.
• [0, 1]: uses multiple GPUs in parallel for distributed training.

batch_size

Determines the number of images processed simultaneously during training.

• Small batch (8-16): suitable for GPUs with limited VRAM, but convergence may be less stable.
• Large batch (32-64): training acceleration through better parallelism, but requires more GPU memory.
• batch=-1: lets Ultralytics automatically determine the optimal size based on available memory.

Advantages and Limitations of YOLO

Advantages

1. Unmatched real-time speed: YOLOv8 and later versions easily achieve over 100 FPS on a modern GPU, making them ideal for applications requiring minimal latency such as robotics or live video surveillance.
2. Unified and simple pipeline: unlike multi-stage methods, YOLO requires a single model and a single pass, greatly simplifying production deployment.
3. Excellent accuracy/speed trade-off: the different model sizes (nano, small, medium, large, xlarge) allow adapting YOLO to the desired balance between precision and speed.
4. Contextual robustness: by looking at the entire image, YOLO is less likely to confuse background textures with actual objects.
5. Mature ecosystem: with Ultralytics, the YOLO community benefits from ready-to-use training, export (ONNX, TensorRT, CoreML), tracking, and segmentation tools.

Limitations

1. Difficulty with very small objects: YOLO still struggles to detect objects occupying less than 1% of the image, as grid quantization limits spatial resolution in distant areas.
2. Sensitivity to very close objects: when multiple objects of the same class heavily overlap, YOLO may only detect one due to Non-Maximum Suppression.
3. Geometric bias of anchor boxes: since anchor boxes are determined by clustering on the training set, the model may underperform on images with unusual proportions not represented during training.
4. Data requirements: like all deep learning models, YOLO requires a large and well-annotated training dataset to achieve optimal performance. Manually annotating thousands of images can represent a considerable investment.
5. Less precise localization than two-stage methods: YOLO sacrifices some precision in the exact localization of bounding boxes in favor of speed, which can be problematic for applications requiring pixel-level accuracy.

4 Concrete Use Cases of YOLO

1. Autonomous Driving and ADAS Systems

In autonomous vehicles, YOLO is used to detect pedestrians, other vehicles, traffic signs, traffic lights, and road obstacles in real time. Inference speed is critical: at 120 km/h, a vehicle travels 33 meters per second. Every fraction of a second counts to avoid a collision. YOLOv8, with its ability to process over 100 frames per second on an embedded GPU, provides the responsiveness needed for emergency driving decisions.

2. Intelligent Video Surveillance

Modern surveillance systems use YOLO to automatically analyze video feeds from hundreds of cameras simultaneously. Applications include detecting intrusions in restricted areas, counting people in public spaces, identifying suspicious behavior (abandoned luggage, unusual movements), and crowd flow management during events. Unlike traditional systems that require constant human monitoring, YOLO automates real-time alerting, significantly reducing the need for operators.

3. Precision Agriculture

In modern agriculture, YOLO is deployed on drones and robots to monitor crops. Concrete applications include automatic weed detection for targeted, pesticide-reduced weeding, fruit counting and tracking on trees to estimate yields, early identification of foliar diseases through visual leaf analysis, and livestock classification in extensive farming. This approach enables farmers to intervene in a precise and localized manner.

4. Medical Imaging Analysis

YOLO also finds promising applications in medical imaging. It is used to automatically detect anomalies in X-rays (fractures, pulmonary nodules), localize tumors in MRI and CT scan images, identify abnormal cells in microscopic smears, and assist surgeons during procedures by identifying anatomical structures in real time. Although these applications require rigorous validation before clinical deployment, initial results are very encouraging and demonstrate the remarkable versatility of the YOLO algorithm.

See Also

• Optimize Your Python: Maximize the Product of an Integer Partition in Python
• Cryptocurrency According to Artificial Intelligence