Intel/heatmap-generation

Heatmap Generation

Property	Value
Category	Object Detection + Speed Heatmap Aggregation
Base Model	YOLO26 (Ultralytics)
Source Framework	PyTorch (Ultralytics)
Supported Precisions	FP32, FP16, INT8 (mixed-precision)
Inference Engine	OpenVINO
Hardware	CPU, GPU, NPU
Detected Class(es)	All 80 COCO classes (heatmap colored by object speed)

---

Overview

Heatmap Generation is a Metro Analytics use case that detects objects across video frames and colors each region of the scene by how fast traffic moves through it. It is built on YOLO26, a state-of-the-art real-time object detector, quantized to INT8 for efficient inference on Intel hardware.

Each detection's per-frame displacement is used as a speed estimate, deposited over the object's footprint and averaged per location with Gaussian smoothing into a color-coded overlay.

The overlay uses the following color scheme:

• Red -- fast-moving traffic.
• Yellow / green -- medium speed.
• Blue -- slow-moving or stationary traffic.

Typical Metro deployments include:

• Traffic Speed Mapping -- highlight fast corridors and slow/congested lanes.
• Congestion Detection -- surface persistently slow (blue) areas for safety planning.
• Pedestrian Flow Analysis -- compare fast throughways against lingering areas.
• Incident Spotting -- flag unusually fast or stalled movement.

Available variants: yolo26n, yolo26s, yolo26m, yolo26l, yolo26x. Smaller variants (yolo26n, yolo26s) are recommended for high-FPS edge deployment; larger variants improve recall for small or distant objects.

---

Prerequisites

• Python 3.11+
• Install OpenVINO (latest version)
• Install Intel DLStreamer (latest version)

Create and activate a Python virtual environment before running the scripts:

python3 -m venv .venv --system-site-packages
source .venv/bin/activate

Note: The --system-site-packages flag is required so the virtual environment can access the system-installed OpenVINO and DLStreamer Python packages.

---

Getting Started

Download and Quantize Model

Run the provided script to download, export to OpenVINO IR, and optionally quantize:

chmod +x export_and_quantize.sh
./export_and_quantize.sh

This exports the default yolo26n model in FP16 precision.

Optional: Select a Different Variant or Precision

./export_and_quantize.sh yolo26n FP32   # full-precision
./export_and_quantize.sh yolo26n INT8   # quantized
./export_and_quantize.sh yolo26s        # larger variant, default FP16

The script performs the following steps:

1. Installs dependencies (openvino, ultralytics; adds nncf for INT8).
2. Downloads a sample test image (test.jpg) and a sample test video (test_video.mp4).
3. Downloads the PyTorch weights and exports to OpenVINO IR.
4. (INT8 only) Quantizes the model using NNCF post-training quantization.

Output files:

• yolo26n_openvino_model/ -- FP32 or FP16 OpenVINO IR model directory.
• yolo26n_heatmap_int8.xml / .bin -- INT8 quantized model (only when INT8 is selected).

Precision / Device Compatibility

Precision	CPU	GPU	NPU
FP32	Yes	Yes	No
FP16	Yes	Yes	Yes
INT8	Yes	Yes	Yes

OpenVINO Sample

The sample below runs YOLO26 inference on a video, estimates each object's speed from its per-frame displacement, and writes a speed-colored heatmap overlay (red = fast, blue = slow) to output_openvino.mp4. It also saves the final speed heatmap as heatmap.jpg. Change the device string to run on CPU, GPU, or NPU.

import cv2
import numpy as np
import openvino as ov

CONF_THRESHOLD = 0.4
INPUT_SIZE = 640
HEATMAP_ALPHA = 0.55
MATCH_DIST = 80.0   # max px between frames to treat detections as the same object
MAX_SPEED = 20.0    # px/frame that maps to full red


def render_speed_heatmap(frame, speed_sum, count, alpha):
    """Color traffic by average speed: blue = slow, yellow = medium,
    red = fast. Only regions where vehicles were seen are tinted, so
    empty background keeps its original color."""
    avg = np.zeros_like(speed_sum)
    seen = count > 0
    avg[seen] = speed_sum[seen] / count[seen]
    avg = cv2.GaussianBlur(avg, (0, 0), sigmaX=15)
    presence = cv2.GaussianBlur(seen.astype(np.float32), (0, 0), sigmaX=15)
    norm = np.clip(avg / MAX_SPEED, 0, 1)  # 0 = slow (blue), 1 = fast (red)
    color = cv2.applyColorMap((norm * 255).astype(np.uint8), cv2.COLORMAP_JET)
    weight = (np.clip(presence, 0, 1) * alpha)[..., np.newaxis]
    overlay = frame.astype(np.float32) * (1 - weight) + color.astype(np.float32) * weight
    return overlay.astype(np.uint8), color


core = ov.Core()
model = core.read_model("yolo26n_openvino_model/yolo26n.xml")

# Change device to "GPU" or "NPU" to run on integrated GPU or NPU.
compiled = core.compile_model(model, "CPU")

cap = cv2.VideoCapture("test_video.mp4")
fps = cap.get(cv2.CAP_PROP_FPS) or 30.0
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
writer = cv2.VideoWriter(
    "output_openvino.mp4", cv2.VideoWriter_fourcc(*"mp4v"), fps, (width, height))

speed_sum = np.zeros((height, width), dtype=np.float32)
count = np.zeros((height, width), dtype=np.float32)
prev_centroids = []
heatmap_color = None
frame_idx = 0
total_dets = 0

while True:
    ok, frame = cap.read()
    if not ok:
        break
    frame_idx += 1
    h0, w0 = frame.shape[:2]
    sx, sy = w0 / INPUT_SIZE, h0 / INPUT_SIZE

    blob = cv2.resize(frame, (INPUT_SIZE, INPUT_SIZE))
    blob = cv2.cvtColor(blob, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
    blob = blob.transpose(2, 0, 1)[np.newaxis, ...]

    output = compiled([blob])[compiled.output(0)][0]
    dets = output[output[:, 4] >= CONF_THRESHOLD]
    total_dets += len(dets)

    cur_centroids = []
    for det in dets:
        x1, y1 = int(det[0] * sx), int(det[1] * sy)
        x2, y2 = int(det[2] * sx), int(det[3] * sy)
        x1, x2 = max(0, x1), min(width, x2)
        y1, y2 = max(0, y1), min(height, y2)
        cx, cy = (x1 + x2) / 2, (y1 + y2) / 2
        cur_centroids.append((cx, cy))

        # Speed = displacement from the nearest detection in the previous frame.
        speed = 0.0
        if prev_centroids:
            d = min(np.hypot(cx - px, cy - py) for px, py in prev_centroids)
            if d <= MATCH_DIST:
                speed = d
        speed_sum[y1:y2, x1:x2] += speed
        count[y1:y2, x1:x2] += 1.0
    prev_centroids = cur_centroids

    overlay, heatmap_color = render_speed_heatmap(frame, speed_sum, count, HEATMAP_ALPHA)
    cv2.putText(overlay, f"Detections: {len(dets)}", (10, 30),
                cv2.FONT_HERSHEY_SIMPLEX, 0.8, (255, 255, 255), 2)
    writer.write(overlay)

cap.release()
writer.release()

if heatmap_color is not None:
    cv2.imwrite("heatmap.jpg", heatmap_color)
    print("Saved: heatmap.jpg")

print(f"Processed {frame_idx} frames, {total_dets} total detections", flush=True)

Device targets:

• "CPU" -- default, works on all Intel platforms.
• "GPU" -- Intel integrated or discrete GPU.
• "NPU" -- Intel NPU (validate with benchmark_app -d NPU).

Expected Output

DLStreamer Sample

The pipeline below runs the FP16 YOLO26 detector via gvadetect. A buffer probe estimates each object's speed from its per-frame displacement and overlays a speed-colored heatmap (red = fast, blue = slow) on each frame before encoding to output_dlstreamer.mp4.

Notes on running this sample:

• Use the FP16 IR (yolo26n_openvino_model/yolo26n.xml). Class names are read automatically from the model's embedded metadata.yaml by DLStreamer 2026.0+ -- no external labels-file is required.

• Export PYTHONPATH so the DLStreamer Python module is importable:

  source /opt/intel/openvino_2026/setupvars.sh
  source /opt/intel/dlstreamer/scripts/setup_dls_env.sh
  export PYTHONPATH=/opt/intel/dlstreamer/python:\
  /opt/intel/dlstreamer/gstreamer/lib/python3/dist-packages:${PYTHONPATH:-}
  

import gi

gi.require_version("Gst", "1.0")
gi.require_version("GstAnalytics", "1.0")
from gi.repository import Gst, GLib, GstAnalytics

import numpy as np

Gst.init([])

# Import cv2 after Gst.init to avoid GStreamer re-initialization conflicts.
import cv2

INPUT_VIDEO = "test_video.mp4"
HEATMAP_ALPHA = 0.55
MATCH_DIST = 80.0   # max px between frames to treat detections as the same object
MAX_SPEED = 20.0    # px/frame that maps to full red


def render_speed_heatmap(frame, speed_sum, count, alpha):
    """Color traffic by average speed: blue = slow, yellow = medium,
    red = fast. Only regions where vehicles were seen are tinted, so
    empty background keeps its original color."""
    avg = np.zeros_like(speed_sum)
    seen = count > 0
    avg[seen] = speed_sum[seen] / count[seen]
    avg = cv2.GaussianBlur(avg, (0, 0), sigmaX=15)
    presence = cv2.GaussianBlur(seen.astype(np.float32), (0, 0), sigmaX=15)
    norm = np.clip(avg / MAX_SPEED, 0, 1)  # 0 = slow (blue), 1 = fast (red)
    color = cv2.applyColorMap((norm * 255).astype(np.uint8), cv2.COLORMAP_JET)
    weight = (np.clip(presence, 0, 1) * alpha)[..., np.newaxis]
    overlay = frame.astype(np.float32) * (1 - weight) + color.astype(np.float32) * weight
    return overlay.astype(np.uint8)


# For CPU: change device=GPU to device=CPU.
# For NPU: change device=GPU to device=NPU (batch-size=1, nireq=4 recommended).
pipeline_str = (
    f"filesrc location={INPUT_VIDEO} ! decodebin3 ! "
    "videoconvert ! video/x-raw,format=BGR ! "
    "gvadetect model=yolo26n_openvino_model/yolo26n.xml "
    "device=GPU "
    "threshold=0.4 ! queue ! "
    "appsink name=sink emit-signals=false sync=false"
)
pipeline = Gst.parse_launch(pipeline_str)
sink = pipeline.get_by_name("sink")
pipeline.set_state(Gst.State.PLAYING)

speed_sum = None
count = None
prev_centroids = []
writer = None
frame_idx = 0
total_dets = 0

while True:
    sample = sink.emit("pull-sample")
    if sample is None:
        break
    buf = sample.get_buffer()
    caps = sample.get_caps().get_structure(0)
    width = caps.get_value("width")
    height = caps.get_value("height")

    if speed_sum is None:
        speed_sum = np.zeros((height, width), dtype=np.float32)
        count = np.zeros((height, width), dtype=np.float32)

    ok, mapinfo = buf.map(Gst.MapFlags.READ)
    if not ok:
        continue
    frame = np.ndarray((height, width, 3), dtype=np.uint8,
                       buffer=mapinfo.data).copy()
    buf.unmap(mapinfo)
    frame_idx += 1

    rmeta = GstAnalytics.buffer_get_analytics_relation_meta(buf)
    cur_centroids = []
    det_count = 0
    if rmeta is not None:
        idx = 1
        while True:
            ok_od, od = rmeta.get_od_mtd(idx)
            if not ok_od:
                break
            _, x, y, w, h, _ = od.get_location()
            x1, y1 = max(0, int(x)), max(0, int(y))
            x2, y2 = min(width, int(x + w)), min(height, int(y + h))
            cx, cy = (x1 + x2) / 2, (y1 + y2) / 2
            cur_centroids.append((cx, cy))

            # Speed = displacement from the nearest detection last frame.
            speed = 0.0
            if prev_centroids:
                d = min(np.hypot(cx - px, cy - py) for px, py in prev_centroids)
                if d <= MATCH_DIST:
                    speed = d
            speed_sum[y1:y2, x1:x2] += speed
            count[y1:y2, x1:x2] += 1.0
            det_count += 1
            idx += 1
    prev_centroids = cur_centroids
    total_dets += det_count

    overlay = render_speed_heatmap(frame, speed_sum, count, HEATMAP_ALPHA)

    if writer is None:
        writer = cv2.VideoWriter(
            "output_dlstreamer.mp4", cv2.VideoWriter_fourcc(*"mp4v"),
            30.0, (width, height))
    writer.write(overlay)
    print(f"Frame {frame_idx}: detections={det_count}", flush=True)

pipeline.set_state(Gst.State.NULL)
if writer:
    writer.release()
print(f"Processed {frame_idx} frames, {total_dets} total detections", flush=True)

Device targets:

• device=GPU -- default in the sample code.
• device=CPU -- change device=GPU to device=CPU.
• device=NPU -- change device=GPU to device=NPU; use batch-size=1 and nireq=4 for best NPU utilization.

Expected Output

---

License

Licensed under the MIT License. See LICENSE for details.

References

• YOLO26 Documentation
• Ultralytics Heatmap Guide
• OpenCV Color Maps
• OpenVINO Documentation
• Intel DLStreamer