NVIDIA DeepStream Edge Video Analytics Setup

Imagine: 200 RTSP streams from cameras, but a CPU server chokes on 10 streams. DeepStream on Jetson Orin solves this—hardware decoding and GPU inference deliver 32 streams on a single board. But configuring the pipeline requires understanding GStreamer and TensorRT. We configure video analytics pipelines based on NVIDIA DeepStream for edge devices. The DeepStream SDK is a GStreamer-based framework with hardware acceleration: RTSP decoding, object detection, tracking, and metadata publication to Kafka—all on Jetson or GPU. On Jetson Orin we process 16+ HD streams in real time, which is 10 times more than similar setups using OpenCV and Intel Xeon. Our team has 10+ years of edge AI experience and 30+ successful deployments. As a certified NVIDIA partner since 2018, we have delivered 30+ edge video analytics projects.

Architecture of a DeepStream Pipeline

GStreamer plugins from DeepStream form a linear pipeline:

[RTSP/File/USB] → nvv4l2decoder → nvstreammux → nvinfer → nvtracker
                                                    ↓
[Kafka/MQTT/File] ← nvmsgbroker ← nvmsgconv ← nvdsosd ← nvinfer (secondary)

nvv4l2decoder: Hardware decode (H.264/H.265/AV1) via Jetson VPU. Zero-copy into GPU memory—no CPU→GPU transfer.

nvstreammux: Multiplexes N streams into a batch. batch-size=8 allows one inference call for 8 frames simultaneously.

nvinfer: TensorRT engine inside. Supports detector (YOLO, SSD), classifier, segmentor. Configuration via .txt file—model engine, batch size, precision.

nvtracker: Multi-object tracking. Algorithms: IOU, NvSORT, NvDeepSORT (with ReID), NvDCF (correlation filter). DeepSORT uses a Re-ID network to maintain IDs through occlusions.

nvmsgconv + nvmsgbroker: Converts metadata (bbox/trackID/class) to JSON and publishes to Kafka, MQTT, Azure IoT Hub, AWS IoT.

Why DeepStream on Edge Outperforms CPU Solutions

A comparison with OpenCV on CPU shows: the GPU-accelerated framework is better than OpenCV by 10 times in stream processing capacity. On Jetson Orin AGX, it handles 32 HD streams with YOLOv8n FP16 at 30 FPS, while OpenCV on Intel Xeon manages 2–3 streams. The reason is hardware decoding, zero-copy, and TensorRT on GPU. Savings on cloud computing can reach 3–5 times. For example, switching to Jetson can save $500–$2000 per month in cloud costs. A typical deployment on Jetson Orin NX costs $3,000 and saves $1,500 per month in cloud costs. DeepStream reduces hardware costs by up to 80% (e.g., from $15,000 to $3,000 per node).

Configuration for Specific Tasks

[primary-gie]
model-engine-file=yolov8n.engine
batch-size=16
interval=0          # every frame
network-type=0      # detector

[tracker]
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/lib/libnvds_nvmultiobjecttracker.so
ll-config-file=nvdcf_tracking.yml

Production quality control (1–2 high-resolution cameras): batch-size=1, interval=0, primary detector → secondary classifier (defect/normal). High resolution: tile-based inference via nvdspreprocess with overlapping tiles.

People counting in zones: nvinfer → nvtracker → nvdsanalytics. nvdsanalytics provides ROI counting, line crossing detection, direction detection. All configured without writing code.

How to Integrate Custom YOLO Models

1. Export the model to ONNX: yolo export model=yolov8n.pt format=onnx
2. Convert ONNX to TensorRT engine: trtexec --onnx=yolov8n.onnx --saveEngine=yolov8n.engine --fp16
3. Write a custom parser (C++) to parse the output tensor into NvDsInferObjectDetectionInfo.
4. Connect the engine and parser in the DeepStream config.

If you'd rather not write a parser from scratch, we use ready-made solutions: Ultralytics GitHub for direct export to TensorRT or DeepStream-Yolo with ready parsers for different YOLO versions.

Example custom parser:

extern "C" bool NvDsInferParseCustomYoloV8(
    std::vector<NvDsInferLayerInfo> const& outputLayersInfo,
    NvDsInferNetworkInfo const& networkInfo,
    NvDsInferParseDetectionParams const& detectionParams,
    std::vector<NvDsInferObjectDetectionInfo>& objectList) {
    // parse tensor → NvDsInferObjectDetectionInfo
}

Python Bindings (pyds)

For custom logic in probe callbacks:

def osd_sink_pad_buffer_probe(pad, info, u_data):
    gst_buffer = info.get_buffer()
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))

    for frame_meta in pyds.NvDsFrameMetaList(batch_meta.frame_meta_list):
        for obj_meta in pyds.NvDsObjectMetaList(frame_meta.obj_meta_list):
            if obj_meta.class_id == PERSON_CLASS:
                # custom logic: filtering, alerts, recording
                obj_meta.rect_params.border_color.set(1.0, 0, 0, 1.0)

A probe on any pipeline pad gives full access to metadata without stopping the stream.

Deployment on Jetson

JetPack: The latest version includes all dependencies. Install with apt install deepstream-7.0. Our certified NVIDIA engineers set up the environment.

TensorRT engine generation:

trtexec --onnx=yolov8n.onnx \
        --saveEngine=yolov8n.engine \
        --fp16 \
        --workspace=2048 \
        --minShapes=input:1x3x640x640 \
        --optShapes=input:8x3x640x640 \
        --maxShapes=input:16x3x640x640

The engine is generated on a specific device—it is not portable between different Jetson SKUs.

Containerization: nvcr.io/nvidia/deepstream:7.0-gc-triton-devel. Docker on Jetson with --runtime nvidia. Orchestration via docker-compose or K3s.

Scaling and Monitoring

Metrics: DeepStream Prometheus exporter—FPS per source, inference time, tracker update time, drop frame ratio. Grafana dashboard.

Multi-node: Kafka as transport for metadata between nodes. Each Jetson is a producer. A central server is a consumer plus aggregation.

Remote management: DeepStream App Framework with REST API (ds-server): add/remove RTSP sources without restarting the pipeline.

Performance Comparison

The Jetson platform delivers 10 times more streams at significantly lower cost. Implementing it can reduce server hardware expenses by up to 80%. Jetson Orin also offers 50% better efficiency than GPU-based x86 solutions. With over 10 years in edge AI and 30+ successful video analytics deployments, our team is a trusted NVIDIA partner.

What's Included

• Audit of current infrastructure and requirements.
• Pipeline design: model selection, parameters, integrations.
• Model customization (fine-tuning, quantization INT8/FP16).
• Development of custom parsers and probe callbacks.
• Integration with VMS (Milestone, Genetec) and external systems (Kafka, MQTT).
• Deliverables: Documentation (pipeline config, API docs), source code and Docker images, access to NVIDIA certified engineers.
• Training: 2-day workshop for operators.
• Support: 6 months of stable operation guarantee.

Investment in DeepStream pays off in 6–12 months by eliminating expensive CPU servers and reducing cloud traffic.

Timelines: 4–8 Weeks

Basic configuration with a ready-made model: 1–2 weeks. Custom parsers, VMS integration, complex business logic: 6–8 weeks.

Contact us—we'll assess your project in 2 days. Get a consultation from an NVIDIA certified engineer. GStreamer

Edge AI and Optimization: How to Deploy Models Without Cloud?

Imagine: your face recognition model has 4 seconds latency on Jetson Orin, the battery runs out in an hour, and the model crashes with OOM. We are a team of Edge AI engineers with 5+ years in production — we have optimized over 150 models for edge devices. Without profiling and proper choice of quantization or distillation, the project is doomed. The gap between research code and edge deployment is a separate engineering discipline; we help you master it in 2–16 weeks turnkey. Edge AI and model optimization services are not just export, but systematic work with hardware.

Why Simply Exporting a Model Doesn't Work?

A PyTorch model with float32 and batch_size=32 is not ready for edge. Typical problems:

• ResNet-50 in fp32 occupies 98 MB, inference on Cortex-A78 — 380 ms. After INT8 quantization via torch.ao.quantization — 24 MB, 95 ms. Export to ONNX + TensorRT on Jetson — 28 ms.
• YOLOv8m on Raspberry Pi 5 in fp32 — 2.8 fps. TFLite INT8 — 9.4 fps. With XNNPACK delegate — 14 fps (1.5× faster than pure INT8).
• Transformer encoder on mobile CPU: MobileBERT in fp16 via CoreML on iPhone 15 — 18 ms/inference. distilbert-base-uncased in ONNX — 42 ms.

The problem is not choosing "quantize or not" — the right path is determined by the device, task, and acceptable metric degradation. We offer an assessment of your project: within 24 hours we will tell you how feasible it is to speed up the model.

How to Choose Quantization Method for Your Task?

PTQ (Post-Training Quantization) — a quick path. Take a trained model, run a calibration dataset (200–1000 samples), get INT8 or INT4 weights. Tools: torch.ao.quantization, ONNX Runtime quantization tool, bitsandbytes. Accuracy degradation: 0.5–2% on classification. Red zone — small object detection and segmentation, where PTQ gives -4–8% mAP.

QAT (Quantization-Aware Training) — training with simulated quantization noise. More expensive (retraining), but degradation 0.1–0.5%. Justified when PTQ is unacceptable. In PyTorch — torch.ao.quantization.prepare_qat().

GPTQ / AWQ — for LLMs. AWQ better preserves quality at 4-bit quantization. llm-compressor from Neural Magic or autoawq are the main libraries.

Potential savings from choosing the right method can be substantial — for example, reducing cloud inference costs by up to 70% when deploying to edge. Project cost is calculated individually based on model complexity and target platform.

When to Use Pruning vs Distillation?

Structural pruning removes channels or layers. torch.nn.utils.prune — basic tool. For transformers — attention head pruning (LTP, movement pruning). Result: ResNet-50 after removing 40% of channels with fine-tuning — -35% size, -28% latency, -1.2% top-1 accuracy.

Knowledge distillation — train a small student to mimic a large teacher. Classic via KLDivLoss on soft labels. Feature distillation on intermediate layers is more effective. Hugging Face DistilBERT: 66M vs 110M parameters, -40% latency, -3% on GLUE. This is a model compression technique.

Combined approach: distillation → pruning → QAT. Gives maximum effect on limited hardware. We recorded a case where a client achieved 70% reduction in cloud compute spend after moving to edge with this pipeline.

Target Platforms and Tools

TensorRT — the main tool for NVIDIA edge. TRT builds a graph with operator fusion, selects optimal kernels. On Jetson AGX Orin YOLOv8m in TRT INT8 gives 78 fps vs 22 fps in fp16 PyTorch — 3.5× improvement.

Practical Case: How We Detected Defects on a Production Line (Our Client)

Task: real-time scratch detection on metal, 30 fps, camera to Jetson Xavier NX (16GB). Original model YOLOv8l mAP50 0.91, server inference 28 ms, on Jetson in fp16 — 110 ms (9 fps). Not suitable.

Optimization steps we performed for our client:

1. Switch to YOLOv8m — mAP50 0.887 (-2.3%), 68 ms
2. Export to TensorRT FP16 via yolo export format=engine half=True — 31 ms (32 fps)
3. INT8 calibration on 500 frames — 22 ms (45 fps), mAP50 0.879

Result: 3.5% degradation at 5× speedup. Client received engine and documentation. We guarantee metric will not drop below agreed threshold — specified in contract.

What is Included in the Work?

• Report on model profiling on target device (layer latency, bottlenecks)
• Selection and justification of optimization methods (quantization / pruning / distillation)
• Optimized model (TensorRT engine / TFLite / CoreML / ONNX)
• Configs for reproducibility (scripts, Docker image, instructions)
• Testing on real device (at least 10,000 inferences)
• Training of your team (2 hours online)
• 1 month support after delivery

How to Order Model Optimization

1. Submit a request on the website or contact us in any convenient way.
2. We perform free profiling of your model on the target device within 24 hours.
3. We prepare an optimization plan with trade-off estimates (speed vs quality).
4. You approve the plan — we start work.
5. After completion, we deliver the optimized model, configs, and documentation.
6. We train your team and provide monthly support.

Timeline: optimization of an existing model — 2–4 weeks. Development from scratch for edge — 6–16 weeks.

Get a consultation — we will evaluate your model for free and offer a plan within 24 hours. Order free profiling now. For complex projects, contact our engineering team to discuss custom optimisation strategies.