Introduction
The Multi-Camera Composition System addresses the challenge of rendering real-time video composition from 8 camera capture inputs using a low-power embedded platform. These inputs come from diverse sources such as RTSP streams and V4L2 devices, with a strict requirement for high performance and minimal resource usage under constraints like limited processing power, varied input formats, real-time responsiveness, and power efficiency.
Key Features
- Real-Time Multi-Stream Composition: Efficient handling of 8 concurrent camera streams.
- Support for Multiple Input Sources: RTSP streams and V4L2 camera devices.
- Hardware-Accelerated Processing: Utilization of embedded hardware decoders for optimal performance.
- Low Level Optimization: ARM NEON instruction based accelerated video processing with parallel ops, fast format conversion, efficient scaling, and optimized chroma/alpha handling.
- GPU-Based Rendering: Efficient video composition using OpenGL ES.
- Modular Pipeline Design: GStreamer-based implementation for flexibility and scalability.
- High Resource Efficiency: Optimized use of CPU and memory bandwidth.
Technical Highlights
- Zero-Copy Media Pipeline: Avoids unnecessary memory transfers between pipeline stages, enhancing throughput.
- Hardware-Accelerated Decoding: Leverages platform-specific video units (e.g., i.MX VPU, Xilinx VCU, NVIDIA HW Codecs).
- OpenGL ES Rendering: GPU-based video composition for high-performance output.
- GStreamer Integration: Comprehensive media pipeline built with GStreamer for robust media handling.
- Performance Boost: Achieved 70% improvement in processing efficiency compared to baseline implementations. Optimized Neon vectorization delivers lower CPU usage.
- Real-Time Operation: Maintained stable frame rates across all streams while significantly reducing resource usage.
Supported Use Cases
- Surveillance Systems
- Smart Traffic Monitoring
- Embedded Vision Applications
- Drone and UAV Video Aggregation
- Custom Streaming Module
- Dynamic Camera Switching Application
Architecture Overview
The system handles real-time composition of up to 8 concurrent camera streams through dedicated media pipelines, using embedded hardware decoders and GPU-based OpenGL ES rendering.
ARM NEON SIMD optimizations process multiple data elements simultaneously in 128-bit registers, delivering 2-16x performance improvements in pixel manipulation, colour space conversion, and filtering operations.
Services offered
| Service offered | Description |
| Media Processing | Efficient ingestion and decoding of multiple video streams using hardware acceleration. Optimized media pipeline with zero-copy architecture for minimal latency and resource use. |
| Composition and Rendering | Real-time video composition of multiple streams using GPU-based OpenGL ES rendering. Supports flexible layout configurations for combined video output. |
| System Optimization and Integration | Performance-tuned design using GStreamer for modular, real-time media applications. Minimizes CPU and power consumption while enabling seamless integration on embedded platforms. |