GStreamer – Video Pipelines for phyCAM & Vision Kits

With our Ezpire module for GStreamer, you receive pre-configured video pipelines for all supported phyCAM modules within the Vision BSPs. The demo scripts for live streaming, JPG/RAW capture, and more are integrated directly into the phytec-vision-image and ready to use immediately. Use them as a rock-solid foundation for your own image processing or video applications.

What is GStreamer?

GStreamer, is an open-source multimedia framework for Linux, perfectly suited for embedded video applications. It allows the creation of flexible pipelines for capturing, processing, encoding, and outputting video and audio data.

In the PHYTEC context, GStreamer uses the  V4L2 (Video4Linux2) interface to control phyCAM cameras. phytec-vision-images contain pre-configured pipelines for all supported phyCAM modules (VM-x16, VM-x17, VM-x20, VM-x24, etc.) alongside detailed, easy-to-read scripts.

GStreamer out-of-the-box

We take the complexity out of camera integration. GStreamer is already fully embedded into our phytec-vision-images (Vision BSPs) and optimized across all our Video Kits. You receive a collection of ready-to-run demo scripts covering the most essential video use cases:

  • Live Video Streaming: Output the camera signal directly and smoothly to a connected display (e.g., via HDMI).

  • Image and Video Capture: Seamlessly save images as compressed JPGs or in lossless RAW format at full sensor resolution.

  • Sensor-Specific Optimization: We provide tailored configurations for our various phyCAM modules. The scripts automatically detect the connected camera type (color/monochrome) and adjust parameters optimally.

  • Hardware acceleration (ISP):  Configuration files are required to utilize an available Image Signal Processor (ISP). Selected board-level cameras from our phyCAM series are pre-calibrated for specific processor boards. This allows you to leverage fundamental ISP features right away, including Demosaic, Denoise/Sharpen Filters, AEC (Auto Exposure Control) AWB (Auto White Balance) BLC (Black Level Correction) Defect Pixel Cluster Correction and WDR3 (Wide Dynamic Range).

  • Multi-Camera Setups: Out-of-the-box support for synchronous dual-camera operation, ideal for complex machine vision scenarios. For processors featuring "virtual channel" technology, we offer dedicated support, enabling multiple cameras to connect via a single MIPI CSI-2 channel. Simple configuration scripts let you pre-set both the camera and the processor's camera interface. The custom-configured image/stream is then exposed to the user as a standard VXNUMXLXNUMX device.

Simple configuration scripts allow for presetting in the camera and the processor's camera interface. The user then has access to the image/stream configured according to their wishes as a V4L2 device.

Technical Deep Dive

Our Vision Images utilize V4L2 subdevices combined with media-ctl for dynamic camera configuration (routing and format negotiation). GStreamer then fetches the image data directly from the video device via V4L2. Each of our phyCAM modules is mapped to a dedicated video device. The included shell scripts, located in the /gstreamer-examples/ directory of the root partition, cleanly abstract the complexity of these calls.

  • Live streaming (Wayland/Framebuffer): When executing a script like cam-fbdev_1280x720.sh, the system first calls the underlying configuration script and sets specific camera properties via v4l2-ctl. Finally, it constructs a GStreamer pipeline that outputs the V4L2 video stream to the display using an appropriate sink (e.g., waylandsink or directly to the framebuffer), completely optimized for minimal CPU load.

  • Raw and JPG Capture: Scripts like cam-save_jpg_full_res.sh trigger pipelines that capture the video feed, compress it via jpegenc, and save it using filesink. Conversely, cam-save_raw_full_res.sh dumps the raw, unprocessed sensor data directly to the file system.

A look under the hood reveals how GStreamer fully utilizes the SoC hardware. For a VM-016 module (AR0144), an efficient H.264 encoding pipeline looks like this:
 
bash
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-bayer,format=grbg,width=1280,height=800 ! bayer2rgbneon ! queue ! vpuenc_h264 ! rtspclientsink

Here, the raw Bayer data is converted using the highly optimized bayer2rgbneon plugin (leveraging the ARM NEON instruction set). The converted image is then fed into the NXP SoC's dedicated hardware video encoder (vpuenc_h264), freeing up valuable CPU resources for the actual application.

The V4L2 nodes are set up modularly. Cameras are loaded during boot via U-Boot overlays defined in bootenv.txt (e.g., overlays=imx8mp-isi-csi1.dtbo imx8mp-vm016-csi1.dtbo).
If an ISP is available, we provide advanced pipelines in directories marked with "isp". In this case, the V4L2 stream is not accessed directly but routed through the NXP chip's hardware ISP. GStreamer then taps into /dev/video[X], which provides fully processed YUV/RGB images where features like Auto-Exposure, Auto-White-Balance, and Debayering have already been applied in hardware with zero latency.

Recommended Hardware

To evaluate our System on Modules and phyCAM modules in practice, we offer tailored Embedded Vision Development Kits These bundles contain everything you need to get started: a System on Module (SoM), baseboard, camera, lens, and a pre-installed Vision BSP including all GStreamer examples.

For demanding video applications, we particularly recommend:

Our reference platform for embedded vision — featuring a dual ISP and a dedicated AI accelerator (NPU) for high-performance multi-camera setups.

The next generation of architecture, equipped with massive AI compute power and the latest MIPI-CSI interfaces for high-resolution streaming.

You can find additional vision setups for other SoCs and form factors on our Imaging Kit Overview Page.

The PHYTEC phyCAM Portfolio
The GStreamer integration unlocks its full potential when paired with our extensive range of industrial-grade camera modules (phyCAM series).. Whether you need a rolling shutter or global shutter, monochrome or color, from 1 megapixel up to 4K resolutions — our hardware and software drivers are perfectly matched.

Which setup best suits your requirements? Thanks to the standardized phyCAM interface, camera modules can be easily interchanged. 
Reach out to us – we are happy to help you select the right hardware.

Getting Started

Get started with GStreamer and your phyCAM in just a few steps.

All GStreamer demo scripts and the required camera drivers are already integrated into our dedicated phytec-vision-image Depending on the hardware you are using, getting started works as follows:

If you are using a PHYTEC Imaging Kit:
If you purchased one of our Vision Kits (e.g., the phyBOARD-Pollux Imaging Kit), you don't need to do anything else. The phytec-vision-image comes pre-installed from the factory. Just boot up and start right away.

If you are using a Standard Kit or Single Board Computer (SBC):
If you are using a regular Development Kit or a standard board, you can easily flash the Vision Image yourself. Simply visit the product page of your module (e.g., i.MX 8M Plus) and navigate to the DownloadsThere, you will find the latest phytec-vision-image ready to be downloaded and flashed onto an SD card.

Where can I find the GStreamer scripts and instructions?
Once the Vision image is running on your hardware, use our Getting Started Guide, to start the GStreamer pipelines.

  • Find the Documentation: Go to your module's product page and navigate to  Downloads > Image Processing > Quick Start Guides with camerasDownload the "Getting Started Guide phyCAM".

  • Run the GStreamer Demos: In a dedicated section of these documents, you will find a detailed overview and instructions for all the pre-configured GStreamer demo scripts (such as live streaming, image capture, and hardware-accelerated encoding).