3.00 PM - 3.30 PM
Aula Magna

Pixelweave Chromascale: an open source multi-platform library for efficient video frame conversions on the GPU

Jose Aguerre @ Evercast

Across video streaming, editing, and conferencing workflows, handling high-quality video demands significant computational overhead and memory consumption. Working with a diverse set of video sources (cameras, capture devices, video files, etc) requires dealing with multiple pixel formats such as RGB and YCbCr, with varying chroma subsampling. Color depths may be represented in 8, 10, 12 or 16 bits, but not all devices support high-color. As a result, most video applications rely on per-frame conversions in order to unify the different sources and process/render video in real time. The state-of-the-art libraries which offer this are mostly CPU-based: libswscale, used by the industry standard solution ffmpeg; libyuv, from Chromium’s WebRTC implementation; and zimg, a project-independent library that uses vectorized operations to improve performance. Being CPU-based, these solutions do not take advantage of modern GPU computing, which would enable higher performance by relieving congestion on the primary processing unit of the system.

This paper presents the open-source project Pixelweave Chromascale, a multi-platform library that leverages GPU computing to perform video frame conversions. Pixelweave is based on the multi-platform graphics library Vulkan, allowing for a single implementation of GPU kernels that is compatible with most modern graphics hardware. The library supports multiple pixel formats such as RGBA, planar YUV with 420, 422, and 444 chroma subsampling, YV12, NV12 and other interleaved formats, using up to 16 bits per channel. The implemented compute shaders use a novel generic approach that samples 2x2 pixel windows, making the rest of the code easily extensible and independent of the input’s chroma-subsampling. Initial tests show that Pixelweave is an order of magnitude faster than the state-of-the-art libraries on integrated GPUs, such as on the Apple Ms processors. It also performs faster on high-end discrete GPUs, allowing the CPU to offload resource-demanding tasks.

Jose Aguerre is a computer scientist from Montevideo, Uruguay. He holds a bachelor’s degree in Computer Engineering from the University of the Republic, and a PhD in Computer Graphics from the Université de Pau, France. He has over 15 peer-reviewed journal and conference papers, as well as 10+ years of experience in software development. Currently Director of Engineering and R&D at Evercast, Jose leads research projects related to video conferencing/streaming, real-time graphics, and GPU computing.