Abstract
Modern image signal processors (ISPs) provide crucial features to allow computer vision and AI subsystems to exploit their full potential in whichever environmental condition, taking care of conditioning the physical parameters of CMOS image sensors as well as constructing complex tone mapping for standard and HDR imaging. ISPs can easily eat hundreds of GFLOPs of computing power, and are usually implemented as dedicated logic circuitry in mobile phones and digital cameras.
The talk will focus on describing the journey of porting ISPido - a fully fledged ISP pipeline of algorithms - to a full Rust implementation, capable to leverage on CPU and GPU capabilities to be portable and effective on a large family of processors, including those powering low-power embedded devices. Portability is a key factor when dealing with embedded devices, to limit the number of platforms to be actively maintained down to the bone, as normally this family of algorithms require a low-level access to the computational resources. Last but not least, image processing is a memory intensive process, and many algorithms can require complex memory allocation strategies, all-but-linear access patterns and counterintuitive implementation to avoid poor cacheing performance.
By using Rust as primary language and WGPU as primary crate for accessing GPU resources, the developers of ISPido can show how to efficiently use fragment and compute shader with no major complexity in the system architecture, and the smallest loss of readability in the resulting code. WGSL shaders with their Rust-y syntax help the overall system homogeneity, and provide a familiar environment for implementing the low-level pixel processing portions of the pipeline. Smart threading functions in Rust (std::thread and a few helper crates, like rayon and crossbeam) make managing parallel workload immediate and natural.
Comparative benchmarks vs. C++ and Scala implementations will be shown, both in terms of relative speed of calculation and overall code readability and maintainability.
