Streaming image processing with Rust: an embedded developer perspective

Image and AI processing require low-level, close to metal and black magic arts to result performant in embedded systems, with low chances of portability and safety. This argument can be subverted by using Rust and WGPU for implementing a portable ISP camera stack, portable, safe, readable and fast.


Modern image signal processors (ISPs) provide crucial features to allow computer vision and AI subsystems to exploit their full potential in whichever environmental condition, taking care of conditioning the physical parameters of CMOS image sensors as well as constructing complex tone mapping for standard and HDR imaging. ISPs can easily eat hundreds of GFLOPs of computing power, and are usually implemented as dedicated logic circuitry in mobile phones and digital cameras.

The talk will focus on describing the journey of porting ISPido - a fully fledged ISP pipeline of algorithms - to a full Rust implementation, capable to leverage on CPU and GPU capabilities to be portable and effective on a large family of processors, including those powering low-power embedded devices. Portability is a key factor when dealing with embedded devices, to limit the number of platforms to be actively maintained down to the bone, as normally this family of algorithms require a low-level access to the computational resources. Last but not least, image processing is a memory intensive process, and many algorithms can require complex memory allocation strategies, all-but-linear access patterns and counterintuitive implementation to avoid poor cacheing performance.

By using Rust as primary language and WGPU as primary crate for accessing GPU resources, the developers of ISPido can show how to efficiently use fragment and compute shader with no major complexity in the system architecture, and the smallest loss of readability in the resulting code. WGSL shaders with their Rust-y syntax help the overall system homogeneity, and provide a familiar environment for implementing the low-level pixel processing portions of the pipeline. Smart threading functions in Rust (std::thread and a few helper crates, like rayon and crossbeam) make managing parallel workload immediate and natural.

Comparative benchmarks vs. C++ and Scala implementations will be shown, both in terms of relative speed of calculation and overall code readability and maintainability.

RustLab is a conference made by Develer.
Develer is a company based in Campi Bisenzio, near Florence. Our motto is : "Technology to give life to your products". We produce hardware and software to create exceptional products and to improve industrial processes and people's well being.
In Develer we have passion for the new technologies and we offer our clients effective solutions that are also efficient, simple and safe for the end users. We also believe in a friendly and welcoming environment where anybody can give their contribution. This passion and this vision are what we've been driven to organize our conference "made by developers for developers".

Subscribe to our newsletter

We hate spam just as much as you do, which is why we promise to only send you relevant communications. We respect your privacy and will never share your information with third parties.
©2024 RustLab | The international conference on Rust in Florence-Design & devCantiere Creativo-Made withDatoCMS