Streaming image processing with Rust: an embedded developer perspective

Image and AI processing require low-level, close to metal and black magic arts to result performant in embedded systems, with low chances of portability and safety. This argument can be subverted by using Rust and WGPU for implementing a portable ISP camera stack, portable, safe, readable and fast.

Place
RustLab
Length
60 min
When
November 21st, 2023
14:30

Abstract

Modern image signal processors (ISPs) provide crucial features to allow computer vision and AI subsystems to exploit their full potential in whichever environmental condition, taking care of conditioning the physical parameters of CMOS image sensors as well as constructing complex tone mapping for standard and HDR imaging. ISPs can easily eat hundreds of GFLOPs of computing power, and are usually implemented as dedicated logic circuitry in mobile phones and digital cameras.

The talk will focus on describing the journey of porting ISPido - a fully fledged ISP pipeline of algorithms - to a full Rust implementation, capable to leverage on CPU and GPU capabilities to be portable and effective on a large family of processors, including those powering low-power embedded devices. Portability is a key factor when dealing with embedded devices, to limit the number of platforms to be actively maintained down to the bone, as normally this family of algorithms require a low-level access to the computational resources. Last but not least, image processing is a memory intensive process, and many algorithms can require complex memory allocation strategies, all-but-linear access patterns and counterintuitive implementation to avoid poor cacheing performance.

By using Rust as primary language and WGPU as primary crate for accessing GPU resources, the developers of ISPido can show how to efficiently use fragment and compute shader with no major complexity in the system architecture, and the smallest loss of readability in the resulting code. WGSL shaders with their Rust-y syntax help the overall system homogeneity, and provide a familiar environment for implementing the low-level pixel processing portions of the pipeline. Smart threading functions in Rust (std::thread and a few helper crates, like rayon and crossbeam) make managing parallel workload immediate and natural.

Comparative benchmarks vs. C++ and Scala implementations will be shown, both in terms of relative speed of calculation and overall code readability and maintainability.

Stay connected

You can subscribe to our newsletter to stay updated on the latest news and announcements. Follow us on social media to join our community and engage with other attendees, speakers, and staff. We're on Twitter, Mastodon, Youtube and Facebook. Don't forget to use our official hashtag #RustLab2023 to share your experience and connect with other attendees on social media.

Subscribe to our newsletter

We hate spam just as much as you do, which is why we promise to only send you relevant communications. We respect your privacy and will never share your information with third parties.
©2023 RustLab | The international conference on Rust in Florence-Design & devCantiere Creativo-Made withDatoCMS