For the past few weeks, I have been focusing most of my time at work on a project, which we have coined the spark. In its essence, it is a data gateway built on top of a Raspberry Pi, which bridges machine and energy metrics from existing infrastructure to our data pipeline.
As our main focus was set on high frequency read-outs at up to 100Hz, it was quickly sorted out that python, usually my go-to language, wasn’t the way to go. After some research, I quickly settled on Rust due to its cutting edge performance and ‘if i compiles, it’s safe’ mantra.
But as always, everything comes with a price: While Rust is blazingly fast and enables us to develop and ship new features and fixes with confidence, it needs to be compiled for a particular architecture — in our case armv7. Working on a MacbookPro, I initially tackled this problem by pulling the entire repo and building the application on the Raspberry Pi itself, which resulted in up to 30 minutes of impatience.
In search of a more streamlined build process, I ended up implementing a cross-compilation pipeline built on top of Docker, which I would like to share with you.
What’s In It For You?
This post is designed to be a step-by-step manual for you to build your own automated cross-compilation pipeline:
- First, we will set up a small Rust program, which we will later cross-compile for armv7.
- Second, we will create a custom Docker image, which will handle the heavy lifting of cross-compiling system dependencies as well as our application.
- Finally, we will write a shell script to execute the cross-compilation process.
Now that we’ve set our scope, let’s get ready for take off!
Preparing our Rust Application
As this post mainly focuses on automating the cross-compilation of Rust applications, we will set up a bare scaffold and add a system dependency, which I found particularly tricky: OpenSSL.
We start by creating a new directory named hello-rpi and cding into it. Then, we initialize a new cargo project by running cargo new --bin hello-rpi in our terminal of choice. Now, we should have a project directory called hello-rpi with the following folder structure:
hello-rpi||--- hello-rpi* |--- Cargo.toml||--- src* |--- main.rs*
With our minimal application generated, add openssl = “0.10.5” as a dependency in our Cargo.toml file and declare openssl as an external crate in main.rs. After our small adjustments, our Cargo.toml and main.rs should something like this:
Cargo.toml
[package]name = "hello-rpi"version = "0.1.0"authors = ["your handle <your@mail.com>"]
[dependencies]openssl = "0.10.5"
main.rs
extern crate openssl;
fn main() {println!("Hello, world!");}
Finally, go ahead and add the following lines to your crate’s cargo config file found in hello-rpi/hello-rpi/.cargo/config :
[target.armv7-unknown-linux-gnueabihf]linker = "arm-linux-gnueabihf-gcc"
This will tell cargo which linker to use, when we specify armv7-unkown-linux-gnueabih as our target, which is our Pi’s architecture. With these few lines of code, we’re done on the Rust end.
Creating our Custom Docker Image
With our minimal application set and ready to go, we now focus on automating the cross compilation process of our application. To this end, we will create a Docker image, which will handle cross-compiling our application as well as our dependency, i.e. OpenSSL for us.
Fortunately, we don’t have to start from scratch — Stephen Thirlwall’s Docker image raspberry-pi-cross-compiler, which may be found here, provides an awesome boilerplate to build up on. When run without any arguments, a container based on this image returns a helper script, which in turn let’s you interact with the container and start the actual cross-compilation.
So let’s dive right in! Create a new folder in our project’s (not crate!) root named rpxc, create a Dockerfile and paste the following into it:
Wow, there’s a lot going on here — let’s break it down a bit: First, we point our base image to sdthirwall’s raspberry-pi-cross-compiler and install some packages via apt-get — nothing out of the ordinary.
We then use the ENV expression to set environment variables pointing to our Cargo and Rustup installations as well as our OpenSSL lib and include directories.
The subsequent RUN block is borrowed from the rust:1.26.0 image and deals with installing the Rust toolchain and adding armv7-unknown-linux-gnueabihf as a compilation target. As indicated by its name, this is the compiler for our Raspberry’s architecture.
The second RUN command then downloads, un-tars and cross-compiles OpenSSL for our target architecture. Note that our environment variables pointing to OpenSSL’s lib and include directory match up with our install’s path.
Not that complicated, right? What’s more, this setup facilitates including any additional system dependency in no time by simply extending our Dockerfile — it doesn’t get easier than that.
Automating The Build Process
With our Dockerfile set up, we’re meremly one step short of our build pipeline. More precisely, we want to
- Build our Docker image we just defined
- Start a container to obtain the cross-compilation script I mentioned above
- Use the script to start the cross-compilation process
Transforming this procedure into a shell script and adding some clean up yields six actual lines of code, which need to be placed in the rpxc directory to work correctly.
And that’s it! We have defined custom Docker image, which handles cross-compiling system dependencies at build time and written a shell script to actually build and cross-compile the entire application. Now, you can simply cd into rpxc, run the script above, sit back and watch the magic happening.
Summary
In this post, we built a build pipeline, which let’s us cross-compile any Rust application for the armv7 architecture by simply executing a single shell script. To this end, we built a custom Dockerfile, added application specific system dependencies and wrote a small shell script, which kicks off the cross-compilation process.
This setup is favourable for multiple reasons: First and foremost, you can build your application on any machine and platform, as long as Docker is installed. Thus, you can easily integrate the entire build process as the final step of your continuous integration pipeline. What’s more, you can add any additional system dependency by extending the Dockerfile, which makes this solution scalable towards more complex applications, which may depend on a large number of system dependencies.
That’s it from my side! I hope you’ve enjoyed my first post and that I was able to get you kick started with cross-compiling Rust for Raspberry Pis. If you have any questions or critique regarding my post, let me know in the comments below!
Full disclosure: I am one of the co-founders of Enlyze. We are a startup based in Aachen, Germany, which treats power consumption as a universal indicator for any appliance’s state of health. Our team of six consists of passionate AI researchers and electrial as well as software engineers. If you’d like to get in touch, shoot us a mail to hello@enlyze.com.