Look, Ma! No JS! - Compiling Rust to WebAssembly

Why Rust and WebAssembly?

I have recently been on a journey to learn Rust, after only hearing good things about it. I worked through the excellent Rust book and needed to apply my knowledge to really make it sink in. I have also been curious about WebAssembly for some time now and, since Rust has very mature tooling for compiling to WebAssembly, it felt only natural to combine the two. I created a small example app that runs in the browser with basically no JavaScript, and want to share what I learned about Rust’s WebAssembly toolchain in the process, especially wasm-bindgen.

Why should I read this?

Rust’s WebAssembly book is excellent, but a fairly long read and relies heavily on templates to get you up and running quickly. This is definitely a good didactic approach, but hides the finer details of how everything works, which left me unsatisfied. You can dive into the wasm-bindgen guide or the wasm-pack book (both of them are very good), but they are, again, very verbose and far from a quick read. With this post, I want to create a more practical and concise(-ish) introduction to the world of Rust and WebAssembly, focusing on what’s going on behind the scenes. Basically, I want this to be the guide, that I wish I had, when I started learning about this stuff.

There are many interesting things to talk about, but I want to focus on the following points:

• Setting up a minimal Rust project that compiles to WebAssembly and runs in the browser.
• Explaining how the interaction between JavaScript and WebAssembly works and what the (current) limitations are.
• Introducing wasm-bindgen and wasm-pack and why they are such useful tools.
• Have a closer look at wasm-bindgen and how it works.

I’ll try to be as brief as possible, but also link to more advanced reading-material. So hopefully, this post will be interesting to many people, no matter how much they already know about Rust and WebAssembly.

Okay, can’t you just show me some code?

Of course! If you’d prefer, I have the small example project for this tutorial on GitLab. Additionally, I have a more complex application (a small game) on GitLab that’s written in pure Rust and runs in the browser.

Both are fairly rough around the edges, though, and don’t make much of an effort to explain how they work. So I certainly recommend that you read on here instead. 😉

Compiling Rust to WebAssembly

I already spent too much time talking, so let’s get into it straigt away. Note: If you want to follow along, you need to have Rust (ideally via rustup) and wasm-pack installed.

Setting up a Rust project

If you spent any time with Rust, then you’re familiar with cargo, Rust’s package manager. Like everything about Rust, it is very simple to use. We can create a new library project with the following command:

It is important that we create a library project and not a binary project, but more about this later.

The resulting my-wasm-app folder is very simple and only contains two files: our Cargo.toml package definition and a Rust source file src/lib.rs.

Both those files are basically empty and don’t contain anything of interest to us, but we are now set up and could start writing some Rust code. Unfortunately, our project is still a normal Rust project that will compile to native code instead of WebAssembly:

Building the project will create a .rlib file in the target directory (that is the default for a “lib” crate and can change at any time):

rlib is Rust’s static library format. It is basically a platform specific binary with some metadata attached. If we were to build a binary executable that uses our library as a dependency, then Rust would know how to statically link this dependency into the executable.

Compiling to WebAssembly

Unfortunately, an .rlib file will not work when we want to compile to WebAssembly. We need to change the type of our binary to generate a dynamic library (“cdylib”) instead of a static one. We can do this by adding the following to our Cargo.toml:

Compiling the project will now generate an .so file (or .dll file on Windows):

Note that I left the “rlib” crate-type in the configuration, so the .rlib file is still created. This is not strictly necessary, but generally recommended. If you really want to remove it, you can, but I’ve had some trouble running wasm-bindgen-tests in particular without it.

We are now ready to compile the library to WebAssembly, i.e. Rust’s compilation target “wasm32-unknown-unknown”:

This will (assuming you have the “wasm32-unknown-unknown” target installed) generate both a .rlib and a .wasm file:

Don’t worry if you don’t have the “wasm32-unknown-unknown” target installed. It will be installed automatically when we start using wasm-pack instead of cargo to compile our binary. If you really want to install the target manually now, you can do so by using rustup:

Running our WebAssembly in the browser

Our WebAssembly binary isn’t particularly useful, yet. I doesn’t contain any code or export any symbols. We can fix this by adding an exported function to src/lib.rs. It doesn’t do much. It just takes an integer, doubles it and returns the result.

The “extern” keyword and the “no_mangle” attribute mark this function as part of our foreign function interface (FFI) and tell the Rust linker to generate a binary with an exported function named “double_it”.

When we compile our code, we can inspect the exports of the resulting binary with wasm-nm to see our exported function:

Loading our WebAssembly from JavaScript and calling the exported function is now fairly simple and can be done via the WebAssembly.instantiateStreaming() function of your browser. We can find our function in the “exports” collection of the loaded WebAssembly instance:

This will print out “The result of 12*2 is: 24” to your browser’s debug console:

But what about this mysterious “importObject” parameter that we need to pass to WebAssembly.instantiateStreaming()?

Talking to JavaScript from WebAssembly

So far, we’ve called a Rust function from JavaScript and it was easy enough, but we haven’t yet gone the other way and called JavaScript from Rust. We do this by first defining a JavaScript function that we want to call:

We now add an import for this function into our Rust code, again using the “extern” keyword to mark it as part of the FFI, but now as an imported function instead of an exported one:

We can now use this import to implement our exported “double_it” function as follows. Note that we need an unsafe block when calling the imported function, because the Rust compiler cannot guarantee that it is safe.

When we compile this code and look at the import symbols of the generated binary, we see that there is an import symbol for the “add_integers” function:

But even though this code compiles fine, when we try to load it into the browser, we get a linker error:

What happens here, is that the WebAssembly linker sees the import symbol “add_integers” and doesn’t know what JavaScript code it should be linked to. It doesn’t make any assumptions or guesses, but instead requires us to explicitly state how we want each import to be resolved. We do this by adding an entry to the “importObject” we pass to the linker, where we map the name of the import to the code it should run:

Here, we resolved the symbol with a function of the same name. But you actually don’t need to define a function and could alternatively just use a closure, like this, if you wanted:

Now, after the imports are resolved properly, our example should work again.

Limitations of this approach

As you can probably already tell, things will get messy very quickly as we add more imports and exports. We need to redefine our imports/exports several times over multiple files, keep them in sync, and will only notice if we got something wrong at runtime.

You might also have noticed, that we needed to add very strict type annotations when importing the add_integers() function. While JavaScript is duck-typed and very permissive about the types of parameters that we can pass into the function, Rust is far stricter and forces us to choose a specific type when importing the function.

And we haven’t even discussed the most important limitation: WebAssembly can only export or import functions, that deal with number types, specifically i32, i64, f32, and f64.

This is why we’ve been using a very contrived example about adding integers so far. Choosing a more interesting example with more complex data types would have been impossible. There are workarounds for this limitation, though. While a WebAssembly process has no access to the memory of the JavaScript that loaded it, the opposite is not true. JavaScript does have access to the memory of the Webassembly process, meaning it can write and read values there as it pleases. This makes arbitrary interfaces between WebAssembly and JavaScript in both directions possible, but requires extensive glue code to be written.

wasm-bindgen and wasm-pack to the rescue

Fortunately for us, the Rust ecosystem solves this problem for us with a tool called wasm-bindgen. We can very easily define rich and arbitrary imports/export for our WebAssembly code and we won’t have to write anything by hand. wasm-bindgen consists of two components:

• A Rust-library with powerful macros that generate exports/imports and necessary glue code on the Rust side.
• A CLI-tool for generating a JavaScript wrapper that makes our WebAssembly loadable as an ES module.

While it is possible to install the wasm-bindgen tool and use it directly, it is common to use the wasm-pack tool instead, which is very powerful and will simplify our build-process for WebAssembly significantly. Among other things it will:

• Call cargo for us and compile our code to WebAssembly.
• Call wasm-bindgen to generate a JavaScript-wrapper around the WebAssembly.
• Optionally run tools like wasm-opt over the generated WebAssembly to optimize it for size or speed.
• Generate a NPM .package definition for our .wasm and accompanying .js files.
• Integrate nicely with bundlers (at the moment only webpack).

It is not strictly necessary to use wasm-pack, but very convenient, so from now on we’ll use it to build our project instead of using cargo directly.

Since wasm-pack depends on wasm-bindgen, we first need to add it as a dependency to our Cargo.toml file:

Then we can build the project using wasm-pack:

We passed the “–dev” flag to wasm-pack, since it otherwise defaults to building the release version of our binary.

We also pass the “–target web” flag to it. By default, wasm-bindgen will generate .js files that are intended to be consumed by webpack. Specifying the “web” target tells wasm-bindgen that we want to load the JavaScript into the browser directly. This makes it a little easier to understand what’s going on in the context of this post, but in the real world, you would probably want to have your WebAssembly module integrate with your existing npm/webpack setup.

As mentioned, wasm-pack does more than just build our .wasm file and call wasm-bindgen. As you can see from the last line of its output, it created a folder named “pkg”. If you look inside, you’ll find the following structure:

It contains everything we need to ship our code, including a package.json file that allows us to create an npm package for our WebAssembly and accompanying JavaScript. It even contains Typescript declaration files (*.d.ts) for our .wasm and .js files. Those files are very useful, but we’ll just focus on the JavaScript and WebAssembly files here.

The .wasm file by itself is already pretty interesting: Most obviously, its name has changed and now ends in “_bg.wasm” (the “bg” stands for “bindgen”). Less obviously, the content of the .wasm-file changed as well and is much smaller:

I already mentioned that wasm-pack performs optimizations on the .wasm binary by using wasm-opt, but those optimizations are disabled in the “dev” build. What actually happens here is part of the post-processing that wasm-bindgen does on the .wasm file. I’ll talk about this post-processing step again later, but for now, just remember that the WebAssembly generated by the Rust compiler is (very slightly) modified by wasm-bindgen. In this case, the size difference is caused by wasm-bindgen stripping the debug information from the .wasm file (unless we tell it not to). You can verify this for yourself by using wasm-dis to disassemble both .wasm files to text and compare the contents.

We don’t have any more need for our previous example code, so we delete everything in src/lib.rs and add the following definitions:

This adds an import for the console.log() function, as well as an export for a function say_hello(). The export can be called from JavaScript and takes a string parameter, which would not be possible without wasm-bindgen.

All we have to do now, after we build this code using wasm-pack, is to remove our awkward call to WebAssembly.instantiateStreaming() and replace it with this to load our code:

We can now load the wrapper-JS file like we would load any other module and are completely unaware that we’re actually loading and running Rust code!

When we try this out and open the example in a browser, we get this result:

There is much more to say about wasm-bindgen (enough that people write books about it). If you want to know more, there are many resources available, including the aforementioned book. For the last part of this post, I want to have a closer look at what goes on behind the scenes to make this work.

Behind the scenes of wasm-bindgen

My main goal here is to give you a rough understanding about the inner workings of wasm-bindgen and a foundation for looking at the generated code yourself. For the .js files, this is very easy, but the generated Rust code is also quite easy to access once you have cargo-expand installed.

The first important thing to understand about wasm-bindgen is that it will generate code in two distinct phases:

• The wasm-bindgen macro generates additional Rust code, which gets compiled to WebAssembly.
• The wasm-bindgen executable generates JavaScript wrapper-code for the previously created WebAssembly.

I made a small diagram to illustrate this. You can see that there is generated code that lives both in the Rust and in the JavaScript world. When hand-written JavaScript and Rust interoperate, they never do so directly, but rather through the generated code:

This design has some interesting implications: The wasm-bindgen binary only works on the .wasm file and doesn’t have access to the type information it needs to create JavaScript-wrapper-code for the exported functions. This would normally make generation of the JavaScript-wrapper impossible, but is worked around in an unconventional way: The wasm-bindgen binary gets the type information it needs from special functions in the .wasm file, which where created by the wasm-bindgen macro during compilation and carry type information (more about this here). After the JavaScript wrapper is generated, wasm-bindgen post-processes the .wasm file to remove those temporary exports again. For our purposes, we’ll ignore this temporary code and just focus on the final interaction between Rust and JavaScript.

To start things off, we’ll have a look at the exported symbols from the post-processed binary in the “pkg” folder:

There are two exports for allocating memory and one export for our say_hello() function. If we look at the corresponding JS function, we can see how it is invoked:

This is actually fairly straightforward: The string parameter “name” is copied to the memory of the WebAssembly-process by the function passStringToWasm0() (memory allocation is done via the two allocation exports), and the say_hello() export is called with two integer arguments: A pointer (i.e. index into memory) to the start of the string and the length of string. If you are confused about the WASM_VECTOR_LEN variable: This is a global variable that gets set by passStringToWasm0() and contains the length of the allocated buffer.

The Rust side of the code is also not too complicated. Our say_hello() function, that we annotated with #[wasm-bindgen], doesn’t get modified and remains as we wrote it. What gets generated is an additional function, which converts the arguments passed from JavaScript to actual Rust values and then calls our say_hello() function. Brace yourself, I have not simplified the generated code, but it looks more complicated than it actually is:

I’ll go through this definition step by step, explain everything, and simplify it to make it more understandable.

First, the attributes are fairly simple: The “allow” attributes just disables compiler warnings caused by the linter, while “export_name” changes the name of an exported function. Therefore, this function appears as say_hello() in the export symbols of the .wasm binary and is the one that actually gets called by JavaScript.

The “RefFromWasmAbi” trait marks a type that wasm-bindgen knows how to pass as a reference from JavaScript to Rust. There are also other traits, for example for passing a value by copy or mutable reference, but in this case we’re passing an immutable reference. Rust’s primitive “str” type implements this trait, which provides us with the following:

• The associated type “Abi” (which stands for “application binary interface”) defines the actual type that is passed from JavaScript to Rust. As we already know, for strings, this is a pair of integers. On the Rust side, this pair of integers is represented by the “WasmSlice” struct, which I’ll explain more below.
• The associated function ref_from_abi(), which we can use to get a reference to a normal Rust object from the ABI-representation. For the “str” type, this will create an object of type “Box<str>”.

The generated code also ensures dereferencing of arg0 from “Box<str>” to “&str” by notating it as “&*arg0”. A simple “&arg0” would be enough, in this case, because of Rust’s deref coercion feature, so I’m not sure why a “&*” is used here. My best guess is that “&*arg0” is just more explicit.

The “ReturnWasmAbi” trait marks types that can be passed as return values back to JavaScript. say_hello() doesn’t return anything, so nothing interesting is happening here. “<() as ReturnWasmAbi>::Abi” is just Rust’s “()” type. Also, the “<() as ReturnWasmAbi>::return_abi(…)” function will always just return “()”.

Using what we’ve learned, this is a simplified version of the function that still does the exact same thing:

If you are anything like me, you’re probably confused why this function takes only one parameter, while it is called with two parameters from the JavaScript side (a pointer to the string data and the length of the string). To understand this, we need to look at the definition of WasmSlice:

It is a struct that’s layed out in C-representation, consisting of our two values. The compiler will convert this to two normal integer parameters during compilation. You can verify this by using wasm-dis to disassemble your .wasm file. If you do so, don’t be confused that say_hello() takes two parameters of type i32. WebAssembly only knows one 32-bit integer type, which can hold both signed and unsigned integers.

More about wasm-bindgen

There are, of course, a few more important things about wasm-bindgen that I think you should know about:

So far, we created all bindings for JavaScript functions by hand. That’s generally something you don’t want to do if you can avoid it, since it introduces duplication, takes time, and creates room for mistakes. Luckily, the wasm-bindgen developers already provide you with pre-defined bindings that you should use, if you can. They will make your life much easier:

• js-sys: Bindings for the global JavaScript APIs, that are guaranteed to be present in all runtimes, be it a browser or node.js or something else.
• web-sys: Bindings for the Web API, i.e. for access to the DOM, stuff like console.log(), and so on.

We also only looked at how to export/import functions, but it is also possible to export/import types. The wasm-bindgen book has sections for exporting Rust structs to JavaScript and importing JavaScript types in Rust.

One type that is currently somewhat painful to pass between JavaScript and Rust is closures. Especially passing a long-lived Rust-closure, like an event-callback, to JavaScript is quite messy! It certainly is possible with wasm-bindgen’s Closure type and I do it in my example-app, but be warned.

Final thoughts about Rust/WebAssembly

In general, I was positively surprised by the maturity of Rust’s WebAssembly toolchain and how seamless Rust and JavaScript can integrate, especially since WebAssembly is still basically in its MVP state. Also, developing Rust for the browser was generally very stress-free. I wrote code in Rust, unit-tested it, wrote a DOM-manipulation layer and everything generally just worked. The Rust compiler does protect you quite a bit from making stupid mistakes: My WebAssembly never crashed on me once while I was iterating on the the first prototype for my Rust/WebAssembly example app (try pulling that off in JavaScript), though I did manage to cause a crash later on while I was adding some polish to the UI.

In my opinion, WebAssembly is an extremely interesting technology and a very powerful tool to have at your disposal. I don’t expect Rust to replace JavaScript anytime soon (and it shouldn’t!), but there are some very interesting and promising use-cases for WebAssembly (maybe I’ll write about those in the future).

I hope you enjoyed this post and keep on rusting, friend!