Go and wasm: generating and executing wasm with Go
Today we will see how we can interact with WebAssembly, from Go: how to execute WebAssembly bytecode from Go and how to generate WebAssembly bytecode with Go. But first of all: what is WebAssembly?
WebAssembly
According to webassembly.org, WebAssembly (wasm
for short) is a new portable, size- and load-time-efficient format suitable for compilation to the web.
In a way, wasm is the next evolution of asm.js and PNaCl: it’s a new way to run code, on the web.
But faster.
WebAssembly is currently being designed as an open standard by a W3C Community Group that includes representatives from all major browsers. As of late 2017, WebAssembly is supported by all major web browsers (Chrome, Edge, Firefox and Safari.)
With wasm, developers get:
- a compact, binary format to send over the wire and incorporate into their project,
- a near-native performance boost,
- performant and safe code executed inside the browser sand box,
- another language (besides JavaScript) to develop their projects: one can target wasm from C/C++, Rust, … (and, eventually, Go.)
wasm format
WebAssembly is formally defined by a set of specifications. The wasm binary format is described here.
A .wasm
file is the result of the compilation of C/C++, Rust or Go code with the adequate toolchain.
It contains bytecode instructions to be executed by the browser or any other program that can decode and interpret that binary format.
A .wasm
file contains a wasm module.
Every wasm module starts with a magic number \0asm
(ie: []byte{0x00, 0x61, 0x73, 0x6d}
) and then a version number (0x1
at the moment.)
After that come the different sections of a module:
- the types section: function signature declarations,
- the imports section: imports declarations,
- the functions section: function declarations,
- the tables section: indirect function table,
- the memories section,
- the globals section,
- the exports section,
- the start function section: the
func main()
equivalent, - the code segments section: function bodies, and
- the data segments section.
The full description with all the details is available here. A more gentle and less dry introduction to wasm can be found at https://rsms.me/wasm-intro.
A toolchain like emscripten will thus take a set of C/C++ source code and generate a .wasm
file containing type definitions, function definitions, function bodies with the corresponding wasm instructions (e.g.: i32.store
to store a signed 32b integer to memory, if
and return
control instructions, etc…)
We will see how to generate .wasm
files in a bit but let us first play with it.
Consider this basic.wasm
:
1 2 3 4 5 6 7 8 9 10 11 12 |
$> curl -O -L https://github.com/go-interpreter/wagon/raw/master/exec/testdata/basic.wasm $> ls -lh ./basic.wasm -rw-r--r-- 1 binet binet 38 Dec 12 17:01 basic.wasm $> file ./basic.wasm ./basic.wasm: WebAssembly (wasm) binary module version 0x1 (MVP) $> hexdump -C ./basic.wasm 00000000 00 61 73 6d 01 00 00 00 01 05 01 60 00 01 7f 03 |.asm.......`....| 00000010 02 01 00 07 08 01 04 6d 61 69 6e 00 00 0a 07 01 |.......main.....| 00000020 05 00 41 2a 0f 0b |..A*..| 00000026 |
So all the tools at our disposal agree: it is indeed a wasm file. But couldn’t we do something to extract some more informations about that file?
Like for object files, there is an objdump
-like command that allows to inspect the contents of a binary wasm file.
The wabt project provides a wasm-objdump
command, written in C/C++.
But as this is a Go advent post, we will instead use the one provided by wagon:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
$> go get github.com/go-interpreter/wagon/cmd/wasm-dump $> wasm-dump -help Usage: wasm-dump [options] file1.wasm [file2.wasm [...]] ex: $> wasm-dump -h ./file1.wasm options: -d disassemble function bodies -h print headers -s print raw section contents -v enable/disable verbose mode -x show section details |
Running it with basic.wasm
as argument gives:
1 2 3 4 5 6 7 8 9 |
$> wasm-dump -h ./basic.wasm ./basic.wasm: module version: 0x1 sections: type start=0x0000000a end=0x0000000f (size=0x00000005) count: 1 function start=0x00000011 end=0x00000013 (size=0x00000002) count: 1 export start=0x00000015 end=0x0000001d (size=0x00000008) count: 1 code start=0x0000001f end=0x00000026 (size=0x00000007) count: 1 |
This basic.wasm
file has 4 sections.
Let’s dig deeper:
1 2 3 4 5 6 7 8 9 10 11 |
$> wasm-dump -x ./basic.wasm ./basic.wasm: module version: 0x1 section details: type: - type[0] <func [] -> [i32]> function: - func[0] sig=0 export: - function[0] -> "main" |
This wasm module exports a function "main"
, which takes no argument and returns an int32
.
The content of these sections is the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
$> wasm-dump -s ./basic.wasm ./basic.wasm: module version: 0x1 contents of section type: 0000000a 01 60 00 01 7f |.`...| contents of section function: 00000011 01 00 |..| contents of section export: 00000015 01 04 6d 61 69 6e 00 00 |..main..| contents of section code: 0000001f 01 05 00 41 2a 0f 0b |...A*..| |
If you read wasm speak fluently you won’t be surprised by the content of the next snippet, showing the disassembly of the func[0]
:
1 2 3 4 5 6 7 8 9 |
$> wasm-dump -d ./basic.wasm ./basic.wasm: module version: 0x1 code disassembly: func[0]: <func [] -> [i32]> 000000: 41 2a 00 00 00 | i32.const 42 000006: 0f | return 000008: 0b | end |
It puts the int32
constant 42
on the stack and returns it to the caller.
It’s just the wasm equivalent of:
|
|
Can we test this?
Executing wasm with wagon
wagon
actually exposes a very limited non-interactive (yet!) interpreter of wasm: wasm-run
.
1 2 3 4 5 6 |
$> go get github.com/go-interpreter/wagon/cmd/wasm-run $> wasm-run -h Usage of wasm-run: -v enable/disable verbose mode -verify-module run module verification |
Let’s try it on our basic.wasm
file:
1 2 |
$> wasm-run ./basic.wasm main() i32 => 42 (uint32) |
Victory !
wasm-run is a rather simple and limited (yet!) wasm embedder:
- it reads the provided wasm file,
- it (optionally) verifies the wasm module,
- it creates a VM with
go-interpreter/wagon/exec
, that VM will execute thestart
section of the module (if any) - it runs all the exported functions that take no input parameters
and voila!
Ok, but wasn’t wasm designed for the web and its browsers?
Executing wasm in the browser
Switching gears, let us write a little web server that will serve a simple wasm module:
|
|
Running this in a terminal:
1 2 |
$> go run ./main.go 2017/12/14 12:45:21 listening on ":5555"... |
and then navigating to that location, you should be presented with:
1 2 3 4 5 6 7 8 |
WebAssembly content 00000000 00 61 73 6d 01 00 00 00 01 07 01 60 02 7f 7f 01 |.asm.......`....| 00000010 7f 03 02 01 00 07 07 01 03 61 64 64 00 00 0a 09 |.........add....| 00000020 01 07 00 20 00 20 01 6a 0b |... . .j.| WebAssembly add(1, 2)= 3 |
Victory! again!
For more informations about the JavaScript API that deals with WebAssembly, there are these useful references:
- https://developer.mozilla.org/en-US/docs/WebAssembly/Using_the_JavaScript_API
- https://developer.mozilla.org/en-US/docs/WebAssembly/Loading_and_running
But, up to now, we have only been able to inspect and execute already existing wasm files. How do we create these files?
Generating wasm
We briefly mentioned at the beginning of this post that wasm files could be generated from C/C++ (using emscripten) or from Rust (using cargo or rustup). Instructions related to these tasks are available here:
- https://developer.mozilla.org/en-US/docs/WebAssembly/C_to_wasm
- https://github.com/raphamorim/wasm-and-rust
Compiling Go code to wasm is also doable, but the support for this backend hasn’t been yet integrated into gc
.
An issue is tracking the progress of this feature: https://github.com/golang/go/issues/18892.
As that discussion is quite long, here is the executive summary: a development branch with preliminary support for wasm has been created by @neelance (Richard Musiol) (yeah!).
Here are the instructions to compile a gc
toolchain with a GOOS=js GOARCH=wasm
environment:
1 2 3 4 5 6 7 8 9 |
$> cd somewhere $> git clone https://go.googlesource.com/go $> cd go $> git remote add neelance https://github.com/neelance/go $> git fetch --all $> git checkout wasm-wip $> cd src $> ./make.bash $> cd ../misc/wasm |
The misc/wasm
directory contains all the files (save the actual wasm
module) to execute a wasm
module with nodejs
.
Let us compile the following main.go
file:
1 2 3 4 5 |
package main func main() { println("Hello World, from wasm+Go") } |
with our new wasm-capable go
binary:
1 2 3 4 5 |
$> GOARCH=wasm GOOS=js go build -o test.wasm main.go $> ll total 4.0K -rw-r--r-- 1 binet binet 68 Dec 14 14:30 main.go -rwxr-xr-x 1 binet binet 947K Dec 14 14:30 test.wasm |
Copy over the misc/wasm
files under this directory, and then finally, run the following server.go
file:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
package main import ( "flag" "log" "net/http" ) func main() { addr := flag.String("addr", ":5555", "server address:port") flag.Parse() srv := http.FileServer(http.Dir(".")) log.Printf("listening on %q...", *addr) log.Fatal(http.ListenAndServe(*addr, srv)) } |
Like so:
1 2 3 4 5 6 7 8 9 10 |
$> ll total 968K -rw-r--r-- 1 binet binet 68 Dec 14 14:30 main.go -rw-r--r-- 1 binet binet 268 Dec 14 14:38 server.go -rwxr-xr-x 1 binet binet 947K Dec 14 14:30 test.wasm -rw-r--r-- 1 binet binet 482 Dec 14 14:32 wasm_exec.html -rwxr-xr-x 1 binet binet 7.9K Dec 14 14:32 wasm_exec.js $> go run ./server.go 2017/12/14 14:39:18 listening on ":5555"... |
Navigating to localhost:5555/wasm_exec.html
will present you with a [Run]
button that, when clicked should display "Hello World, from wasm+Go"
in the console.
We’ve just had our browser run a wasm
module, generated with our favorite compilation toolchain!
Conclusions
In this blog post, we have:
- learned about some of the internals of the
wasm
binary format, - inspected
wasm
files, - interpreted a
wasm
file, - served a
wasm
file vianet/http
, and - compiled a
wasm
module with a modifiedgo
toolchain.
I hope this has inspired some of you to try this at home.
WebAssembly is poised to take the Web by storm, bring “native” performances to the Web platform and allow developers to use other languages besides JavaScript to build Web applications.
But even if WebAssembly has been designed for the Web, nothing prevents it from being used outside of the Web.
Indeed, the wasm
binary format and the bytecode that it contains can very well become a very popular (and effective) Intermediate Representation format, like LLVM’s IR.
For example, the go-interpreter/wagon project could build (Help wanted) a complete interpreter in Go, generating wasm bytecode from Go source code, and then executing that wasm bytecode.
“Building an interpreter in Go, for Go”… what’s not to love!?
This could even be used as a backend for Neugram, but that’s another story…