Creating WebGL apps with Go
TL;DR In this article I’ll share my experience building an interactive 3D WebGL-based application for peer-to-peer messaging protocol simulation without writing any single line in JS. You’ll learn how GopherJS and Vecty framework can dramatically lower the complexity of building WebGL-enabled web apps in Go.
It’s often said “A picture is worth a thousand words”, but in the era of high-DPI screens and big data, the new idiom is now truer – “3D interactive visualization is worth a thousand pictures”.
Third dimension and interactivity can add exponentially more to the picture or story you want to show to your users, readers or clients. A single look at the data in 3 dimensions in a split of a second can tell you more than a 30-page report.
However, in 2018 it’s still prohibitively hard to build interactive 3D visualizations. For some common use cases such as statistical analysis of large datasets, there are tools such as Plotly Chart Studio exists. But once you find yourself looking for the non-common case, you have only two options – write it yourself or drop the idea. Very often we choose the latter.
Go, on the other hand, is famous for its simplicity, readability and performance. Wouldn’t it be nice to write 3D with WebGL in Go?
But, first, let’s do a quick recap of what’s WebGL and how does it really work.
On top of WebGL there are third-party libraries and frameworks exist – like Three.js, Babylon.js or A-Frame. They abstract many of the complexities of working with raw shader code, points and triangles, and provide higher-level tools like “material”, “mesh”, “geometry”, “camera”, “light” etc.
The most popular library is probably Three.js, and it has really nice and clean API. I would love to have something similar in Go. Maybe we can use GopherJS to automatically transpile Go code into JS?
GopherJS is incredible. Seriously.
When I first heard about Go-to-JS compiler a few years ago, my first reaction was “cool, but it’s unlikely to change anything”. Later, on dotGo 2016 conference in Paris, one of the GopherJS developers, and now a member of the Go team, Dmitry Shuravlyov gave a fascinating talk about GopherJS - go watch it, even if it’s 2 years old. When you realize that even the networking code like
net/http just works and you can send AJAX-queries by calling Go’s
http.Get(), you start realizing the power of this piece of technology.
A lot of already written Go code can be immediately available for use in the browser in a few seconds. Many JS libraries – like React, Angular, VueJS, D3, jQuery etc – already have bindings for GopherJS, and it’s usually easy to create custom bindings for the JS library you really need. In most cases, you may implement bindings just for a subset you need.
Installation is really simple, just run
go get -u github.com/gopherjs/gopherjs:
Building our first WebGL app in Go
So how can we create a web app with GopherJS and Three.js? We’ll need bindings for Three.js first.
Compare JS and Go usage of three.js.
The similarity is so striking that it’s probably feasible to write a static-analysis tool that will convert Three.js based code into Go code automatically!
Three.js Hello, World in Go
Now, let’s create the new project:
mkdir $(go env GOPATH)/src/github.com/divan/go-webgl-example cd $(go env GOPATH)/src/github.com/divan/go-webgl-example
And create two files –
index.html, which we will use to start our app in the browser:
index.html does nothing more than just defining a single canvas tag, which will be used for WebGL rendering, and including
three.js code with our GopherJS-generated
main.go is responsible for the high-level logic of our 3D app – setting up the scene, the camera, lights, objects, and handling animations and user interaction:
All you have to do now is just run:
and in a split of a second, it’ll generate two files -
go-webgl-example.js.map in the current directory.
Next, point your browser to the
index.html file in the current directory, and you should see this:
This example is not quite impressive, plus it’s not structured well – everything is in one function, and even
animate callback had to be implemented as closure to avoid making common variables, such as
renderer global ones.
For real applications we would want to have a much more flexible structure, ability to abstract things into different types and sub-packages, refactor it easily, write tests for logic, etc. We also need something more than just WebGL canvas – probably some introduction text, input controls for animation parameters, ability to upload files with data, etc. But, wait, for that we need DOM manipulation, and write code in HTML and CSS, and use popular JS frameworks. Luckily, there is as a fantastic framework exists that allows you to write web components in pure Go. It’s called Vecty, is obviously based on GopherJS, is quite mature and it’s fantastic.
Vecty – write your next frontend in Go
Vecty is a library for GopherJS that allows you to build frontend apps using high-level components (or widgets) and providing the abstraction layer over DOM and many browser’s JS APIs. In a way, it’s similar to React (and even advertised as a React-like library), but I’m happy to disagree with this – it is way simpler and much more pleasant to work with.
The basic concept in Vecty is the Component interface - any type implementing it can represent a visual component within a Vecty application. To define new Component, you simply create a type that embeds
vecty.Core and implement
Don’t be scared by those verbose lines in
Render() method. As soon as you start building DOM tree that way you realize how incredibly useful it is: no open/closing tags, super easy copy/paste operation (if you need to move some DOM nodes around), the structure is clear and makes sense, and it’s all strongly typed!
gopherjs build and open
index.html, which is as simple as this:
The most impressive thing with this basic example is how much simpler everything seems to be comparing to frontend development in the traditional way:
- there is no need for project bootstrap generators (like
yo) that generate tons of files just for the empty file
- there are no overcomplicated building tools that require you to read books on how to use them – build pipeline is crazy fast and simple (and you always have
go generateat your service for the rest)
- a magic-free code – it’s just an ordinary Go code, you can understand every single part of it just by reading it
- it is strongly typed – in practice, you will almost never see errors in browser’s console, but when you do, it’ll show you stack traces with .go files and line numbers
Now, Vecty is positioning itself as an experimental work-in-progress project, which it definitely is, but from my experience, it’s already more than useful for writing relatively decent small web apps.
I wrote a few apps with it already, and I never had more fun writing frontends!
So I want to share with you my experience building web app for p2p-messaging protocol visualization in pure Go, using Vecty and three.js bindings.
P2P messaging visualization
I’ve been working recently on peer-to-peer messaging protocol for Ethereum network, called Whisper (not to be confused with Signal’s Open Whisper Systems or Whisper media platform), and there was one problem – most people don’t have intuition about message propagation in p2p-networks. We’re so used to the centralized systems, that all our understanding of scales, timings, limits and weak points are based purely on the hierarchical topologies and client-server model.
And there is no wonder – to understand the behaviour of the complex network system, we need data, and for data, we collect metrics. Peer-to-peer networks are often used and designed to combat issues that centralization creates, and centralized data collection is one of them. So generally we don’t have metrics and data in large-scale p2p-network, which is good for the security of those systems, but bad for developers’ understanding of its behaviour.
So the obvious solution is to use network simulation software like OMNET++ or PeerSim, but in order to use it with custom p2p protocol code, that should have been ported first to C++ to use with those libraries. In the case of Whisper protocol, it’s written in Go, as a part of go-ethereum package, and the complexity and usefulness of the task was daunting. I wanted to have a simulator, where I can change algorithm or tune parameters, and in a couple of minutes see changes visualized and collect data from the simulator.
And that was the idea – to build a simulator, network graph visualization in 3D, collect data and visualize message propagation, add interactivity to the interface, where I could choose different networks, choose simulation parameters, replay simulation and analyze stats.
For that, I needed to code a frontend app with 3D visualization, write a simulator that can use an existing Go code and connect them. Most of the work had to be on the frontend part.
So that’s where I decided to combine my experiments with GopherJS and WebGL, and to give Vecty a try.
UI and widgets
The first thing you realize is that you’ll have to write your own widgets. Luckily there are no special rules for that – just create your widgets as types in separate .go files, and move them into the separate package when it makes sense.
The whole UI for the app consisted from sidebar with a bunch of configuration selectors and file uploads handling, and three tabs with different screens – first one is WebGL canvas with visualization.
Each widget is just a Vecty component, created and rendered from within the other component. You specify the layout, classes, CSS styles right there in
Render() method. Here is a code for a network selector widget with custom upload option, for example:
There are a few interesting things to note here:
- style and classes (markup) should be inside the
vecty/eventpackage provides handlers for most DOM events, and you just have to provide function or method in a callback fashion
- whenever you want to replace verbose parts with something more descriptive, it’s easy just to make a new function with the code, and simply call it from Render – like
- for conditional rendering, Vecty offers to nice helpers -
vecty.MarkupIf(), which allows you to build pretty advanced presentation logic of your components
Another essential part of the app was the ability to render the network graphs. Traditionally, such graphs are drawn using force-directed graph algorithm. The idea is simple:
- place all network nodes in 3D space pseudo-randomly
- apply repelling force between nodes (each node repels from each)
- apply spring-like force between nodes that have links (when the distance is closer than some value L, nodes repel, and attract otherwise)
- run this in the loop sufficient amount of times, till the system reach stable energy state
My implementation of it is far from perfect and has a lot of space for optimization, but it works fast and accurate enough for my case. For step 2 I use Barne-Hut approximation method to speed up the calculation.
The output of the algorithm is a 3D coordinates for each node. All we have to do now is to render objects and links with the given coordinates via WebGL.
Simulation code as well is written in Go, and currently does relatively simple thing: starts in-memory network, launches Whisper nodes (based on the real code), sends a message, and records events like “message received” or “message sent” from each node in the network, and dumps it into special log file, called “propagation log”, which can be analyzed and visualized further.
Simulation part is the heaviest one, so it’s not wise to run it in the browser – despite the goroutines support in GopherJS, the code is still limited by JS engine design and can use only one core in the browser. So this part can be run natively on the server and talk to frontend via the network.
Putting it together
The coolest thing about current setup is that you can choose which part will run in the browser, and which on the backend almost on the fly – it’s the same Go code. For example, in a first attempt, I designed app in a way that physics simulation for force-directed graph was running on the backend, and then sending the result to the browser via WebSocket. But If I wanted to change the simulation parameter or run a single iteration on a keypress, I had to implement it as one more command for WebSocket protocol, and it was tedious. It turned out to be so much simpler just to move this part of code into the browser and call natively from the Vecty keypress handler.
The result, while still being a work in progress, is so much more than I had imagined when I started to work on it.
You can see a demo of the app here:
It’s not finished yet, and definitely not the coolest WebGL app you’ve seen, but it’s pretty good for pure Go code, right?
You can try it right yourself online at https://divan.github.io/whispervis/ as well (without the simulation part).
While main hopes and expectations in the frontend world revolve around WebAssembly and the upcoming possibility to write code for browsers in pretty much every language, GopherJS and Vecty already provide an excellent playground to test out UI framework designs. There is a pending PR into Vecty with WebAssembly support, by the way.
To me, using Go in a web application has a nice property of making typically overly complicated process so much simpler.
- it’s easier to start (compare “hello, world” in Vecty with
create-react-apphelper tool output – it’s 2 files and 592 bytes vs 29890 files and 1MB + 270MB of dependencies)
- it’s easier to develop – the only command you have to learn is
gopherjs build, the rest is the same, you can use Go tooling for building, testing, benchmarking, static analysis, etc.
- it’s easier to read and write code – Vecty was designed with strong a focus on simplicity and performance, so you will not find new weird concepts, so typical for frontend frameworks. There is no
mapDispatchToProps(dispatch, [props])madness, no implicit magic behind every concept, no breaking changes every single release, no multiple ways to do the same thing and no permanently outdated tutorials. It’s just pure developer’s pleasure to work with.
As I said earlier, that’s the coolest experience with frontend development I ever had.
However, there were drawbacks too:
- Go frontend community is young, and there is no abundance of component libraries. So the most widgets you’ll have to implement yourself. (The good news, of course, is that it’s really easy)
It’s also important to understand, that large-scale web applications (like Gmail-scale) will probably require much more maturity from the Go framework. And Vecty definitely not a production ready replacement for the modern Web development practices. But for many cases where you need small decent frontend apps, it’s actually a great fit already. Just a couple of days ago I’ve read a post where author was building web app for displaying and analyzing graphs of connections between composers. That’s the perfect target audience :)
As for WebGL, ability to use from Go opened a whole new world of possibilities for me, giving the promise of writing well-structured, readable, testable and maintainable apps, that will work in a few years without any changes. It’s good to keep an eye on advancements in 3D graphics for the web like WebGPU, but if you want to write it today, WebGL is a way to go.
And finally, while learning and working with GopherJS, Vecty and three.js bindings, I had a pleasure to one more time realize how incredibly friendly and helpful Go community is. When I just started playing with Vecty and was a bit confused with the proper way to use WebGL canvas from within Vecty, I went straight to the #vecty channel on Gophers’ Slack and humbly asked for advice. The Vecty author, Stephen Gutekanst, not only answered my question in a great detail, but also created and published the library that does exactly what I needed, and thoroughly commented it and explained how to use it.