Orca now has a new build system that leverages the Zig toolchain, replacing the old python-based build system. Immediate benefits include:
Orca is a project with a lot of moving parts. There are several distinct pieces of the Orca project itself:
Additionally, there are several dependencies:
Needless to say, wrangling all these components with their individual build needs can get complicated! We needed a cross-platform build system that could handle the complexity without making it too difficult to add new pieces.
The old build system was a homegrown python script that Ben Visness initially wrote and other maintainers extended over time. It was relatively simple to read and easy to maintain, especially compared to other build systems (e.g. cmake). However, as the Orca project grew larger and added more artifacts and dependencies, it began to show limitations.
If you wanted to build and run Orca from scratch, you would first
have to explicitly build two expensive dependencies, angle
and dawn
. Next you’d need to build the rest of the Orca
project, then manually install it to a particular location that’s in
your PATH
.
./orcadev build-angle
./orcadev build-dawn
./orcadev build
./orcadev install ~/bin/orca
Building angle and dawn in particular is a lengthy process, and can
take over an hour. Building the rest of the Orca project takes under a
minute, but since orcadev
didn’t support intermediate
caching, if you made a change to the low-level platform layer, you’d
need to rebuild the whole thing again.
There were commands to rebuild specific parts of the Orca project -
for example, running ./orcadev build-orca-libc
when making
changes to the Orca libc. But even that command could take a long time,
since it would blindly rebuild all the source files that went into that
library.
Included in the Orca repo are a set of samples that demonstrate how to write Orca apps. They show off how to use the Orca, C, and graphics APIs, build them into a wasm module using the Orca libraries, and bundle them into a standalone app. These also serve as important testbeds for ensuring the wasm-side functionality still works while the test suite gets fleshed out. However, iterating with them is a bit painful, since they each have their own individual build script. For example, when making a platform change, you need to:
src/
./orcadev build && ./orcadev install
cd
into the sample directory:
cd samples/clock
./build.sh
open Clock.app
(MacOS) or
./Clock/bin/Clock.exe
(Windows)Additionally, there was no easy way of running all the tests at once - you’d have to manually build and run each test:
cd tests
pushd files
./build.sh
./bin/test_files.exe
popd
pushd file_dialog
./build.sh
# etc...
Similarly, there was no easy way of building and running the sketches
- small, standalone programs that use the Orca platform layer to
explore/exercise some feature of the API, such as the vector drawing or
UI APIs. Again, you had to cd
into every directory and
build each one individually. While they aren’t a core part of the
user-facing Orca story, it’s nice to have a little code sample that acts
as a showcase for how to use some part of the API. But without CI
integration, many had bitrotted over time.
Cross-platform toolchain compatibility was also a pain. Apple’s
version of clang, shipped with XCode tools, doesn’t support WebAssembly,
so you had to install the version shipped with Homebrew. Additionally,
because clang doesn’t bundle WebAssembly compiler runtime support by
default, you had to manually install the
libclang_rt.builtins-wasm32
file from the wasi SDK
(https://github.com/WebAssembly/wasi-sdk/releases) for all
platforms.
Maintenance of this toolchain was also a pain. At one point GitHub changed the version of clang installed by default on the Windows runners, which removed support for WebAssembly, requiring us to manually install a version that did support it. Fun times!
The Zig programming language has been making some waves with it’s compatibility story around C/C++ projects. It has first-class support for building C/C++ source, even if you never run any Zig code. The compiler and build system offer:
It promised to solve a lot of the problems with the old build system, so we decided to port it and have been happy with the result.
In the new build system, to build and install from a fresh checkout, you simply run:
zig build -Dsdk-path=~/bin/orca
This will download any needed dependencies (i.e. Angle and Dawn), generate any needed source files, build all the libraries and exes, and package the SDK to the specified path, all in one line. You don’t need to install any additional toolchains or dependencies manually, other than the Zig compiler (https://ziglang.org/download).
The best part is if you make a change to any file that’s participating in the build graph, only the downstream dependencies of the file are invalidated and rebuilt. For example, editing these files:
src/graphics/graphics_common.c
: rebuilds the platform
library and downstream dependenciessrc/graphics/wgsl_shaders/raster.wgsl
: regenerates
wgpu_renderer_shaders.h
and rebuilds the platform
librarysrc/wasmbind/core_api.json
: regenerates the core API C
bindings and rebuilds the wasm SDK librarysrc/tool/bundle.c
: rebuilds the CLI toolWe’re also leveraging the Zig package manager to download and cache
Angle and Dawn dependencies to avoid putting the burden of building
those on the local user. These dependencies have their own repos and
build scripts. We build them and create a release in CI that contains
the artifacts for each platform. The build.zig.zon
file
references the specific release artifact URL hosted on GitHub and saves
its’ hash to ensure newly-downloaded archives are well-formed. The
result is a ~4MB one-time download to get the platform-specific
dependency, instead of 1+ hour builds and additional dependency
management to get these libraries to build. This shaves a huge amount of
friction off the onboarding process for anyone trying out the
project.
Regarding samples, tests, and sketches, these are also now one-click builds:
zig build samples # builds and bundles all samples
zig build sample-clock # builds, bundles, and runs the clock sample
zig build test # builds all tests and runs select tests without a window
zig build sketches # builds all sketches
Below is an image that shows a high-level overview of the build graph:
Thanks for reading! If you are interested in Orca and want to try it out, there’s been no better time to clone the repo and build it from scratch. ;) If you want to keep following along with development, feel free to subscribe to the newsletter. And finally, if you want to support the project, you can do so via Github Sponsors.
Special thanks to Julian for his great code review suggestions and proofing this post.