I recently merged changes that touch two core idioms of the Orca codebase.

Allocator Interface:

I added an “allocator interface” that is meant to replace the arena parameter in most memory-allocating functions. This allows passing a different allocator implementation such as a heap or a pool when needed. The interface exposes a single function to push data onto the allocator, with a few variants for alignment / initialization and typed objects or arrays allocation. A concrete allocator has a pointer to its allocator interface that you can pass to allocating functions, eg:

oc_str8 str = oc_str8_pushf(arena->allocator, "Hello, world!");

// or

oc_str8 str = oc_str8_pushf(heap->allocator, "Hello, world!");

Deallocation is controlled by the caller using the concrete allocator API (e.g. freeing or clearing), e.g.:

oc_arena_clear(arena);

// or

oc_heap_free(heap, str.ptr);

If the callee needs to free things before returning, this means it can allocate them on a callee-owned allocator (e.g. a scratch arena), so the generic allocator interface never needs to expose an API to free memory.

Typed Linked Lists:

I added a new intrusive linked-list type that is generic and type-safe, to gradually replace untyped linked lists. This uses a neat trick to attach type info and integer constants to a type definition without increasing the memory footprint of that type. I describe this technique in more details in this blog post. Usage is as follows:

// Define a type that is "listable":

typedef struct my_type
{
    // this makes my_type listable
    oc_typed_list_links links;
    f32 x, y;
} my_type;

 // Declare a list type that can hold items of type my_type, linked through the 'links' field.
typedef oc_typed_list(my_type, links) my_type_list;

// Create a list and push some items
my_type_list list = {0};
my_type t1 = {.x = 0, .y = 1};
my_type t2 = {.x = 2, .y = 4};
my_type t3 = {.x = 3, .y = 0};

oc_typed_list_push_back(&list, &t1);
oc_typed_list_push_back(&list, &t2);
oc_typed_list_push_back(&list, &t3);

// Loop through the list and print items
oc_typed_list_for(list, elt)
{
    printf("(%f, %f) ", elt->x, elt->y);
}

The list is type safe, i.e. trying to insert an element of another type than it was decalred for would fail to compile. The list is also safe with respect to which field it uses to link nodes, which is very useful when elements can be in multiple lists using several oc_typed_list_links fields.

This update introduces major changes in the distribution model of both the Orca tooling and Orca apps:

• Orca apps are now bundled as .orca files that contain only the WebAssembly modules and the app’s resources. (You can still create standalone apps if needed by passing the appropriate flag to the CLI tool.)
• The runtime, SDK and tooling are distributed as a single package (an Orca.app bundle on macOS, and an application folder with all executables, DLLs, etc on Windows). This serves both as the launcher for .orca apps, and as the tooling to bundle apps.
• The launcher and tooling can be invoked from the CLI, e.g. on macOS if you added Orca.app/Contents/macOS to your PATH environment variable, you can then run orca bundle --name MyApp main.wasm to bundle an app, and orca run MyApp.orca to run it.
• Starting Orca without argument or double-clicking the app executable will open a launcher GUI. This is still a work in progress and is disabled by default for now.

This update also introduces several related changes to the following APIs or internal systems, including:

• More complete file API to support filesystem operations needed by the new tooling. This includes functions to create/remove/copy temporary or persistent files and directories.
• Modified UI layout to support layouting needs of the launcher UI, and fix some quality of life issues.
• The interface definition files used to generate bindings of native functions to wasm imports were combined and updated to use the same schema as the API description files used to generate the documentation. The was we bind wasm imports is also simplified to put all the “host calls” in one place and avoid excessive marshalling of structs.
• Fixes to invalid wasm modules handling (pass all assert_invalid tests).
• Subprocess API to support the launcher (native-only).
• CLI argument parsing helper for the tooling.
• Fixes to path handling.
• count in oc_list
• Testing helpers and updated tests.

I made some modifications to the layout algorithm for our user interface system. There’s still some polish and testing to do so this is not merged yet, but here are the main changes in flight:

• Simplified the relax attribute, which specifies by which amount a box can be shrunk when there’s not enough space in its parent.
• Boxes can now grow to fill remaining space in the parent, following the same logic as shrinking.
• Added minimum and maximum size parameters to the size attribute.
• Split the floating attribute to a position attribute (which controls how a box is positionned) and a footprint attribute (which controls how a box takes space in its parent).
• Added a wrap attribute which allows wrapping children boxes to newlines when they exceed their parent’s size in the main layout axis. This part required substantially refactoring the layout algorithm, as you can’t treat each axis completely separately anymore, and contraints in one axis can invalidate computations you did in the other.

I also started a small experiment on a “code canvas” where you can arrange cards, that could later hold code editors for individual functions or type declarations, or graphical widgets to display and manipulate live data. Depending on what I find working on this, it might become a basis for exploring code, call graphs, and variables in the integrated debugger, and/or to directly modify running Orca apps.

• One goal of this experiment is to see how we can allow relatively freeform placement of cards while minimizing the need for manual re-arrangements. For example, when you open a new card on the canvas, other cards are moved out of the way to avoid overlap, while trying to minimize the distance from their original position.
• Another goal is to find lightweight ways of grouping cards to form “islands” that can be moved/closed/recalled/operated on as a whole. Here cards automatically form groups when they’re “close enough” to each other, and groups that are close to each other automatically get merged into a single group. We also want groups to stay “stable” when moving a card or another group through them, while still showing what groups would be formed at each point along the path if the mouse was released.

Here is an early proof of concept of the spacing and grouping behavior:

In september I have been working on the architecture for “bare” Orca apps (as opposed to standalone apps that are bundled with the runtime executable). In this model, the app is distributed as a zipped folder containing the app’s resources and WebAssembly modules, and run by a trusted runtime on the user’s machine.

This also changes the distribution of the Orca environment itself, as we now have three components to distribute:

• A launcher, which can download and run apps from URLs, and also provides access to a local library of cached apps. It also handles “open” actions on bare apps (i.e. it will run app files when you double-click on them).
• The Orca runtime proper. This needs to be split from the launcher because we also want to allow bundling standalone applications, and we don’t want to bundle the whole launcher with them. It can be run by the launcher, which passes the app’s file as a command line argument. In the case of a standalone app, it is run with no argument, and finds the app’s file relative to itself. The runtime then decompresses the app’s file in a temporary folder, loads the main WebAssembly module, and starts running the app.
• The SDK and tooling needed to develop new apps for the platform. This comprises header files for the Orca APIs, a WebAssembly blob for the support library, and the commands to bundle WebAssembly modules and resources into bare or standalone applications.

Conveniently, this can be distributed as a single application which contains these three components. By default, this application opens as the launcher, but it can also be called from the command line to give access to the developper tooling and SDK. For example, on Windows you could double click on Orca.exe to open the launcher, but you could also use ./Orca.exe bundle --name MyApp main.wasm to package main.wasm into a new Orca app.

In the process of working on this, I started implementing a few utility APIs that will also become part of the SDK:

• A versatile command-line parser with subcommands, short and long options, optional/repeated/multiple argument options, automatic help messages, etc.
• A subprocess API, which allows spawing child processes, communicating with them, waiting for them, collecting their outputs and exit status. This is native-side only for now, but it will be a basis for a capability-guarded API that you can use from Wasm to call into external programs.
• I also reworked the file IO API and added a number of file system interfaces, like creating directories, copying and removing files or directory trees. Along with Reuben’s work on directory listing, this gives a more complete set of functions to work with the host file system.

That’s all for now! see you next time, and take care ~

Our custom WebAssembly backend is now merged, and replaces our previous third-party interpreter (wasm3). This gives us freedom to design the interpreter in a way that supports our integrated debugger, and will also allow us to experiment with interpreter tech and WebAssembly extensions.

The new backend is called Warm, for WebAssembly Register Machine. Upon loading a module, it performs validation and compiles its functions to a custom unstructured control flow, register-based bytecode (as opposed to the WebAssembly structured control flow stack machine model). This eliminates stack book-keeping by encoding the source and destinations operands in the instructions themselves, and unfolds control flow down to simple jumps. The virtual machine then is a dead-simple loop-over-switch interpreter.

The backend passes the core wasm testsuite. It is currently slower than wasm3 because it doesn’t use threaded code yet. So one obvious avenue for improvement is to do that, either with tail calls (what wasm3 does) or computed gotos (my preference for debuggability and readability reasons, but we’ll have to try). Another one is to do a better job at register allocation, as I chose the very simplest strategy (an infinite register file and a freelist) for this first iteration. Another thing I want to try is to have a simple template JIT, as an intermediate step towards more optimized JIT tiers.

We just landed a file enumeration API in the Orca WebAssembly SDK. This allows apps to query for all files within a given directory, which is performed in a single native call, and then iterate through them on the wasm side. An example with the C API:

void printFilesInDir(oc_arena* arena, const char* path)
{
    oc_file dir = oc_file_open(path, OC_FILE_ACCESS_READ, OC_FILE_OPEN_RESTRICT);
    if (oc_file_last_error(dir)) {
      return;
    }
    oc_file_list entries = oc_file_listdir(arena, dir);
    if (oc_file_last_error(dir)) {
      return;
    }
    oc_file_list_for(entries, elt) {
        oc_log_info("Found '%.*s' with type %d\n", oc_str8_ip(elt->basename), elt->type);
    }
}

There are still a few holes in the file IO API, but we’ll be working to flesh that out in the coming months.

In addition to regular unit/integration tests, Orca has test apps that have a special oc_on_test() hook. The runtime runs this hook and checks the exit code, relying on the app to log any failures. This can be used to test specific parts of the Orca SDK from the wasm side. These test apps are built and bundled the same way as normal Orca apps. Some recent improvements to the build system now allow bundling apps without requiring a system-wide install of the orca CLI tool. Now the CI can build and run the wasm tests just like the native tests, giving better visibility on potential SDK API breakage.

I fleshed out our Github Sponsors page and added some monthly tiers. They come with symbolic rewards, from getting a distinguishing role in the Orca Discord server, to having your name and link to your homepage listed at the bottom of the landing page of the Orca website. Higher tiers are a good option for companies that want to support Orca through Github Sponsors, and include some amount of consultancy. I also plan on later adding access to private repos to specific tiers, to share bonus content with sponsors (think tutorials, screencasts, or advanced example programs).

If you want to help our work, now is the perfect time to do so :) Thanks for your support!

Martin

We just merged the new Zig build system to replace our old Python build script. Reuben wrote about it here.

The Orca debugger is shaping up. It can pause on breakpoints and step in and out through source code or instructions, navigate call frames, and inspect most variables in unoptimized code.

I was invited on Zig Showtime to talk about Orca:

Our WebAssembly Debugger can now single step through source lines.

I started working on an integrated debugger for Orca. Here’s its very first steps, moving through the bytecode. You’ll notice it’s not WebAssembly bytecode. Instead our custom backend compiles WebAssembly to a register-based machine with jumps for more efficient interpretation (and later JIT-ing).