Compiler change

A couple of days ago I got really frustrated by trying to use tinygo as my “client side” go compiler. This particular frustration was that it does not support the full go reflection API. This is probably fine for the their intended audience, but since I use code generated by the protobuf->go generator and it uses reflection a lot, I was seriously hosed. I tried a few things to work around it, but eventually just started swearing and got up from my desk for more tea.

Similarly, I was cursing the name of tinygo because I was irritated about the calling conventions it was using. Tinygo has some pretty weird ideas about return results. Return results are placed “normally” on the stack if they meet some criteria; this criteria, in the case of WASM-in-browser, is that they basically are a 32 bit integer (I32), 32 bit float (F32), or 64 bit float (F64). No 64 bit integers, no structures. In the other cases, it passes a pointer to a return area in the first parameter to the called function. Now, since it is a stack machine the return result in first parameter makes a little bit of sense since it would be the last thing you push onto the stack but it still seems weird to me.

After the gnashing of teeth and tea above, I decided to try using the normal go compiler (1.19.1) with the GOOS=js GOARCH=wasm flags turned on. I had chosen Tinygo before because it produces WASM that is vastly smaller than the normal go compiler. It produces a “hello, world” that is something like 10k because it doesn’t include the go runtime, as the normal go compiler does. That runtime is measured in megabytes. With my unlinking efforts, I might be able to get the minimum size of a parigot program down below 1 MB mark.

It didn’t start out well because the startup sequence of the any code compiled with the go compiler’s WASM support goes through some javascript shenanigans to get started. Since I didn’t have the javascript runtime machinery in parigot, I decided to try to emulate just enough of it to let the program get started. I reasoned that no client program of parigot was going to actually exercise the javascript interface, at least not now, and so all I had to do was enough to get the darn thing started.

Richard Musiol wrote the compiler backend for WASM for the go team, as well as the javascript runtime. The trickery that he uses down in the bowels of that runtime took me a lot of time to understand because it is quite javascript-specific. I can hardly blame him, it’s a javascript runtime! Anyway, the key nugget to understanding the runime is this: He keeps all the javascript-side objects in a table and hands out the index of the table to WASM code instead of the object itself. Giving WASM the javascript object would be pretty useless anyway. Then—under the proverbial covers—he does an encoding that “hides” the id of the object inside a floating point number. Basically, he uses the uppermost 16 bits being the NaN value to then use the other bits to hide the index! So, the 64 bit js “float” actually is hiding two things inside it, a typecode (bits 32-35) and an index into the table (bits 0-31).

This encoding was kinda irritating at first, but I see why he did it. Javascript only has 64 bit floating point numbers, no ints. (Note, the INT64 type in WASM can’t be represented exactly in a 64 bit floating point, so that its own problem….) His encoding works something like this, given a 64 bit entity that might be a js “object” or a number:

1. Are all the bits zero? If so, this “undefined” in javascript land.
2. Do the bits form a legit floating point number? If so, this is the number representing itself.
3. Since the the bits are a “NaN” in floating point, mask off the upper 16 bits, and pull two ints out of the rest. These are the JS object’s type code and index in the object table.

It actually didn’t take that long to build the code that could emulate the interface to his javascript-connected go binaries. I probably overdid it, to be honest, because I started seeing how it had to work after looking at his api and figuring out the encoding above. It took only about 4-5 hours before I got a WASM program to start up without the javascript support he had written. I did get to thinking at some point that this strategy (emulating his JS world) might actually be a way to somehow run parigot programs in the browser, yet be linked against my faux-javascript runtime and not using his stuff. Not sure if this would work but it was interesting thing to think about.

So, now I am using the standard go compiler to compile client programs that are built against parigot’s API. There are maybe 5 calls that the go code makes at startup that it expects to be implemeted by javascript and then it’s just “run baby run”.