The WebRacket language is a subset of Racket that compiles to WebAssembly

157 pointsposted 20 days ago
by mfru
(github.com)

38 Comments

soegaard

16 days ago

Hi All,

It's still early days for the WebRacket project.

Racket is a huge language, so be patient wrt features.

To keep motivation high I decided to implement a subset that can be used to built practical applications - and then extend the supported features from there. Hopefully, this strategy will also lead to some early adopters that can help me prioritize which features to add.

Some features are simply "more of the same". In this category falls more types of hash tables. Supporting bignums are also a matter of just doing it.

Other features require more work. I have already done some work on implementing modules in terms of linklets. When linklets/modules work, we can reuse the existing implementation of regular expressions.

Adding continuation marks and delimited continuations require adding a CPS-pass. This is certainly doable. Postponing it has been great though. Having a direct style compiler means the generated code follows the structure in the input source code. And that makes debugging easier. Now that bugs have become rarer, it makes sense to look at CPS.

Enjoy.

/Jens Axel

michaelsbradley

16 days ago

Are there any architectural similarities or crossover with Hoot (Guile Scheme -> Wasm) or are you taking a completely different approach?

soegaard

16 days ago

I am using a similar representation of immediates as Hoot and wasm_of_ocaml.

The representation is explained here:

https://github.com/soegaard/webracket/blob/main/compiler.rkt...

Internally the compiler uses a series of passes implemented using Nanopass.

    (generate-code
     (flatten-begin
      (closure-conversion
       (anormalize
        (categorize-applications
         (assignment-conversion
          (α-rename
           (explicit-case-lambda
            (explicit-begin
             (convert-quotations
              (infer-names
               (flatten-topbegin
                (parse
                 (unexpand
                  (topexpand stx)))))))))))))))
The code generator is inspired by "Destination-driven Code Generation" by Dybvig, Hieb and Butler. There are some differences however. The code generator in the paper generates "flat" code (assembler) whereas I generate nested Web Assembly instructions.

This approach generates reasonable code without having to implement a register allocator. Also, I believe I saw a Wasm to Wasm compiler that improved register allocation (maybe it was a switch for wasm-tools?).

If (when?) WebRacket becomes a success, we can always switch out individual passes.

titzer

16 days ago

> Adding continuation marks and delimited continuations require adding a CPS-pass.

Have you considered targeting the stack switching proposal?

soegaard

16 days ago

Maybe. My main problem is to get light-weight support for continuation marks.

If I need a CPS-pass for continuation marks, I might as well use it for continuations as well.

It would be great if it were possible to avoid a CPS-pass though.

titzer

16 days ago

With stack-switching you won't (shouldn't?) need a CPS pass.

soegaard

16 days ago

I need to study the stack-switching proposal in more detail.

However, I don't see an obvious way of attach and probe continuation marks to the continuations (including the current one).

I am not an expert in continuation marks, so I'll just link to this presentation by Matthew Flatt (which you probably already know).

https://github.com/WebAssembly/meetings/blob/main/stack/2021...

publicdebates

16 days ago

I read the WASM spec and became somewhat of an expert in it for the purpose of eventually designing a low-level language specifically for wasm, to be the most efficient and lowest-level language you could possibly have for wasm, and then gradually add high level concepts into it to make it more convenient. Kind of like building C on top of asm, and then gradually evolving it into C++. That project never came about though due to lack of community interest and time on my part, but I like seeing languages that take a fresh look at how to integrate with wasm using novel techniques to aim for maximal efficiency.

spankalee

16 days ago

I'm working on something like this right now, targeting WASM GC. I started with functions, basic numeric types, arrays, and structs. Then added blocks, control flow, and strings. Then interfaces, mixins, classes, and extension classes. It's now something like a statically typed mashup of TypeScript, Swift, and Dart, all done in the best way I could figure out specifically for WASM GC.

It's been a really fun side project.

I do think there is a market for something like this - optimizing an existing language for WASM, or ephemeral networked code delivery in general, can be really hard. And a statically typed, WASM-oriented, very familiar high-level language, that can give very good static errors, and quickly run in a secure sandbox might be a good target for LLM-generated code.

Imustaskforhelp

16 days ago

> to be the most efficient and lowest-level language you could possibly have for wasm, and then gradually add high level concepts into it to make it more convenient. Kind of like building C on top of asm, and then gradually evolving it into C++. That project never came about though due to lack of community interest and time on my part, but I like seeing languages that take a fresh look at how to integrate with wasm using novel techniques to aim for maximal efficiency.

I wonder how much efficient would such wasm be compared to native itself theoretically?

I really like libriscv as well, what are your thoughts on libriscv and now the recent project created by legendary fwsgonzo about looongarch.

Although I would say that libriscv/loongarch are more focused on trying to start the fastest as much as possible instead of pure performance (for which if atleast for the purpose of sandboxing, fwsgonzo has also created tinykvm which is really quite close to native performance)

brabel

16 days ago

I had the exact same idea around 5 years ago, and actually built a language. But without a stdlib it’s a bit useless and as there was no component model at the time it was impossible to have one.

koolala

16 days ago

A SIMD ECS language would probably be that today and be more modern than C.

iainctduncan

16 days ago

People into this sort of stuff might be interested to know that s7 Scheme also runs really well in WASM. It's 100% ansi C and uses its own GC, so getting it compiled and running in WASM is very simple. I use it in both my audio project (Scheme for Max, an s7 interpreter in Max/MSP) and in a browser based set of music practice tools I am working on as a solopreneur. It's fantastic to be able to write the music theory engine in Scheme instead of JS.

An example page (not mine) of it running in the browser is here: https://actondev.github.io/s7-playground/

I really hope the racket effort gains traction too! Excited to see this. In comparison, s7 is much more minimal. Though this also means the FFI is dead simple too, so extending it and bridging to js functions is much easier, and everything in s7 is available now in WASM - CL macros, keywords, hashtables, first class environments, continuations, etc

gcr

16 days ago

Is there any shared lineage between this and Whalesong, a previous Racket->JS compiler?

Of course both projects have the same maintainer if I recall, Jens Axel Søgaard is a rockstar :)

soegaard

16 days ago

No, there is nothing in common with Whalesong.

Whalesong used the built-in bytecode compiler and compiled the bytecode to JavaScript. Reusing the bytecode compiler is in principle a good idea - but each time the bytecodes are changed, Whalesong needs to be updated.

And after the move to Chez Scheme as backend, the bytecode compiler is no longer a part of the main compilation path.

brabel

16 days ago

JVM languages always target bytecode because it’s much simpler and stable than Java as a language. It almost never changes and when it does it normally won’t break code generation since it’s only adding type system information, for example, as with records.

Is Racket bytecode different?

soegaard

16 days ago

> Is Racket bytecode different?

Changes to the bytecode representation were indeed rare also in Racket.

The Whalesong project was written as part of a dissertation - and when people graduate and get jobs, projects are often abandoned.

noelwelsh

16 days ago

I love this. Racket is the future we were promised.

neilv

16 days ago

Speaking of prolific Racketeers... Noel! Just an hour ago, on a walk, I was thinking, "I should work through that one LLM book, and implement it in Racket." (But have started job-hunting, so will probably be Python.)

dunham

16 days ago

Which one LLM book?

I've got so much other stuff I'd rather learn and code I'd rather write (C/wasm backend for my language), but I've also started job hunting and probably should understand how this latest fad works. Neural networks have long been on my todo list anyway.

titzer

16 days ago

I noticed the --expose-gc. Does this mean it's using the (now standardized) Wasm GC feature?

soegaard

16 days ago

Yes. I am following the Scheme tradition of representing immediate values as tagged pointers. And (ref i31) is the obvious choice when using WebAssembly. I am happy you and the team added GC to WebAssembly.

Details on the representation.

https://github.com/soegaard/webracket/blob/main/compiler.rkt...

I am more or less only using the linear memory for the JavaScript FFI. FASL-encoded values are passed back and forth to JavaScript.

KarenDaBass

16 days ago

Is a (Web)Racket engineer a racketeer?

gcr

16 days ago

They're certainly a schemer. :-)

dfajgljsldkjag

16 days ago

It is interesting to see another language target WebAssembly especially one like Racket. The fact that it compiles to a subset of the language limits its utility right now. I think it is a neat proof of concept but it needs full language support.

gcr

16 days ago

For folks curious about the supported language subset, here's a summary:

- Modules aren't implemented yet, but are high on the list

- Continuation-based control flow isn't supported yet, including break and promises. Tail calls are supported though.

- No support for complex numbers, bignums, weak hashtables, immutable hashtables, prefab structs, regexp, or file I/O support

Most of the rest of racket/base should work, according to the README. There's also a FFI for javascript, including bindings for Math, DOM, Canvas, MathJax, XTermJS, and JSXGraph. Overall feels like you can use most of the language right now unless you need odd Racket-isms or call/cc.

volemo

16 days ago

As far as I know compiling full Racket to WASM is impossible because of continuations.

soegaard

16 days ago

I wouldn't say compiling full Racket to WebAssembly is impossible. But I think the consensus is that one can't add a WebAssembly backend to the compiler in the same manner as the x86 and arm backends. These backends manipulate the stack in ways WebAssembly prohibits.

This forces an Racket implementation to make continuations explicit. And that will most likely mean a WebAssembly backend will be slower than the native backends.

davexunit

16 days ago

It's possible to do continuations on Wasm now and it will be made even easier by the stack switching proposal.

kg

16 days ago

You could probably model continuations using the wasm GC feature since you can then pass around function references and strongly typed continuation objects, but making it work certainly wouldn't be trivial.