The slowest part is always emulating the MMU, as that adds a 2-3× overhead to every memory access. In that sense ARM Sv39 is not a great choice.
Usually emulators use a direct-mapped cache (basically a software TLB).
In this case though, since you control the kernel, it could use something like the MIPS software-managed TLB of the olden days, with bespoke paging data structures and fast-path page fault handlers, perhaps also using PowerPC Inverted Page Tables as inspiration. (and eg. changing page allocation strategies to constrain pages in the lower address space region to simplify cache look up, and something to prevent cache set conflicts).
Given that the working set is small tho in this use case, eg. just a kernel-managed direct-mapped `page_number[MAX_PAGES] -> base_address` table is sufficient and fast. (with some trick for page faults, also who needs granular RWX access permissions...)
Slow, but that’s expected. Otherwise ran Postgres unmodified. Not sure if this will be useful for anything serious, but demos of real Linux software have become zero friction today. Amazing.
Awesome! I'd love to see userspace work in wasm too though. Someone once did a PoC, but scheduling processes was a pain - AFAIK noone knows how to force preemption?
Wow, what a great nerd snipe for a Sunday morning.
I launched it, installed Claude CLI, had it poke around, then I installed Docker and had to poke a bit to get it running, but I got the hello-world container to work!
Eventually… But I'd guess for now memory limits, the size of a dev env, and overhead when mounting in whatever files you actually need access to make that less attractive than it sounds.
I doubt anything practical would come from this project*. Which is fine. Doing something for educational purposes, or even just for fun, is reason enough in my opinion.
* all the use cases people have speculated about feel more cumbersome and/or less reliable than established methods already used in the industry.
That's the thing, isn't it. Even if you did it manually, people would just assume it was slop.
In fact, even if you spent several years doing it manually, and then AI came along that could do a better job in mere minutes. (Not that I'd know anything about that...)
I guess it turns out we were always more impressed with the suffering than with the end result.
A project of this complexity essentially _would guarantee_ you an L8/9 position at any FAANG, _if_ FAANG hiring wasn't overrun by rampant nepotism and incompetence.
Another way to run whole operating systems in the browser is to put a second JIT on top of the browser's: an instruction set emulator that compiles guest code to JavaScript, which V8 then compiles to native code.
Here is the original PDP-7 Unix running that way. Log in as dmr password dmr and type ls and cat the source code to his PDP-7 assembler in PDP-7 assembler and B compiler in B. The b_readme tells how to build, assemble, and run B code.
The higher level JavaScript targeting JIT speculates and inlines based on the code that's actually running, so if you generate JavaScript it can optimize well, it does a lot of the work for you. Drag the speed slider all the way to the right to MAX and in the upper right corner it shows the performance, which is over 250x real time on my M4 Mac.
Vanessa Freudenberg made this argument for the SqueakJS JIT here on HN in 2023: "That would potentially give us more speed than even WASM." and "My plan is to do as little as necessary to leverage the enormous engineering achievements in modern JS runtimes." (https://news.ycombinator.com/item?id=38774277)
The emulator's CPU is built on the design of Korz, David Ungar's successor to Self (with Harold Ossher and Doug Kimelman at IBM Research). Self's adaptive compilation is where V8's techniques came from, so the lower JIT gets fed what it was built for. Click on the SLOTS and KORZ buttons to inspect the JS code it's generating and running.
The same PDP-7 runs Mitch Bradley's PDP-7 Forth, PIXIE from 1971, and Unix. There's also a Korz Apple ][ emulator running Little Computer People, and I measured it against microM8, a good Apple ][ emulator written in Go, compiled to WebAssembly. In multiples of a real Apple ][+:
That's 15 to 52 times faster, but most of the gap isn't WASM itself. microM8 builds the picture inside every instruction, calls memory and devices through Go interfaces, and carries the Go runtime, scheduler and garbage collector on the same thread as the CPU loop, in a 41 MB module. A plain switch emulator written so V8 can optimize it already runs at 130 million cycles a second, and it beats my JIT on the DOS boot and on Applesoft. The JIT wins when a block of guest code stays put and repeats: it turns that block into a JavaScript function, and V8 specializes it. A WASM optimizer can't do that step for you, because to it the guest's opcodes are just data.
I think the most interesting point this illustrates is that Korz is a great way to write Just In Time compiling emulators that run extremely efficiently in the browser, in way that can achieve much higher performance than WASM. And Korz is generally useful for a lot more than just emulators.
Claude Shannon built Marvin Minsky an actual physical "Useless Machine", which Marvin preferred to call the "Ultimate Machine", whose only purpose was to turn itself off:
Here is a fictional Ultimate Machine PDP-7 device emulator, with a demo written in Forth that draws with turtle graphics on the Type 340 vector graphics display:
Here is a fictional Data Hand PDP-7 device emulator, that uses webcam video instead of an actual glove, also with a demo written in Forth that draws with turtle graphics on the Type 340 vector graphics display:
Now here's a Forth program orchestrating both of those fictional emulated fictional PDP-7 devices, that lets you turn the Ultimate Machine off with your own hand:
Usually emulators use a direct-mapped cache (basically a software TLB).
In this case though, since you control the kernel, it could use something like the MIPS software-managed TLB of the olden days, with bespoke paging data structures and fast-path page fault handlers, perhaps also using PowerPC Inverted Page Tables as inspiration. (and eg. changing page allocation strategies to constrain pages in the lower address space region to simplify cache look up, and something to prevent cache set conflicts).
Given that the working set is small tho in this use case, eg. just a kernel-managed direct-mapped `page_number[MAX_PAGES] -> base_address` table is sufficient and fast. (with some trick for page faults, also who needs granular RWX access permissions...)
http://bellard.org/jslinux
https://joelseverin.github.io/linux-wasm/
WASM Wayland Compositor made by the same author
Totally! So we can run a browser inside Linux that runs in a browser.
EDIT: Looks like there is some networking support https://github.com/pithings/unkernel#add-a-network
It would be interesting to fake CPU architecture that is optimal for WASM.
I launched it, installed Claude CLI, had it poke around, then I installed Docker and had to poke a bit to get it running, but I got the hello-world container to work!
[1] https://www.destroyallsoftware.com/talks/the-birth-and-death...
I doubt anything practical would come from this project*. Which is fine. Doing something for educational purposes, or even just for fun, is reason enough in my opinion.
* all the use cases people have speculated about feel more cumbersome and/or less reliable than established methods already used in the industry.
Really, every single interesting project I see these days I'm completely unable to feel awe or curiosity since I just assume it was AI-made.
In fact, even if you spent several years doing it manually, and then AI came along that could do a better job in mere minutes. (Not that I'd know anything about that...)
I guess it turns out we were always more impressed with the suffering than with the end result.
https://www.destroyallsoftware.com/talks/the-birth-and-death...
Here is the original PDP-7 Unix running that way. Log in as dmr password dmr and type ls and cat the source code to his PDP-7 assembler in PDP-7 assembler and B compiler in B. The b_readme tells how to build, assemble, and run B code.
https://hyperties.org/cabinet/systems/pdp-7-unix/
The higher level JavaScript targeting JIT speculates and inlines based on the code that's actually running, so if you generate JavaScript it can optimize well, it does a lot of the work for you. Drag the speed slider all the way to the right to MAX and in the upper right corner it shows the performance, which is over 250x real time on my M4 Mac.
Vanessa Freudenberg made this argument for the SqueakJS JIT here on HN in 2023: "That would potentially give us more speed than even WASM." and "My plan is to do as little as necessary to leverage the enormous engineering achievements in modern JS runtimes." (https://news.ycombinator.com/item?id=38774277)
The emulator's CPU is built on the design of Korz, David Ungar's successor to Self (with Harold Ossher and Doug Kimelman at IBM Research). Self's adaptive compilation is where V8's techniques came from, so the lower JIT gets fed what it was built for. Click on the SLOTS and KORZ buttons to inspect the JS code it's generating and running.
The same PDP-7 runs Mitch Bradley's PDP-7 Forth, PIXIE from 1971, and Unix. There's also a Korz Apple ][ emulator running Little Computer People, and I measured it against microM8, a good Apple ][ emulator written in Go, compiled to WebAssembly. In multiples of a real Apple ][+:
That's 15 to 52 times faster, but most of the gap isn't WASM itself. microM8 builds the picture inside every instruction, calls memory and devices through Go interfaces, and carries the Go runtime, scheduler and garbage collector on the same thread as the CPU loop, in a 41 MB module. A plain switch emulator written so V8 can optimize it already runs at 130 million cycles a second, and it beats my JIT on the DOS boot and on Applesoft. The JIT wins when a block of guest code stays put and repeats: it turns that block into a JavaScript function, and V8 specializes it. A WASM optimizer can't do that step for you, because to it the guest's opcodes are just data.I think the most interesting point this illustrates is that Korz is a great way to write Just In Time compiling emulators that run extremely efficiently in the browser, in way that can achieve much higher performance than WASM. And Korz is generally useful for a lot more than just emulators.
Korz dispatch for emulators, in TypeScript:
https://github.com/SimHacker/WillWrightShowForFood/blob/main...
What Korz is:
https://github.com/SimHacker/moollm/blob/main/designs/korz/R...
Benchmarks:
https://github.com/SimHacker/WillWrightShowForFood/blob/main...
Go and TypeScript: The Wasm build waits:
https://github.com/SimHacker/WillWrightShowForFood/blob/main...
The demo that goes with the talk:
https://www.youtube.com/watch?v=I-Rf3XP_D0g
Forth Day talks, Mitch Bradley on PDP-7 Forth and me on the PDP-7 and Apple ][ emulators:
https://www.youtube.com/watch?v=k2pwJ294r8o
PIXIE and FORTH on PDP-7 Cabinet Emulator with Type 340 Vector Graphics Display:
https://www.youtube.com/watch?v=lo8kdY-5i6c
More about PDP-7 Forth:
https://github.com/SimHacker/WillWrightShowForFood/blob/main...
More about Korz:
https://github.com/SimHacker/moollm/blob/main/designs/korz
Alan Kay thinks about Claude Shannon's way of dealing with noisy channels every day:
https://www.youtube.com/watch?v=Cjntrqhn8pk
Claude Shannon built Marvin Minsky an actual physical "Useless Machine", which Marvin preferred to call the "Ultimate Machine", whose only purpose was to turn itself off:
https://en.wikipedia.org/wiki/Useless_machine
Here is a fictional Ultimate Machine PDP-7 device emulator, with a demo written in Forth that draws with turtle graphics on the Type 340 vector graphics display:
https://hyperties.org/cabinet/devices/ultimate-machine/
VPL developed an actual physical Data Glove device that Jaron Lanier used to perform virtual musical instruments in virtual reality:
https://www.youtube.com/watch?v=E1q9zDSAQEw
Here is a fictional Data Hand PDP-7 device emulator, that uses webcam video instead of an actual glove, also with a demo written in Forth that draws with turtle graphics on the Type 340 vector graphics display:
https://hyperties.org/cabinet/devices/datahand/
Now here's a Forth program orchestrating both of those fictional emulated fictional PDP-7 devices, that lets you turn the Ultimate Machine off with your own hand:
https://hyperties.org/cabinet/devices/ultimate-hand/
Claude usually doesn't speak like this. Sounds like an engineer that wants to share things, perhaps polished with AI