A real Linux kernel, compiled to WebAssembly

(github.com)

143 points | by DonAtimator 4 days ago

24 comments

  • jaen 41 minutes ago
    The slowest part is always emulating the MMU, as that adds a 2-3× overhead to every memory access. In that sense ARM Sv39 is not a great choice.

    Usually emulators use a direct-mapped cache (basically a software TLB).

    In this case though, since you control the kernel, it could use something like the MIPS software-managed TLB of the olden days, with bespoke paging data structures and fast-path page fault handlers, perhaps also using PowerPC Inverted Page Tables as inspiration. (and eg. changing page allocation strategies to constrain pages in the lower address space region to simplify cache look up, and something to prevent cache set conflicts).

    Given that the working set is small tho in this use case, eg. just a kernel-managed direct-mapped `page_number[MAX_PAGES] -> base_address` table is sufficient and fast. (with some trick for page faults, also who needs granular RWX access permissions...)

    • sandos 37 minutes ago
      I learned this from inspecting the code that TCG on QEMU generates... was very surprised when a load ballooned into tens of instructions!
  • solyin 1 hour ago
    Fabrice Bellard beat you to it, with jslinux in 2011.

    http://bellard.org/jslinux

    • dansquizsoft 1 hour ago
      It’s hard to know which one came first really…
    • exe34 54 minutes ago
      We don't really count that, if it's technically possible, fbellard has done it already. We care about what mere mortals do.
      • Riemer 45 minutes ago
        I know you jest but pithings used LLMs in this project
  • ivaniscoding 17 minutes ago
    I was expecting the architecture to be riscv64 but it’s aarch64 which is more convenient!
  • baq 1 hour ago
    Slow, but that’s expected. Otherwise ran Postgres unmodified. Not sure if this will be useful for anything serious, but demos of real Linux software have become zero friction today. Amazing.
  • d33 3 hours ago
    Awesome! I'd love to see userspace work in wasm too though. Someone once did a PoC, but scheduling processes was a pain - AFAIK noone knows how to force preemption?

    https://joelseverin.github.io/linux-wasm/

    • vinnymac 2 hours ago
      https://wland.vercel.app/

      WASM Wayland Compositor made by the same author

    • DeathArrow 2 hours ago
      >Awesome! I'd love to see userspace work in wasm too though.

      Totally! So we can run a browser inside Linux that runs in a browser.

  • rickcarlino 2 hours ago
    I could see this being useful if there was a TUN/TAP device that talks to a backend server over websockets.

    EDIT: Looks like there is some networking support https://github.com/pithings/unkernel#add-a-network

  • Tepix 45 minutes ago
    Why would it emulate aarch64? Because Linux supports it?

    It would be interesting to fake CPU architecture that is optimal for WASM.

    • baq 12 minutes ago
      Packages just work.
  • kator 1 hour ago
    Wow, what a great nerd snipe for a Sunday morning.

    I launched it, installed Claude CLI, had it poke around, then I installed Docker and had to poke a bit to get it running, but I got the hello-world container to work!

  • splittydev 1 hour ago
    Doesn't work in Safari: Failed to load webkit-masked-url://hidden/
    • splittydev 1 hour ago
      Seems like the Noir extension was the issue. Created strange CSP violations I've never seen happening anywhere else before.
  • hizlikovboy27 1 hour ago
    Could this eventually be used to run isolated development envs. directly in the browser without needing a remote server for each session?
    • Tepix 44 minutes ago
      Not just in the browser, wasm is also a sandbox for the server.
    • krautsauer 1 hour ago
      Eventually… But I'd guess for now memory limits, the size of a dev env, and overhead when mounting in whatever files you actually need access to make that less attractive than it sounds.
    • baalimago 1 hour ago
      Sounds like you'd be interested in https://www.wasmer.sh/
  • momojo 2 hours ago
    This sounds cool. Serious question, what are the applications?
    • jrmg 6 minutes ago
      Education? Essentially a linux box anyone can just bring up and use Python etc.
    • microbiont 1 hour ago
      This humorous talk [1] from 2014 outlines it pretty well, skip to 18:40 in the video.

      [1] https://www.destroyallsoftware.com/talks/the-birth-and-death...

    • hhthrowaway1230 2 hours ago
      sandboxes, throw away environments, playgrounds, clusters.. plenty I think
    • szundi 1 hour ago
      [dead]
    • hnlmorg 1 hour ago
      Looks good on a CV.

      I doubt anything practical would come from this project*. Which is fine. Doing something for educational purposes, or even just for fun, is reason enough in my opinion.

      * all the use cases people have speculated about feel more cumbersome and/or less reliable than established methods already used in the industry.

      • andai 1 hour ago
        "Told Claude to build cool thing (2026)"
        • zdimension 1 hour ago
          Finally a modern contender to "for sale: baby shoes, never worn".

          Really, every single interesting project I see these days I'm completely unable to feel awe or curiosity since I just assume it was AI-made.

          • andai 19 minutes ago
            That's the thing, isn't it. Even if you did it manually, people would just assume it was slop.

            In fact, even if you spent several years doing it manually, and then AI came along that could do a better job in mere minutes. (Not that I'd know anything about that...)

            I guess it turns out we were always more impressed with the suffering than with the end result.

        • rfgplk 1 hour ago
          A project of this complexity essentially _would guarantee_ you an L8/9 position at any FAANG, _if_ FAANG hiring wasn't overrun by rampant nepotism and incompetence.
  • v3ss0n 1 hour ago
    So we can build something like browserstack?
  • sllabres 1 hour ago
    • microbiont 1 hour ago
      He was even close on the timeline, putting the birth of the METAL operating system in 2028.
  • kuberwastaken 2 hours ago
    That's nice but can it run DOOM
  • d5lt5 1 hour ago
    Heavy kernel in a browser, and we've just buried electron because it was a heavy kernel in a browser.
    • andai 1 hour ago
      Finally we can have native software again ;)
  • born-jre 1 hour ago
    That was supposed to be comedy, but they did it in real
  • Waraqa 3 hours ago
    If I write any app name, it will automatically start apt-get install that app. Ctrl + C doesn't stop it. Are there different shortcuts?
  • muterad_murilax 2 hours ago
    Is it possible to hide the status bar at the bottom?
  • loftsy 3 hours ago
    Seems to me that Replit should be using something like this rather than running your app remotely.
    • mkgiga 2 hours ago
      you need more than a Linux kernel to host a web app that can run all the software web developers want
  • sooraj_sj 1 hour ago
    how you managed to get this done ?
  • DonHopkins 3 hours ago
    Another way to run whole operating systems in the browser is to put a second JIT on top of the browser's: an instruction set emulator that compiles guest code to JavaScript, which V8 then compiles to native code.

    Here is the original PDP-7 Unix running that way. Log in as dmr password dmr and type ls and cat the source code to his PDP-7 assembler in PDP-7 assembler and B compiler in B. The b_readme tells how to build, assemble, and run B code.

    https://hyperties.org/cabinet/systems/pdp-7-unix/

    The higher level JavaScript targeting JIT speculates and inlines based on the code that's actually running, so if you generate JavaScript it can optimize well, it does a lot of the work for you. Drag the speed slider all the way to the right to MAX and in the upper right corner it shows the performance, which is over 250x real time on my M4 Mac.

    Vanessa Freudenberg made this argument for the SqueakJS JIT here on HN in 2023: "That would potentially give us more speed than even WASM." and "My plan is to do as little as necessary to leverage the enormous engineering achievements in modern JS runtimes." (https://news.ycombinator.com/item?id=38774277)

    The emulator's CPU is built on the design of Korz, David Ungar's successor to Self (with Harold Ossher and Doug Kimelman at IBM Research). Self's adaptive compilation is where V8's techniques came from, so the lower JIT gets fed what it was built for. Click on the SLOTS and KORZ buttons to inspect the JS code it's generating and running.

    The same PDP-7 runs Mitch Bradley's PDP-7 Forth, PIXIE from 1971, and Unix. There's also a Korz Apple ][ emulator running Little Computer People, and I measured it against microM8, a good Apple ][ emulator written in Go, compiled to WebAssembly. In multiples of a real Apple ][+:

      DEX/JMP loop:        Korz JIT 150x, Korz switch 128x, microM8 2.9x
      DOS 3.3 boot:        Korz JIT  38x, Korz switch  80x, microM8 2.5x
      Applesoft GOTO loop: Korz JIT  78x, Korz switch  93x, microM8 2.9x
    
    That's 15 to 52 times faster, but most of the gap isn't WASM itself. microM8 builds the picture inside every instruction, calls memory and devices through Go interfaces, and carries the Go runtime, scheduler and garbage collector on the same thread as the CPU loop, in a 41 MB module. A plain switch emulator written so V8 can optimize it already runs at 130 million cycles a second, and it beats my JIT on the DOS boot and on Applesoft. The JIT wins when a block of guest code stays put and repeats: it turns that block into a JavaScript function, and V8 specializes it. A WASM optimizer can't do that step for you, because to it the guest's opcodes are just data.

    I think the most interesting point this illustrates is that Korz is a great way to write Just In Time compiling emulators that run extremely efficiently in the browser, in way that can achieve much higher performance than WASM. And Korz is generally useful for a lot more than just emulators.

    Korz dispatch for emulators, in TypeScript:

    https://github.com/SimHacker/WillWrightShowForFood/blob/main...

    What Korz is:

    https://github.com/SimHacker/moollm/blob/main/designs/korz/R...

    Benchmarks:

    https://github.com/SimHacker/WillWrightShowForFood/blob/main...

    Go and TypeScript: The Wasm build waits:

    https://github.com/SimHacker/WillWrightShowForFood/blob/main...

    The demo that goes with the talk:

    https://www.youtube.com/watch?v=I-Rf3XP_D0g

    Forth Day talks, Mitch Bradley on PDP-7 Forth and me on the PDP-7 and Apple ][ emulators:

    https://www.youtube.com/watch?v=k2pwJ294r8o

    PIXIE and FORTH on PDP-7 Cabinet Emulator with Type 340 Vector Graphics Display:

    https://www.youtube.com/watch?v=lo8kdY-5i6c

    More about PDP-7 Forth:

    https://github.com/SimHacker/WillWrightShowForFood/blob/main...

    More about Korz:

    https://github.com/SimHacker/moollm/blob/main/designs/korz