Design and Implementation of Sprites

(fly.io)

65 points | by sethev 2 hours ago

11 comments

  • raro11 46 minutes ago
    Tried it. Docker wasn't preinstalled so asked Claude to do it and make sure it's running (supervisord or the likes isn't preinstalled).

    It neatly did so and "registered" it as a sprite service. Then I exited my session, waiting for the sprite to go idle, but I don't think it ever does.. Still have it active. Don't know how to idle it.

    Can't tell for sure if this means I'm losing credits as there is no billing usage shown anywhere.

    Also waiting for the moment where I can launch a sprite from another's checkpoint.

    • panarky 19 minutes ago
      There's no way to stop sprites from the CLI.

      Supposedly they auto-stop when inactive.

      But I've tried multiple times and they don't stop, and it's not just Docker that prevents them from stopping.

      I created a new sprite and installed ffmpeg. Then exited. Next day I run `sprite ls` and it's been running continuously for 23 hours.

      No way to tell if I'm being billed for it or not.

      And the per-hour pricing is extremely expensive.

      So for now it's `sprite destroy -s spritename`.

      Maybe I'll check it out again in a few months after the fly team has iterated on this a few times.

      • tptacek 14 minutes ago
        Sprites are active when:

        * They're servicing an incoming HTTP request.

        * You're interacting with a console.

        They're hair-trigger inactive otherwise. They don't bill CPU unless they're active. The idea is that there isn't really any uncertainty about when it's running; when you stop interacting with it it stops metering.

        This is a new shape for a cloud computing thingy and there'll be snags this week with it, but we don't make our money by billing people for stuff they don't want. We've always gone out of our way not to nickel-and-dime casual users and we're trying hard to find new ways to lean into that here.

        (Destroying a Sprite you're done with is a perfectly reasonable move; they're disposable.)

        • panarky 7 minutes ago
          No console activity, no HTTP requests, but it doesn't stop.

          No place in CLI or web UI that I can find to see how many minutes are charged.

              $ sprite ls
              Sprites in organization <redacted>:
          
              ┌────┬───────┬────────┐
              │NAME│STATUS │CREATED │
              ├────┼───────┼────────┤
              │duh │running│ 23h ago│
              └────┴───────┴────────┘
          
              Total: 1 sprite(s)
      • boundlessdreamz 14 minutes ago
        Yeah. My sprites never idled inspite of having nothing running in them and had to be destroyed. Ideally there should be two settings

        1. A timeout after the last console session is exited 2. Force idle using the CLI

    • ktaraszk 30 minutes ago
      Just use container-os as your runtime image: https://hub.docker.com/r/miget/container-os

      and you should be good

      • zrail 12 minutes ago
        Could you explain more about how this helps and how one might use a different runtime image on Sprites?
    • chrismccord 43 minutes ago
      We had a bug where some sprites would fail to properly suspend while entering their suspended state. You're not eating into credits so no worries there. We've been rolling out a fix across the fleet today so you should be seeing proper status soon.
  • roywiggins 1 hour ago
    I've been having fun with OpenCode's webui in a Sprite, set it up right in Termux and you can vibe code a website with a full backend from your phone. I have a Termux shortcut that sets up a port forward and pops open my web browser to the OpenCode webui. OpenCode can pick up the Sprite skill built into the containers so it can manage the instance itself (mostly setting up and destroying services).

    Of course you can do similar with a VPS, but this makes it very easy (and cheap!) to spin up an entire new machine with a public url and HTTPS whenever you want to.

    • adriand 49 minutes ago
      What's the option for iOS? I've tried installing ish and a-shell and can't get the sprites CLI installed on either of them.
  • benatkin 1 minute ago
    To me, this sounds more monolithic than containers. I think I'd like something less monolithic. However, those who like monorepos might be more quick to develop an interest in this. I could of course use containers within the MicroVM, which is what I really want anyways, because I want lighter weight containers than MicroVMs for sandboxes.

    While looking into giving fly another shot as a cloud provider even though I think it's still pretty much a commodity for me, I found an issue in Google: I searched for "fly.io sao paolo" and the title of the first result on fly.io is "Regiones · Fly Docs", translated from english to Spanish. While I find the translation in titles on Google annoying, I haven't often seen the characters messed up like this. I reproduced this in Incognito at this URL: https://www.google.com/search?hl=es&q=fly.io%20sao%20paolo

  • MrOrelliOReilly 1 hour ago
    I appreciate the Fly.io team’s enthusiasm and am optimistic this will mature into a product I’d pay for, but my initial impression was of a lack of polish.

    Documentation is sparse, or not even available? The API docs don’t tell you much about the service itself, and a Google search for docs returns an inaccessible website as the first result (https://docs.sprites.dev). Blog posts and forum threads and Claude skills shouldn’t be a substitute.

    The snappiness of the sprites is very cool and I can definitely see it integrating into future Claude Code workflows. But the lack of a base container images means you’re still doing setup work on the sprite before you can begin development. I understand the philosophy is that sandboxes should be persistent, but Claude Code sessions also work better when isolated from each other, so it’d be nice to have some precepts to get a workspace setup quickly (given agentic coding is clearly a target).

    I also found the CLI unintuitive but maybe that was just me!

    So very cool idea but left with the impression that the Fly.io team’s should have spent a couple weeks on polish before shipping.

    • tptacek 1 hour ago
      You're not wrong. The documentation actually had a hallucinated link to an Anthropic dependency in it when we shipped. Right now the attitude is mostly "if we have to document it extensively, we're doing something wrong". It's been in the works for awhile, with a small team, and we're just getting it out there right now.

      I've been needling Kurt for several months now that if we wait until it's polished enough that we don't see comments like this, we're doing it wrong.

      • mcpherrinm 22 minutes ago
        I'm sure this is a difference-of-learning or whatever, but I'm usually unwilling to try a product until I can understand it and how it works from the documentation
        • tptacek 20 minutes ago
          Understandable. Our current take is that there's not really much to know, and that the people this will really light up are good with that. Of course, we'll flesh out documentation!

          I'm really jazzed about this particular product as a product (I just really enjoy using it), but the post is mostly about how we built it, and deliberately not much about how best to use it.

  • boundlessdreamz 12 minutes ago
    Is there a way to copy files to/from the sprite? Couldn't see an option for that in the CLI
    • tptacek 7 minutes ago
      There's an FS API that shipped yesterday:

          /fs/read?path=X   | GET           | Read file (supports Range headers)           
          /fs/write?path=X  | PUT           | Write file (body = raw bytes)                
          /fs/list?path=X   | GET           | List directory 
          /fs/delete?path=X | DELETE/POST   | Delete files/dirs                            
          /fs/rename        | POST          | Rename/move                                  
          /fs/copy          | POST          | Copy files/dirs                              
          /fs/chmod         | POST          | Change permissions                           
          /fs/chown         | POST          | Change ownership                             
          /fs/watch         | GET (WebSock) | Watch for filesystem changes                 
      
      It'll get CLI support very soon.

      But I've just been using magic-wormhole for this.

  • lbrito 24 minutes ago
    >Any time anybody needs a pen, several are at hand, in exactly the first place they look.

    >Now, today, under the hood, Sprites are still Fly Machines. But they all run from a standard container.

    So its basically a faster VM?

    What are some use cases for this that benefit from the faster boot time?

  • tom_ 33 minutes ago
    The sprite is a pretty outdated thing these days, but is the term sufficiently unused to make it free for reuse?
  • tptacek 1 hour ago
    I wish I'd had more space to write about the global orchestrator design, because it's fun.

    The Fly Machines orchestrator goes through some trouble to keep the source of truth for each VM decentralized, owned by the physical it runs on. But there's still global state --- apps, organizations, services. That stuff is all on Postgres. Postgres keeps up with it just fine but I'd be lying if I didn't say we're always looking out the corner of our eyes on metrics.

    The global state for Sprites is on object storage. Each organization gets a separate SQLite database, and that database is synchronized to object storage with Litestream.io (Lightstream is load bearing in a bunch of places here; solid as a rock for us).

    I think people really still sleep on the "multiple SQLite database" backing store design.

    • chrismccord 53 minutes ago
      I've been working on the orchestrator side with Elixir and Phoenix, so happy to continue the discussion for curious minds. One of the coolest things we can do is things like this in Elixir - from any node we can reach out to a sqlite db across the planet:

      OrgTracker.with_repo(org_id, fn ->

        repo.all(from sprite in "sprites", select: ...)
      
      end)

      That will find or place an Elixir process on the cluster and rpc the target node with our code. Placements can be sticky so they pin to a machine so we don't have to suck down the db every start, but we also balance out the load and handle failover of durable processes automatically. Combined with litestream, the result is distributed sqlite with failover while treating it essentially like a locally reachable sqlite db. Yes there is the speed of light to contend with, but by sending the execution across the wire rather than individual queries, we only ever pay a single hop to reach the process/sqlite.

    • underdeserver 1 hour ago
      For those who missed it, tptacek wrote TFA.
  • fuddle 42 minutes ago
    How does Sprites compare to E2B sandboxes?
  • neuroelectron 23 minutes ago
    This is not the kind of sprites I'm interested in
    • rvz 15 minutes ago
      Thought this was going to be a deep dive into sprite sheets.

      Low and behold, yet another AI product selling advertisement + veiled promotion on HN.

      It's almost as if the guidelines don't matter anymore. From [0].

      'Please don't use HN primarily for promotion. It's ok to post your own stuff part of the time, but the primary use of the site should be for curiosity.'

  • ZeroConcerns 1 hour ago
    More US-and-AI-centric dreck, unfortunately. So, right in the HN wheelhouse, I'm sure...
    • tptacek 1 hour ago
      It's a hardware-isolated Linux shell with a durable disk you can conjure out of the sky on 1.5 seconds notice that costs virtually nothing when you're not actively using it. If we could have shipped this in 2021, before anyone thought coding agents would work, we'd have barfed on our shoes with excitement.
      • ZeroConcerns 51 minutes ago
        Yeah, I'm barfing alright. Maybe take a step back from the relentless self-promo, and see what you're actually achieving. It's not what you think it is... (and yes, I know who I'm replying to -- where's n-gate.com when we truly need it...)
    • teach 33 minutes ago
      I guess I can see how pre-installing some LLM agents makes it potentially seem "AI-centric", but I don't understand at all how this could be "US-centric".
    • webdevver 22 minutes ago
      the US is the centre of the world, and AI is at the centre of US attention!

      why would anyone do anything else, anywhere else?