Libraries Run Rust Inside Python (With PyO3)

(belderbos.dev)

13 points | by lumpa 58 minutes ago

5 comments

  • simonw 29 minutes ago
    I was a bit nervous about this trend when it started picking up because I care about Pyodide (Python running via WebAssembly) and libraries that use PyO3 might not work in Pyodide.

    Thankfully that's now been mostly solved - you can compile and publish WASM builds of Rust or C extensions on PyPI now and a Pyodide can then use them.

    Here's the WASM build of the Rust-including Pydantic-core package for example: https://pypi.org/project/pydantic_core/#pydantic_core-2.49.0...

    • saghm 10 minutes ago
      I'm kind of surprised to hear that was a concern because my perception is that PyO3 is more analogous to wrapping native C code in Python libraries (which is pretty well established since long before either Rust or WASM), and Pyodide seems more like something for use in browsers rather on the server-side. When I looked into this earlier this year, trying to run Python via WASM on the server side with Pyodide had a bunch of hoops I needed to jump through, and it ended up being a lot simpler to use a build of Python from one of the core Python contributors that directly targeted WASM.
  • roywiggins 24 minutes ago
    > reach for, a Rust extension does the work

    ai; dr, sorry

  • skeledrew 13 minutes ago
    But does it work everywhere Python works? My main issue with this Rust move has always been compatibility. Python can be embedded and ran in a heck of a lot of places. What's the story when libraries that I may want to depend on are actually implemented in Rust and my target doesn't/can't handle the toolchain and there's no build target?
    • the__alchemist 4 minutes ago
      It won't work in those cases. Do you have explicit examples? I suspect this won't come up much as Rust has, IME, much broader target support. (Including platforms without a GPOS)
  • the__alchemist 34 minutes ago
    This is a nice party trick! I use it to provide easy installation of software that is written in rust, but is primarily used by Python users. (e.g.: Biology tools) They can do `pip install <name>`, since some people prefer this over downloading executables/installers. Bonus: Maturin/PyO3 can build "manylinux" binaries automatically, which helps with the Linux ABI diaspora.
  • hk1337 12 minutes ago
    Why not just run C inside of Python?
    • saghm 9 minutes ago
      For the same reasons someone might prefer Rust to C in general; safety, ergonomics, etc. If you want to write C in Python, nothing is stopping you.
    • the__alchemist 6 minutes ago
      You can do that too. I generally prefer rust as it's a nice lang to work with.