Much of the difference in optimizability is that C/C++ allows you to do fairly extreme copy elision. This is important because memory bandwidth tends to be a major performance bottleneck.
C/C++ code can trivially reach into some remote bit of state and mutate it directly. The compiler rarely objects, no matter how ugly the access method or pattern. It is up to the programmer to make sure the invariants hold and the compiler can’t always see through it. These elided copies add up, though it usually has a maintainability and safety cost.
It is related to why functional languages tend to be mediocre for supercomputing in practice despite having nice properties for massive parallelism in theory.
Clojure creator Rich Hickey describes traditional imperative programming as "PLace Oriented Programming (PLOP)" when talking about how Clojure's approach is better for certain kinds of problems.
I agree with his point, and Clojure is my favorite programming language, but the computer itself is place-oriented. It runs on reading from and writing to locations in memory, and extracting maximum efficiency from it requires direct control over that memory.
Modern computers are exquisitely sensitive to spatial and temporal locality because many of the hardware features that speed your code up (for free!) take advantage of data or instructions that are “next to each other.” This can be a challenge in a functional program, because not only do you often have little control about where things go, the abstractions that make cool things possible (eg, closures, or lazy evaluation) naturally introduce indirection that hurts locality.
You can have great abstractions on the upper levels that lower to fast primitives. See the articles [1] Push your "ifs" up and "fors' down and [2] Vectorized interpreters for more details.
You could similarly argue that the computer is in fact fundamentally functional, since the core instruction cycle (Fetch Decode Execute) does not modify any memory.
The key insight into that system is that we've embedded a functional system within a mechanism that makes it imperative. The functional cycle can output commands to fetch it more data, that it can then use in the next cycle. It becomes imperative by the addition of time, which it needs because it's limited in the amount of input and output.
The execute stage very much modifies state; in fact it always modifies state. Any non-memory instructions modify registers, and any memory instructions either write to registers or to memory. Even a noop will increment the instruction pointer, so you're always modifying a register regardless.
Are there any common resources for this kind of access methods and patterns that specifically aim to bridge the soft-/hardware gap? The best one I can come up with is Performance Analysis and Tuning on Modern CPUs[0] by Denis Bakhvalov.
> lean-zip beats every single compression level miniz_oxide
Miniz_oxide is an intentionally simple implementation. It would be more fair to compare it to the newer zlib-rs[1], which even beats zlib-ng (fastest implementation in C)[2].
So, lean-zip "outperforming an equivalent Rust package that crates.io says has been downloaded over a billion times". Yes, zlib-rs has only been downloaded 200 million times... But it is way faster!
I appreciate that about D. I've always wondered though, if the convention isn't backwards, like how Lua forces you to specify local instead of global, TS isn't strict by default, most languages have default mutable rather than default immutable, etc.
I always thought having a working program should be opt-out, not opt-in.
(For example, my ideal language supports dynamic typing, with the compiler directive #jammode, or alternatively #explodeinprod)
I don't know about roc specifically, but the best projects often come out of something someone built for a particular need. The open source / public version will often need work to strip out stuff specific to the original project, as well as refinement to make it more suitable for generic public consumption, such as adding support for edge cases, features that the original project didn't need but others likely will, support for multiple platforms for deployment, etc.
Have you considered exposing these optimisations through a library so programmers can request optimisation passes and have the compiler check the preconditions?
Asking cuz I'm preparing an article on how fast JS can get and found the same thing, you can get in 10% of optimised C-code with basically any code if you torture V8 enough (excepting SIMD where there seems to be a bug in V8 where it doesn't optimise to SIMD even when beyond obvious). Figured out I could extract it in a lib and do it more streamlined but not sure how well manual transforms generalise.
Also, when working on the roc-deflate, did you find recurring transformations that could fit that model or were most improvements specific to compression?
Yes, it does beat -O3 C sometimes. I took code from diverse 'language speed' benchmarks from the internet and had to rearrange them to get -O3 to beat equivalent-looking JS (Node 25.8.1 w\ V8 14.1 and clang 21 on MBP3).
i've tried various options, Firefox prefers ` for (let i = n; i--;) out[i] = a[i] * b[i] + 0.25;`
Its already at peak optimization so no form is significantly better.
Uh, so the program is almost as fast as libdeflate at compressing at a given level. But the post makes absolutely no attempt at establishing that a given level compresses as well as the corresponding level in libdeflate. In the linked post there is a graph detailing that libdeflate does indeed compress quite a lot better than the original lean-zip at the same level.
Without data on compression levels it is hard to make any useful conclusions.
roc-deflate produces identical outputs to libdeflate - I mentioned it briefly, although in retrospect the wording could have been clearer:
> I didn't expect to beat libdeflate's performance across the board, but I wanted to see how close it could get on the non-streaming APIs (as roc-deflate hasn't implemented streaming yet), using the classic Silesia corpus for benchmarking and verifying that it produced identical outputs.
I wonder if Roc will eventually have high-level abstraction to handle SIMD support. Perhaps by bringing some ideas from Futhark [1], a functional language focused on data parallelism. IIRC, the author has already been interviewed on Richard's podcast.
We should probably all be programming in a FP language, with behind the scenes an AI translating it to something faster using correctness preserving transforms.
I suspect this will be the best way to delegate things to AI while still staying in driver's seat.
The snarky answer to that is that you just described a compiler with extra undefined behavior sprinkled in for extra excitement.
Compiled ones already translate to the theoretically fastest imperative language (machine code) using correctness preserving transforms. And plenty of them already occupy the top tier of performance for garbage collected languages. But being able to match the performance people can achieve with C, C++, etc. requires manual memory management, and that is a difficult nut for functional languages to crack because it interacts poorly with closures.
saying it "beats a crate that has been downloaded X times" is not saying anything other than that is popular. It'd be nice to see slightly more detail there.
This is kind of hacker news post and comments I like; extreme languages on extreme performance with extreme features, not types being added to a language ten gizzilion times slower than a tortoise and the post making anti-type flat-earthers getting freaked out if they have to vaccinate next.
This seems to raise the question: is libdeflate fast? My overall impression of libdeflate is that it produces a similar output size to zstd but takes 40 times longer to do it. I realize these are different algorithms of course, but they aren't that different, both being in the Huffman line of the LZ family. So why is libdeflate considered a performance benchmark?
It’s probably just because a zstd implementation in Lean would be more difficult. The whole project depends on being able to have a correctness proof for the implementation, and implementing that for the DEFLATE algorithm is simpler.
Problem specifically in Haskell is that you have to worry about implementing tail recursion to get efficient stack usage. That's not necessary for loops, which Haskell doesn't support. Though other functional languages do support them.
Digging through the current Haskell standard library - for_ calls traverse_, which calls foldr, which calls foldMap, which calls… foldr. Found the recursion! The person you’re replying to is correct - you can mimic the style of a for loop, but under the hood all of this stuff is going to be implemented with recursion (and tail recursion, if you don’t want to hate your life).
I also may have gotten some detail slightly wrong, the code in Data.Foldable is kinda weird and I haven’t Haskell’d in a while
Loop constructs like whileM_ require extra IORef effort to the point where people actually recommend recursion (with tail recursion) instead, which is still harder to implement than conventional while loops.
In the age of powerful LLMs, is there really any reason to program in any higher abstraction than C? Seems like most other languages can be put out to pasture. They were just compromises and shortcuts to make things easier for humans.
Because LLMs make mistakes, and you want to prevent that?
No wonder Rust is becoming the default language for high-performance LLM-written code. I hope Roc becomes a viable option soon, as it reaches stability.
There are no well known programs made with Roc. It's been usable for writing real software since July or so for those of us willing to deal with compiler crashes and regular syntax/feature changes. To my knowledge there are six people producing substantial amounts of software in Roc. I'm using it for a speech recognition/transcription app and the desktop environment and apps on my eink tablet, both on my own UI framework.
Well, it has not even reached 0.1.0, so the language is still in early development phase. But it has interesting properties, so I would recommend experimenting with it.
C in indeed a functional language in that most C projects are things that have been observed in real life and have subsequently been uploaded into computers.
I'd also like to note that performance is the ULTIMATE concern in a lot of cases.
If you have a server that is making 10million requests to a certain method in a minute, then you cannot afford any performance compromise. Any, at all.
Rust will never replace C/C++ with regard to high performance computing.
> Rust will never replace C/C++ with regard to high performance computing.
I don't buy this. Rust is a lot easier to make more performant than C++ and doesn't really trail behind C in anything but .text size last time I looked. Considering you can easily write C-like Rust that ends up in the same LLVM stack as Clang (more or less) I just don't think this is true outside of some very specific contexts.
> Rust is a lot easier to make more performant than C++
I have never seen evidence of this in the wild. C++ is still significantly faster than Rust in practice when doing performance-sensitive parallel development in the two languages.
The reason for the performance gap is mostly understood. Rust's codegen is currently somewhat worse for a variety of reasons. Rust's much weaker compile-time metaprogramming features make some optimizations impractical in the language that are easy in C++. Culturally, "unsafe" minimization in Rust pessimizes code architecture.
Rust can get pretty close to C++ but it often requires writing some very ugly Rust.
anyway this is especially funny because C++ without compiler-specific pragmas/flags is also a bit mechanistically unsympathetic to the hardware, the whole strict-aliasing saga, you turn it on then you'll suffer with UB and has completely different semantics than hw, you turn it off and you have to manually SSA your code to avoid completely unnecessary loads when accessing fields more than once, the STL is slow, idiomatic C++ often allocates like crazy...
then people saw that and decided that we want more of that, so Rust was born;) (yes I'm being sarcastic here)
I guess the reason C++ fares better traditionally is because even though the actual specification is bonkers, you can mostly coax the compiler into "doing the right thing" as long as you don't mind losing portability and ugly stuff...
> C in indeed a functional language in that most C projects are things that have been observed in real life and have subsequently been uploaded into computers.
I don’t think you share the same understanding of functional as TFA uses.
Not functional as in “working”/functioning. Functional is meant here as in functional programming, the paradigm.
Maybe I am misunderstanding what you wrote, but "state" in the context of functional programming refers to the state internal to the program, not the state external to the program. So whether the input data comes from the real world is orthogonal to the question of whether a program preserve state.
Interesting perspective. If I may ask, do you mind dropping 'root' privileges for a better chat / debate experience? That would probably help with your case.
There are no authoritative definition AFAIK, but this is a way too coarse definition to qualify for me at least.
I think many equate FP with referential transparency, garbage collection, first class functions/closures. But usually it’s a sort of mix of all the above or opt-in. Standard ML makes it easy to mutate state for instance.
C is by no means a functional language. It is perhaps the definition of imperative; the poster child.
Is this possible that you could give me some example project, for instance, that makes of C imperatively?
Probably depends on at what level you see things. If you visit and probably live in Japan, then you develop the concept of 'cryptocurrency'. You then subsequently develop intellectual property around the 'concept' of 'cryptocurrency'.
Is this really 'imperative' programming? I'd say not but depending on at what level you see things, then it could be.
Calling C a functional language because you’re able to manually emulate one feature that functional languages give you automatically, is like claiming C is a garbage collected language because ‘at the project level’ you remember to call free().
I can’t follow or understand your japan/crypto analogy, I’m sorry.
C/C++ code can trivially reach into some remote bit of state and mutate it directly. The compiler rarely objects, no matter how ugly the access method or pattern. It is up to the programmer to make sure the invariants hold and the compiler can’t always see through it. These elided copies add up, though it usually has a maintainability and safety cost.
It is related to why functional languages tend to be mediocre for supercomputing in practice despite having nice properties for massive parallelism in theory.
I agree with his point, and Clojure is my favorite programming language, but the computer itself is place-oriented. It runs on reading from and writing to locations in memory, and extracting maximum efficiency from it requires direct control over that memory.
[1] https://debasishg.github.io/blog/push-ifs-up-fors-down/ [2] https://venge.net/graydon/talks/VectorizedInterpretersTalk-2...
You could similarly argue that the computer is in fact fundamentally functional, since the core instruction cycle (Fetch Decode Execute) does not modify any memory.
The key insight into that system is that we've embedded a functional system within a mechanism that makes it imperative. The functional cycle can output commands to fetch it more data, that it can then use in the next cycle. It becomes imperative by the addition of time, which it needs because it's limited in the amount of input and output.
Are there any common resources for this kind of access methods and patterns that specifically aim to bridge the soft-/hardware gap? The best one I can come up with is Performance Analysis and Tuning on Modern CPUs[0] by Denis Bakhvalov.
[0]: https://github.com/dendibakh/perf-book
I also like this very readable paper https://xnning.github.io/papers/perceus.pdf, about reducing the costs of reference counting in context of functional languages.
Miniz_oxide is an intentionally simple implementation. It would be more fair to compare it to the newer zlib-rs[1], which even beats zlib-ng (fastest implementation in C)[2].
[1]: https://lib.rs/crates/zlib-rs
[2]: https://trifectatech.org/blog/zlib-rs-is-faster-than-c/
It's fun trying to reorganize your code to make it pass the `pure` checking.
I always thought having a working program should be opt-out, not opt-in.
(For example, my ideal language supports dynamic typing, with the compiler directive #jammode, or alternatively #explodeinprod)
It then links to a video where a consultant mentions using in production (for several years already?), which confused me.
How can you justify using immature technologies for client work? Or is it an extra benefit when it breaks and you have to fix it?
I'm genuinely curious, maybe it's not as bad as I think. But I've used immature hosting platforms, and that certainly didn't go well for me!
Ask everyone that's using generative AI?
Asking cuz I'm preparing an article on how fast JS can get and found the same thing, you can get in 10% of optimised C-code with basically any code if you torture V8 enough (excepting SIMD where there seems to be a bug in V8 where it doesn't optimise to SIMD even when beyond obvious). Figured out I could extract it in a lib and do it more streamlined but not sure how well manual transforms generalise.
Also, when working on the roc-deflate, did you find recurring transformations that could fit that model or were most improvements specific to compression?
>where it doesn't optimise to SIMD even when beyond obvious What was that code?
> What was that code?
literally:
``` function numericKernel(a, b, out, n) {
} ```Without data on compression levels it is hard to make any useful conclusions.
> I didn't expect to beat libdeflate's performance across the board, but I wanted to see how close it could get on the non-streaming APIs (as roc-deflate hasn't implemented streaming yet), using the classic Silesia corpus for benchmarking and verifying that it produced identical outputs.
[1] https://futhark-lang.org/
I suspect this will be the best way to delegate things to AI while still staying in driver's seat.
Compiled ones already translate to the theoretically fastest imperative language (machine code) using correctness preserving transforms. And plenty of them already occupy the top tier of performance for garbage collected languages. But being able to match the performance people can achieve with C, C++, etc. requires manual memory management, and that is a difficult nut for functional languages to crack because it interacts poorly with closures.
I also may have gotten some detail slightly wrong, the code in Data.Foldable is kinda weird and I haven’t Haskell’d in a while
I'd also like to note that performance is the ULTIMATE concern in a lot of cases.
If you have a server that is making 10million requests to a certain method in a minute, then you cannot afford any performance compromise. Any, at all.
Rust will never replace C/C++ with regard to high performance computing.
I don't buy this. Rust is a lot easier to make more performant than C++ and doesn't really trail behind C in anything but .text size last time I looked. Considering you can easily write C-like Rust that ends up in the same LLVM stack as Clang (more or less) I just don't think this is true outside of some very specific contexts.
I have never seen evidence of this in the wild. C++ is still significantly faster than Rust in practice when doing performance-sensitive parallel development in the two languages.
The reason for the performance gap is mostly understood. Rust's codegen is currently somewhat worse for a variety of reasons. Rust's much weaker compile-time metaprogramming features make some optimizations impractical in the language that are easy in C++. Culturally, "unsafe" minimization in Rust pessimizes code architecture.
Rust can get pretty close to C++ but it often requires writing some very ugly Rust.
anyway this is especially funny because C++ without compiler-specific pragmas/flags is also a bit mechanistically unsympathetic to the hardware, the whole strict-aliasing saga, you turn it on then you'll suffer with UB and has completely different semantics than hw, you turn it off and you have to manually SSA your code to avoid completely unnecessary loads when accessing fields more than once, the STL is slow, idiomatic C++ often allocates like crazy...
then people saw that and decided that we want more of that, so Rust was born;) (yes I'm being sarcastic here)
I guess the reason C++ fares better traditionally is because even though the actual specification is bonkers, you can mostly coax the compiler into "doing the right thing" as long as you don't mind losing portability and ugly stuff...
But, Rust does indeed embedded extra code for much of the functionality that it adds ontop of what C/C++ has.
This obviously has a performance hit. Might not be observable on a desktop application but it will be evident on server applications.
I don’t think you share the same understanding of functional as TFA uses.
Not functional as in “working”/functioning. Functional is meant here as in functional programming, the paradigm.
And, if all you're doing is taking material from real life and uploading into computers, then you're not modifying state at all.
I think many equate FP with referential transparency, garbage collection, first class functions/closures. But usually it’s a sort of mix of all the above or opt-in. Standard ML makes it easy to mutate state for instance.
C is by no means a functional language. It is perhaps the definition of imperative; the poster child.
Regardless of the language used!
> C in indeed a functional language […]
… which it isn’t, it’s imperative ¯\_(ツ)_/¯
Probably depends on at what level you see things. If you visit and probably live in Japan, then you develop the concept of 'cryptocurrency'. You then subsequently develop intellectual property around the 'concept' of 'cryptocurrency'.
Is this really 'imperative' programming? I'd say not but depending on at what level you see things, then it could be.
At project level, then it's purely functional.
Calling C a functional language because you’re able to manually emulate one feature that functional languages give you automatically, is like claiming C is a garbage collected language because ‘at the project level’ you remember to call free().
I can’t follow or understand your japan/crypto analogy, I’m sorry.