Handling errors in compilers has become the most interesting part of my job. If you do the math, most calls to compilers are error-yielding calls. So those calls really need to produce the best errors possible! (Maybe this is less true in LLM land, but certainly it was right before.) Tracking spans, doing extra algorithmic work, and exploring why the expectation failed in order to give a user the most-informative error possible is extra work on the main compute path. But it pays dividends: easily-diagnosable bugs and quick fixes present in the code itself. Tracking spans to point back to origin definitions, etc., always feel like plumbing work until you see the error that explains exactly what went sideways. And once you do, the errors that don't feel like a frustration you are trying to cast your net over.
>Maybe this is less true in LLM land, but certainly it was right before
I disagree:
I was tasked with rewriting a Python application in ucode for use on OpenWRT. It's a low-resource and dynamic language, and sufficiently JavaScript-like that Opus 4.8 couldn't write code without errors that were obvious with my LSP. The interpreter errors were often reported with the the correct line and column numbers, but with the incorrect code and a vague error message. I had to ask it to use my LSP and fix the warnings and errors, which led to it fixing a few dozen bugs.
Just the other day, Fable 5.1 still wrote a lot of code in Rust that had compiler errors. While I'm sure it could have fixed these errors if the Rust compiler also reported incorrect line numbers, columns, code, with a vague error message to boot, I saved a substantial amount of time and tokens because its error reporting is so comprehensive.
Specific and accurate error message reporting will always have the same high value that it does today. Error messages are far cheaper to generate than tokens.
Yes, it is very helpful to have actually directed information. But the need to use a TUI to do it, bugs me. Not coincidentally, I use JAWS most of the time.
I disagree:
I was tasked with rewriting a Python application in ucode for use on OpenWRT. It's a low-resource and dynamic language, and sufficiently JavaScript-like that Opus 4.8 couldn't write code without errors that were obvious with my LSP. The interpreter errors were often reported with the the correct line and column numbers, but with the incorrect code and a vague error message. I had to ask it to use my LSP and fix the warnings and errors, which led to it fixing a few dozen bugs.
Just the other day, Fable 5.1 still wrote a lot of code in Rust that had compiler errors. While I'm sure it could have fixed these errors if the Rust compiler also reported incorrect line numbers, columns, code, with a vague error message to boot, I saved a substantial amount of time and tokens because its error reporting is so comprehensive.
Specific and accurate error message reporting will always have the same high value that it does today. Error messages are far cheaper to generate than tokens.
Yes, it is very helpful to have actually directed information. But the need to use a TUI to do it, bugs me. Not coincidentally, I use JAWS most of the time.
Do it after you have actually made the rust version work the same as the C version.
Insane to be adding complexity, breaking compatibility and introducing new bugs to software that needs to function correctly.