Harness Engineering

(Habitat-Thinking.github.io)

65 points | by tomrod 1 hour ago

14 comments

  • gardnr 27 minutes ago
    Here’s the original article this one is based on: https://martinfowler.com/articles/harness-engineering.html

    And here is the original author on a podcast talking about it: https://open.spotify.com/episode/4FxEdjXldNhoh67KYVmbDu

  • davedx 39 minutes ago
    This part seems relevant - they make explicit the difference between their HARNESS.md document and AGENTS.md. It's actually interesting: https://habitat-thinking.github.io/ai-literacy-superpowers/p...
    • Supermancho 18 minutes ago
      The Agent/harness has no meaningful distinction. The backing model(s) can only read through a harness. Call it an agent, call it a tool, call it a library. They are all harnesses if they are backed by a model.

      Broadly speaking,

      - Copilot only reads AGENTS.md

      - Codex reads AGENTS and CONTEXT

      - Claude reads CLAUDE

      This ai-literacy is noise, muddling the definitions and suggesting yet-another-md-file.

      • mawadev 9 minutes ago
        I'm an AI Agent Harness .md File consultant, you can ask me anything
  • tosh 38 minutes ago
    it lists 'Garbage Collection' as one of the 3 components of a harness

    I don't know any harness that solves 'Garbage Collection' in the way described here

    (most harnesses accelerate context pollution and code base drift via instructions they embed into system prompts, tool descriptions and skills)

    • hedgehog 16 minutes ago
      I think most people that use LLM coding tools enough independently derive most of the stuff in the article. For example my approach to garbage collection is to sample paths within the project and chunks of file content, then assemble context (similarity search results, git blame) for the agent to use to assess cruftiness and if found schedule cleanup.
  • iTokio 48 minutes ago
    It’s fascinating to live through a the emergence of a new technology and to see people trying to make sense of it as they go.

    I personally think that a lot of the words used around AI, harness, SKILLS, agents, RAG.. are make up words or close to it, words that do not have profound semantics, even though people are trying to, often after the fact, make sense of them.

    It’s just popular words that are different enough that people like to use them to claim a new knowledge, or to market a product.

    But we could use AI tooling instead of harness in the abstract, and it would be better to use more precise terms for more concrete use cases, agent loop, CLI, IDE..

    It was a fun game at a time where papers were competing for attention, but now that it has become a proven technology, I hope we can find more precise and meaningful words.

    • tomrod 41 minutes ago
      Hear hear! We techies are bad at naming things. "NoSQL" is probably a top contender there.
  • lacoolj 48 minutes ago
    Not saying this is AI-gen but it's very dense and doesn't read like something I could glean info from easily
  • samuell 20 minutes ago
    I also stumbled upon this site, which on a first glance looks really thorough. Interested to hear if folks have comments on it though:

    https://walkinglabs.github.io/learn-harness-engineering/en/

  • issacnitin 19 minutes ago
    This is perfect, here's a tool for harness engineering to be more effective that I just released today https://github.com/issacnitin/RealDiff
  • siavosh 31 minutes ago
    What’s the best current practice if we want to enable agent code review and approval of GitHub PRs but only for specific users?
  • hnd9q09qk4 15 minutes ago
    This is oddly reassuring
  • ChrisArchitect 10 minutes ago
    Related currently:

    The Harness is the Thing

    https://news.ycombinator.com/item?id=49452346

  • luciandan 1 hour ago
    I like how agent review is a deprecation target. explicitly. Se then a harness matures by getting dumber and cheaper, not smarter.
  • esafak 53 minutes ago
    What even is this "AI Literacy framework"? When I see such overengineering I look at what the author has done. In this case, I find the author runs a consultancy on engineering "Habitats for Humans and AI" (https://www.russmiles.com/), listing a bunch of books he did not write (the author names are conveniently cropped out). This doesn't even belong on LinkedIn.
    • tomrod 43 minutes ago
      I'm not the author -- but I am currently fortunate to be sitting in a workshop he is teaching on the topic and figured HN might like (especially since he gives a lot of his content out for free/OSS).

      As someone who does loads of AI-driven dev and governance, I'm finding there are a lot of great nuggets here. Between him (chaos engineering) and Kent Beck (extreme programming) I'n a kid in the candy store and wanted to share.

  • wrinkl3 38 minutes ago
    "Explain the harness" is apparently the new "explain LLMs" genre of slop blogging, I now see articles to this effect on HN daily.
  • tickerlayer 43 minutes ago
    [dead]