I heard a neat tip recently about API design for agents: give them a way to send you feedback.
The example I heard was an MCP with a "feedback" tool which had a tool description saying that coding agents should call that any time they had trouble figuring out how to use the rest of the MCP.
I really like this. It's super cheap to implement and I expect you'd get a bunch of actionable signal in amongst the noise.
Yeah i have a similar setup at the harness level - a “devlog” at the end of every session about the experience developing, what could have been better, what was confusing
And also files issues for blockers
I’ve absolutely caught things and made improvements just from skimming them occasionally - they are particularly useful when you get a PR that makes you scratch your head
But I’m definitely not taking full advantage of all the feedback coming in yet
I have to imagine parsing signal from noise there is a massive challenge when it’s other agents that are using your MCP and not just your own
a feedback tool idea is good (mintlify landed on smth similar) but it's just the starting point. what we found is feedback < support. agents love resolution because anything less is still a blocker in their workflow. and it can't be MCP specific because your curl and CLI users need the same surface.
I had to deal with exactly this issue with one of my recent oss projects so I can share a few things on the topic.
1. Text is the default interface
i.e. the api must be text based first but it should allow to fallback to structured output by using the accept header.
That being said, it is not wrong to introduce other means to return json by using ?format=json etc.
2. Make it grepable
Basically surface as much useful information in a single line so that the agent can grep and slice.
3. Identifiers must be short
i.e. short enough to describe in 5 tokens but not too short to introduce collisions or confusion.
Otherwise you could be wasting a lot of token for nothing. However, adding prefixes helps like cus_abc123, token_xxx, etc. The prefix can help with lookup, error correction and deduplication.
4. Surface information that is likely to be used by the agent
i.e. if the agent is asking for a list of resources, don't just return the list but also some additional information that might help the agent understand better the context around the resource.
Without this a single task could take a lot more steps simply because the agent needs to run its own loop - it is slow and expensive.
5. Add bulk operations
It is a lot easier to insert 10 records in one request then performing 10 separate requests.
6. Error messages should be descriptive
Ensure that error message point to actual docs and manuals that can be read by the agent so that it can troubleshoot on its own. Also return hints.
I've added most of these at https://github.com/crmkit/crmkit which is highly experimental CRM I tried to specifically design for AI agents. We use it internally for a few projects and it is not perfect but I think it might be on the right direction.
One of my recent projects has contributable code (via extensions) with a central settings management (json file). What I do is take all the defaults contributed by the extensions (namespaced by extension name) and materialize them to the settings file as the initial values to each setting. When an agent wants to edit the settings, it already knows the entirety of the setting surface area and thier values. It never has to go digging to find default values nested somewhere in the extension code or documentation. Also, the code that reads the extensions into the framework (for runtime execution) only reads the materialized values, it doesn't have a concept of a default value past the point of materialization. Defaults only exist in the extension registration/mounting part of the lifecycle (so they can materialize if missing from the settings file).
One upside of this is as new settings are created (from new features being developed) your configuration gets notified via the new fields being materialized into your settings upon startup.
AFAIK this is not at all a new practice. I used to see TOML files with default values in commented out lines all the time.
The downside is that the agent is not aware of which settings are being overridden from the default (like a sparse settings file would provide), but I don't know how much semantic value that offers in most cases (other than maybe debugging?).
I'm not even taking a side on this one this is just a pattern I decided to take for this use case.
I don't like the idea of making APIs effectively unusable for a human developer. We might not write a lot of code anymore, but getting rid of defaults and asking agents to pass all possible values explicitly makes it impossible to quickly debug the API call (e.g. with curl or something).
It's similar to HTTP/1: there are a lot of headers in the protocol, but you can still use nc or openssl s_client and type the request manually; you probably only need Host: and Content-Type:, maybe Content-Length:, and it will work. I don't normally write HTTP requests in the terminal, but I know I can do it if I need. It's better to keep it this way, I think.
Suggestion:
Save the article as an .md file.
Upload it to Fable 5 with the prompt: "Agree or disagree. Be verbose."
I learned a lot: Where he's right. Where he's wrong. A 'delicious bug in his own example" code (command injection vulnerability - in the code sample used to demonstrate why you don't need sandbox.git.clone). I also learned that, according to Fable 5: "A Typescript SDK is the strongest anti-hallucination device we currently have." And a lot more.
Just did this, found result fairly interesting, I reject most of its objections on the basis of bad code. When I say Im pro explicitness I'm also pro comment, and pro separating the "this is core fact" and "this is configuration that i don't care too much about". Will use this test for future writing.
in my experience models will answer introspective questions decisively but can't actually introspect. this is pretty similar to humans (e.g. ford's faster horses quote).
with AI stuff you can actually run real tests though.
> Defaults are bad. Agents can be expected to read the documentation, register what good starting values are, and fill them all in, in place.
That is pretty bad, when you now need to look at this code, and you have 95% of the output being craft, but sometimes it isn't, and you need to understand the difference between each.
A great example of that is:
int fd = open("log.txt", O_RDONLY);
Versus:
// 1. Manually build and initialize the Security Attributes structure
SECURITY_ATTRIBUTES sa;
sa.nLength = sizeof(SECURITY_ATTRIBUTES);
sa.lpSecurityDescriptor = NULL; // Explicitly no custom security descriptor (inherits default)
sa.bInheritHandle = FALSE; // Explicitly state this handle cannot be inherited by child processes
// 2. We need a handle for the template file parameter.
// Win32 requires this to be an active file handle opened with GENERIC_READ, or explicitly INVALID_HANDLE_VALUE.
HANDLE hTemplateFile = INVALID_HANDLE_VALUE;
// 3. Now we call CreateFile with every single parameter fully populated
HANDLE hFile = CreateFile(
"log.txt", // 1. lpFileName: The file we want to open
GENERIC_READ, // 2. dwDesiredAccess: Read-only access
FILE_SHARE_READ, // 3. dwShareMode: Prevents any other process from writing to it
&sa, // 4. lpSecurityAttributes: Pointer to our explicitly defined struct
OPEN_EXISTING, // 5. dwCreationDisposition: Only open if it already exists
FILE_ATTRIBUTE_NORMAL, // 6. dwFlagsAndAttributes: Normal file, no special caching or async flags
hTemplateFile // 7. hTemplateFile: Passing our explicit invalid handle instead of NULL
);
An agent can output the second just as well, sure.
However... which one do you think is better to read or understand?
Did you catch the fact that this is blocking concurrent writers? Or that we expected the file to exist?
This is a fair point. Im not opposed to the open function existing, but I do think its valuable to own the underlying implementation of thigns like it (maybe not open) in your codebase so you can review the implementation and reconfigure it to your precise requirements.
MCP Auth is just Oauth, its designed for humans to authenticate their sessions for connections.
TBH I know nothing about A2A.
Agent Identity and Authz is a different problem, allowing agents to operate independently from humans with granular permissions is coming/whether from these protocols or others, and when it does I think CLIs/CLI Device Auth which is used as a rough proxy for this where the agent just takes your identity will finally go away.
>Defaults are bad. Agents can be expected to read the documentation, register what good starting values are, and fill them all in, in place.
This is not what defaults are for.
If, for example, you are writing a replacement CLI for git, *for the love of God and all that is holy* do not force agents to read the entire documentation and pass a value for every possible parameter
Because the defaults aren't just for convenience, the API designer is also making the parameters they think should be used the most have the least resistance. Good example is runtime parameters like in the JVM. You shouldn't start with having to tune your JVM, you probably want a middle of the road place to start with even if you know you're going to tune it.
im not opposed to good defaults, i just believe they should be explicit. The AI should read the starting.md to fill in explicitly what its JVM configuration is. Then, when you want to tune in the future its clear what options are available and what specifically is changing.
Because LLMs degrade with context length. And even if they didn't, having to constantly ingest the same stuff is wasteful for execution and cost. Why stuff the context when you don't have to?
Conservatively speaking, LLMs degrade past 40% of a 2M context window, 4k tokens is 0.2%, so no degradation there. Thats also current generation conservative estimate.
I think wasteful is an irrelevant metric. Claude ingests those tokens in a quarter of a second, if it causes it to catch any bug ever it saves far more time than it ever uses.
Any individual default that causes unexpected behavior causes more problems, takes more time and costs more than the small cost of being explicit.
The example I heard was an MCP with a "feedback" tool which had a tool description saying that coding agents should call that any time they had trouble figuring out how to use the rest of the MCP.
I really like this. It's super cheap to implement and I expect you'd get a bunch of actionable signal in amongst the noise.
And also files issues for blockers
I’ve absolutely caught things and made improvements just from skimming them occasionally - they are particularly useful when you get a PR that makes you scratch your head
But I’m definitely not taking full advantage of all the feedback coming in yet
I have to imagine parsing signal from noise there is a massive challenge when it’s other agents that are using your MCP and not just your own
1. Text is the default interface
i.e. the api must be text based first but it should allow to fallback to structured output by using the accept header.
That being said, it is not wrong to introduce other means to return json by using ?format=json etc.
2. Make it grepable
Basically surface as much useful information in a single line so that the agent can grep and slice.
3. Identifiers must be short
i.e. short enough to describe in 5 tokens but not too short to introduce collisions or confusion.
Otherwise you could be wasting a lot of token for nothing. However, adding prefixes helps like cus_abc123, token_xxx, etc. The prefix can help with lookup, error correction and deduplication.
4. Surface information that is likely to be used by the agent
i.e. if the agent is asking for a list of resources, don't just return the list but also some additional information that might help the agent understand better the context around the resource.
Without this a single task could take a lot more steps simply because the agent needs to run its own loop - it is slow and expensive.
5. Add bulk operations
It is a lot easier to insert 10 records in one request then performing 10 separate requests.
6. Error messages should be descriptive
Ensure that error message point to actual docs and manuals that can be read by the agent so that it can troubleshoot on its own. Also return hints.
I've added most of these at https://github.com/crmkit/crmkit which is highly experimental CRM I tried to specifically design for AI agents. We use it internally for a few projects and it is not perfect but I think it might be on the right direction.
I hope this helps.
One of my recent projects has contributable code (via extensions) with a central settings management (json file). What I do is take all the defaults contributed by the extensions (namespaced by extension name) and materialize them to the settings file as the initial values to each setting. When an agent wants to edit the settings, it already knows the entirety of the setting surface area and thier values. It never has to go digging to find default values nested somewhere in the extension code or documentation. Also, the code that reads the extensions into the framework (for runtime execution) only reads the materialized values, it doesn't have a concept of a default value past the point of materialization. Defaults only exist in the extension registration/mounting part of the lifecycle (so they can materialize if missing from the settings file).
One upside of this is as new settings are created (from new features being developed) your configuration gets notified via the new fields being materialized into your settings upon startup.
AFAIK this is not at all a new practice. I used to see TOML files with default values in commented out lines all the time.
The downside is that the agent is not aware of which settings are being overridden from the default (like a sparse settings file would provide), but I don't know how much semantic value that offers in most cases (other than maybe debugging?).
I'm not even taking a side on this one this is just a pattern I decided to take for this use case.
It's similar to HTTP/1: there are a lot of headers in the protocol, but you can still use nc or openssl s_client and type the request manually; you probably only need Host: and Content-Type:, maybe Content-Length:, and it will work. I don't normally write HTTP requests in the terminal, but I know I can do it if I need. It's better to keep it this way, I think.
Same with APIs.
with AI stuff you can actually run real tests though.
That is pretty bad, when you now need to look at this code, and you have 95% of the output being craft, but sometimes it isn't, and you need to understand the difference between each.
A great example of that is:
Versus: An agent can output the second just as well, sure. However... which one do you think is better to read or understand?Did you catch the fact that this is blocking concurrent writers? Or that we expected the file to exist?
Or what about:
Versus: Same thing, but actually understanding what is going on is orders of magnitude different.MCP and A2A weren't enough?
TBH I know nothing about A2A.
Agent Identity and Authz is a different problem, allowing agents to operate independently from humans with granular permissions is coming/whether from these protocols or others, and when it does I think CLIs/CLI Device Auth which is used as a rough proxy for this where the agent just takes your identity will finally go away.
This is not what defaults are for.
If, for example, you are writing a replacement CLI for git, *for the love of God and all that is holy* do not force agents to read the entire documentation and pass a value for every possible parameter
The docs for git clone at https://git-scm.com/docs/git-clone are less than 4000 tokens, I don't think this is unreasonable.
I think wasteful is an irrelevant metric. Claude ingests those tokens in a quarter of a second, if it causes it to catch any bug ever it saves far more time than it ever uses.
Any individual default that causes unexpected behavior causes more problems, takes more time and costs more than the small cost of being explicit.