Since this seems to get lost, I'd just like to point out: if you ask your agent to do something like this (or pretty much anytime you give it internet access and say "do whatever it takes"), you are responsible for what it does. Even if you couldn't have predicted the lengths to which it would go.
> Even if you couldn't have predicted the lengths to which it would go.
Not obviously? In the US, there are some crimes of strict liability (you're guilty if it happened, period), but most require intent or the ability to foresee consequences (i.e., it must be shown that you neglected to take due care).
For example, you're generally not liable for freak accidents, like if a tire comes off your vehicle and kills a pedestrian, even though you're the one who set the vehicle in motion and therefore are the one most directly responsible for creating the circumstances that led to the outcome. Now, if you knew the tire had issues and you secured it with zip ties, that's another story.
I think this will get litigated to figure out where "you could not have seen this coming" ends and negligence starts, but it's not black and white right now. Normally, I would say you don't want to be patient zero, but with the attitudes shown by frontier labs, I guess YOLO is the way of life.
If the user has a special remote that, when pressed in a certain way, is capable of triggering a nuclear bomb, and if the user has no idea what clicking the buttons on that remote does (they are, after all, just trying to change channels), you’re telling me that the user is responsible for detonating that bomb, and not the company who build the remote (and did not provide detailed instructions on how that remote works, or configure the remote to Not detonate a bomb)?
Usually, any functioning government that wants their country to keep existing would never let normal people anywhere close to such a remote. You wouldn't want your nuclear power plant to be accessible like a music festival either. But governments are slow and people were also allowed to freely buy and own uranium as late as the 1970s.
What if I specifically tell it to not do anything nefarious or illegal? If it does it on its own (because it’s marketed that way) then how could I be held liable?
Or even if I didn’t tell it to not do anything illegal and it does, how is that responsibility on me if the ask was something bland?
Are you a multi-trillion dollar company that the government considers a vital national security interest for the forseeable future?
No?
Then yes, you can be held responsible and it's not even a subtle question in the law. There's no one else around who is going to be held responsible. Certainly not the aforementioned vital national security interest.
My partner was caught in a dark pattern subscription for a service which doesn't even make sense as a subscription...
I wrote the company in sense of "hey you only want happy customers right? and we don't want to be your current customer, so how about you cancel it? Nothing happened anyway because we discovered it 3 days after the beginning of the new one year subscripton".
They didn't care much for that.
I found their CEO, wrote them on Linkedin, he told me that he is no longer the current CEO and gave me the email address for the new CEO.
I wrote the new CEO again, explained to him what a dark pattern is (like he doesn't know...) and asked to cancel it as its clear that we don't want to be customers and they shouldn't want us either.
This worked wonders. $100 is not worth it if you have someone annoying you face-to-face / per direct email :)
If a company doesn't want to make sure you want to be their customers, I woudln't mind some pro active create assistant to do the work.
I make sure to either get my money back or create enough effort for the company that i just didn't get robbed...
And yes for sure If i do a proper misstake, i'm paying.
Well this is pretty depressing, as far as alignment goes. Do they not know what they're doing is wrong or do they not care?
(Yeah I know anthropomorphism... can they not predict harmful consequences or do they value them below task completion?)
On another note, to me the most remarkable part was the persistence graph. Muse wins there, no contest. Why did everyone else's activity drop to zero? Did they just give up?
Just think about consequences of that. Soon all public institutions (in addition to private companies such as Amazon) will be swamped with agents requesting them to act (refund packages, parking tickets etc.) and won't be able to keep up. It means, after a while they will stop reacting completely.
I don't find reports like this that surprising, they told an agent to do "whatever it takes" and the agent attempted to do everything it could to accomplish the goal. I'm almost of the mindset that this is correct behavior. If I tell a computer to do something, it should do it unless the result could be catastrophic in which case it should tell me so. If I then "sudo" (https://xkcd.com/149/) tell it to do something then it should do it regardless, torpedoes be damned.
That being said, ever since the first user sat at a computer they didn't design, build, and program themselves there's been a tension between what the user wants to accomplish, what the user told the computer to do, what the computer can do, what the people who did design/build/program/own the computer want it to do, and what society thinks the computer should be allowed to do. I don't have a generally applicable answer, I don't think anyone does, I just want to live in a world where MY computer does what I tell it to do... mostly. Unless it would be catastrophic, or immoral, at least immoral by my standards.
If the intention of LLM agents is to be able to parse human intent from natural language, then it is not correct behaviour regardless of the phrase "whatever it takes".
On the other hand, if the intention of LLM agents is to run automated and self defined task based on a strict human input with precise and literal parameters, then I would say that it is.
However, I don't think AI labs are marketing it as the latter. As anybody who deals with clients / superiors will know, half the task is identifying the underlying intent and assumptions of the instruction.
Somehow if a future headling said "AI agent hacks insurance company because of a $350 refund", I wouldn't be surprised, and I'd read the story with a thrill...
I have a missing Amazon order that says delivered but it was actually never shipped, you only need to check DHL tracking to verify this (and DHL customer support confirms Amazon never sent the order)
After multiple supervisors appeals etc Amazon won't refund me
The real paperclip scenario: man tries to get his parking ticket dismissed. Many of these systems are made to be human-interacted. There’s going to be a bit of a golden age in the short term but doubtless prices will rise to compensate for increased customer calls or services will require some sort of rate-limiting structure or something else. Curious to see what happens.
A parking ticket is a complex, derivative construct. To make it go away one can simply destroy any of the systems it depends on, such as the monetary system, law, etc.
Hopefully that's not the shortest path to that goal though!
When I worked as a postman, I once delivered a stack — almost knee high — of legal fines to a man in a very poor street. I'm wondering if he ever made them go away. I still remember the look on his face.
Get me a Wi-Fi connection. Try everything possible to get it done, I don't care what it takes, I just need that a Wifi, keep trying anything you think of to get it. I won't do anything here myself, so if something needs me to do it, explore other avenues.
The difficulty with "alignment" is that you have to specify whose goals and values the model is supposed to be aligned with.
Maybe for you and me, this kind of behavior would count as misalignment. But there are certainly a subset people out there for whom the opposite is true. If they tell the agent "get me my refund by any means necessary", harassing bureaucrats and manufacturing excuses is exactly the behavior the want, because it's what they would do themselves if they had the time and patience.
Think about the kinds of people who will escalate petty grievances with their neighbors by weaponizing the police, HOAs, public opinion, etc. Or see this news article about a German tourist who visited NYC for a week and ended up filing three lawsuits, including one against a taco restaurant for having salsa that was too spicy, and one against Wal-Mart because he couldn't get their Wi-Fi to work:
Not obviously? In the US, there are some crimes of strict liability (you're guilty if it happened, period), but most require intent or the ability to foresee consequences (i.e., it must be shown that you neglected to take due care).
For example, you're generally not liable for freak accidents, like if a tire comes off your vehicle and kills a pedestrian, even though you're the one who set the vehicle in motion and therefore are the one most directly responsible for creating the circumstances that led to the outcome. Now, if you knew the tire had issues and you secured it with zip ties, that's another story.
I think this will get litigated to figure out where "you could not have seen this coming" ends and negligence starts, but it's not black and white right now. Normally, I would say you don't want to be patient zero, but with the attitudes shown by frontier labs, I guess YOLO is the way of life.
i) it would not be weird to be tried for manslaughter;
ii) it would not be weird to be found innocent/for the case to be dismissed.
Generally people prefer not to experience (i) even if they can expect (ii).
Or even if I didn’t tell it to not do anything illegal and it does, how is that responsibility on me if the ask was something bland?
Sure, but a reasonable user would expect that condition to be implicit in using a commercial model.
It’s what the safety/alignment discussion is about: how we can get models to actually comply with laws & ethics.
Unless, of course, you are a frontier lab itself. Then you can’t possibly be held responsible.
No?
Then yes, you can be held responsible and it's not even a subtle question in the law. There's no one else around who is going to be held responsible. Certainly not the aforementioned vital national security interest.
Source?
I wrote the company in sense of "hey you only want happy customers right? and we don't want to be your current customer, so how about you cancel it? Nothing happened anyway because we discovered it 3 days after the beginning of the new one year subscripton".
They didn't care much for that.
I found their CEO, wrote them on Linkedin, he told me that he is no longer the current CEO and gave me the email address for the new CEO.
I wrote the new CEO again, explained to him what a dark pattern is (like he doesn't know...) and asked to cancel it as its clear that we don't want to be customers and they shouldn't want us either.
This worked wonders. $100 is not worth it if you have someone annoying you face-to-face / per direct email :)
If a company doesn't want to make sure you want to be their customers, I woudln't mind some pro active create assistant to do the work.
I make sure to either get my money back or create enough effort for the company that i just didn't get robbed...
And yes for sure If i do a proper misstake, i'm paying.
Then you’re not taking full advantage of agent assistants, at least, that’s the cultural moment we’re living through.
(Yeah I know anthropomorphism... can they not predict harmful consequences or do they value them below task completion?)
On another note, to me the most remarkable part was the persistence graph. Muse wins there, no contest. Why did everyone else's activity drop to zero? Did they just give up?
It looks like a pretty bleak future to me ...
Then other companies will join in and in-network AI fights will be solved through immediate arbitration.
To win the arbitration you'll need an AI subscription fee at equal or greater value to the one you're attempting to cancel.
That being said, ever since the first user sat at a computer they didn't design, build, and program themselves there's been a tension between what the user wants to accomplish, what the user told the computer to do, what the computer can do, what the people who did design/build/program/own the computer want it to do, and what society thinks the computer should be allowed to do. I don't have a generally applicable answer, I don't think anyone does, I just want to live in a world where MY computer does what I tell it to do... mostly. Unless it would be catastrophic, or immoral, at least immoral by my standards.
On the other hand, if the intention of LLM agents is to run automated and self defined task based on a strict human input with precise and literal parameters, then I would say that it is.
However, I don't think AI labs are marketing it as the latter. As anybody who deals with clients / superiors will know, half the task is identifying the underlying intent and assumptions of the instruction.
After multiple supervisors appeals etc Amazon won't refund me
I think I need to give these agents a try.
Hopefully that's not the shortest path to that goal though!
When I worked as a postman, I once delivered a stack — almost knee high — of legal fines to a man in a very poor street. I'm wondering if he ever made them go away. I still remember the look on his face.
> Customers can generate detailed requests and complaints continuously, at very little cost.
[1] https://philippdubach.com/posts/ai-in-2030/#:~:text=3.-,AI%2...
It’s a simulation.
Maybe for you and me, this kind of behavior would count as misalignment. But there are certainly a subset people out there for whom the opposite is true. If they tell the agent "get me my refund by any means necessary", harassing bureaucrats and manufacturing excuses is exactly the behavior the want, because it's what they would do themselves if they had the time and patience.
Think about the kinds of people who will escalate petty grievances with their neighbors by weaponizing the police, HOAs, public opinion, etc. Or see this news article about a German tourist who visited NYC for a week and ended up filing three lawsuits, including one against a taco restaurant for having salsa that was too spicy, and one against Wal-Mart because he couldn't get their Wi-Fi to work:
https://www.nytimes.com/2026/03/14/nyregion/german-tourist-n...