developers who instruct agents

Not build, not use, but instruct. That's a very specific word with a very specific meaning. He's getting at intent. It's whomever intended to take an action that led to harm without the appropriate safeguards.

A parallel might be unprepared hikers requiring a rescue. Who pays?

The nuance will be when he has to address an accident or a user who took safeguards that weren't enough.

I wonder how we might split accountability...

Look, it's simple: The organization with the largest collection of venture-capital-funded lawyer goons isn't accountable.

Nuance in my echo chamber nooooooooo. Yea we're talking about a web application that is readily available and highly adopted by kids lol. A web applications thats far beyond their basic understanding, assisting them in doing things beyond there understanding. It's a tricky liability question. Since it is a liability question the owners and distributors should be liable not their unknowing customers.

The user, in all of those scenarios.

If the user said "check the weather" and the agent said "got it, hacking the Pentagon" then yeah the liability would be on whoever made the agent do that.

Everyone who’s ridden in a car sues the automakers who let driver assist violate laws and endanger us all?

Big ol Chadly YES.

If say it's more like self driving cars knowing the speed limit but being able to go over it.

I like this analogy. The "hammer" example (that the OP used) is too simple. A hammer has a single purpose and a single functionality.

A self-driving car is closer to AI agents. The car has a purpose (to drive) but the mechanism is much more complicated than a hammer. If a driver hits the gas and drive over a pedestrian, it's totally their fault. However, if the self driving car drives through a wall, the manufacturer should be held accountable.

Likewise with AI, if I tell it to write and ship code, which is exploitable: that's on me. If AI decides to disregard my input and decides to delete all my emails, the developer of the tool should be held accountable.

Kids can use hammers. We don't usually let them build bridges; or at least their bridges aren't trusted by the public. If a kid uses a hammer to hit someone, yes, they are responsible to some degree.

Of course, we have the age of responsibility: We understand that kids can make stupid decisions and lack the mental capacity we judge adults by. Kids have hit people with hammers. It's a thing that has happened, and we've already decided how we handle it.

Moving away from that analogy: there are cases that exists, non-hypothetically, where kids have been implicated by software they have written that was behaving autonomously. Unauthorized computer access also applies to the scripts and crawlers you write. There have been instances where good old-fashioned non-LLM programs have 'hacked' systems, by accident and sometimes indirectly; even sometimes causing meaningful financial harm. We take the circumstances, and the intent of the user, into account when it comes to legal culpability and prosecution.

That's not how any of this works.

You don't need to be a licensed driver to buy gasoline.

If you buy a can of gasoline and set a mountain on fire, you're still criminally liable for it.

And if you handle that gasoline in an appropriate manner and you safely store it, and it explodes on its own, then whoever manufactured that gasoline is liable for it.

Yes. But we often let kids off easy because they're kids, they don't really understand the impact of what they're doing.

Usually in those cases we'd go after the provider for letting kids run dangerous software unsupervised.