He proposed a moon cannon. The moon cannon was wrong, as wrong as thinking an LLM can have any fidelity whatsoever. That's all that's needed for my analogy to make the point I want to make. Whether rockets count as artillery or not really doesn't change that.

Cannons are not rockets. LLMs are not thinking machines.

Being occasionally right like a stopped clock is not what "fidelity" means in this context. Fidelity implies some level of adherence to a model of the world, but the LLM simply has no model, so it has zero fidelity.

I feel this also misses something rather big. I find there's a huge negative value of people I have to help through doing a task - I can usually just get it done at least 2x if not 5x or more faster and move on with life. At least with a good intern I can hope they'll learn and eventually actually be able to be assigned tasks and I can ignore those most of the time. Current AI can't learn that way for various reasons, some I think technical, some business model driven, whatever. It's like always having the first day on the job intern to "help".

The other problem is - unless I have 0 data security rules, there's just so much the AI cannot know. Like I thought today I'd have Claude 3.7 thinking write me a bash script. I wanted it to query a system group and make sure the members of that group are in the current users .k5login. (Now, part of this is me not knowing how to prompt, but it's also stuff a decent intern ought to be able to figure out.) One, it's done a lot of code to work out what the realm is - this is useful generically, but is just code that could contain bugs when we know the realm and there's only one it'll ever operate in.

I also had to re-prompt because I realized it misunderstood me the first time, whereas I think an intern would have access to the e-mail context so would have known what I meant.

Though I will say it's better than most scripters in that it actually does a lot of "safety" stuff we would find tedious and usually have to have something go wrong to add in, so ... swings and roundabouts? It did save me time, assuming we all think it's method is good enough - but this is also such a simple task that I think in some ways it's barely above filling out a lot of boilerplate. It's exactly the sort of thing I would have expected to see on stack overflow back in the day.

EDIT: I actually had a task that felt 100% AI could have done... if there was any way for it to know lots and lots of context. I had to basically fill out a long docx file with often AI like text describing local IT security standards, processes, responsibilities and delegations. Probably over 60% I had to "just make up" cause I didn't have the context - for higher ups to eventually massage into a final form. But I literally cannot even upload the confidential blank form, forget about have some magic way for AI to get a brain dump from me about the last 10ish years of spoken knowledge and restricted wiki pages. Anything it could have made up mostly would have "been done" by the time I made a functional prompt.

I don't think we solve this till we can run frontier models locally at prices less than a human salary, with integrations into everything a human in that position could access.