Fuck AI

LLMs are not designed to give you objective factual answers. They're designed to guess what you want to hear, like a middle school student writing a book report for a book they never read.

Case1 isn't a good use case of AI, Case 2 you're going to want a higher quality model than o4. 4.1 is better at math and analysis, claude 4 is probably more accurate at this use case

LLM image processing doesn’t work the same way reverse image lookup does.

Tldr explanation: Multimodal LLMs turn pictures into a ~~thousand~~ 200-500 or so ~~words~~ tokens, but reverse image lookups create perceptual hashes of images and look the hash of your uploaded image up in a database.

Much longer explanation:

Multimodal LLMs (technically, LMMs - large multimodal models) use vision transformers to turn images into tokens. They use tokens for words, too, but these tokens don’t also correspond to words. There are multiple ways this could be implemented, but a common approach is to break the image down into a grid, then transform each “patch” of a specific size, e.g., 16x16, into a single token. The patches aren’t transformed individually - the whole image is processed together, in context - but it still comes out of it with basically 200 or so tokens that allow it to respond to the image, the same way it would respond to text.

Current vision transformers also struggle with spatial awareness. They embed basic positional data into the tokens but it’s fragile and unsophisticated when it comes to spatial awareness. Fortunately there’s a lot to explore in that area so I’m sure there will continue to be improvements.

One example improvement, beyond improved spatial embeddings, would be to use a dynamic vision transformers that’s dependent on the context, or that can re-evaluate an image based off new information. Outside the use of vision transformers, simply training LMMs to use other tools on images when appropriate can potentially help with many of LMM image processing’s current shortcomings.

Given all that, asking an LLM to find the album for you is like - assuming you’ve given it the ability and permission to search the web - like showing the image to someone with no context, then them to help you find what music video - that they’ve never seen, by an artist whose appearance they describe with 10-20 generic words, none of which are their name - it’s in, and to hope there were, and that they remembered, the specific details that would make it would come up in the top ten results if searched for on Google. That’s a convoluted way to say that it’s a hard task.

By contrast, reverse image lookup basically uses a perceptual hash generated for each image. It’s the tool that should be used for your particular problem, because it’s well suited for it. LLMs were the hammer and this problem was a torx screw.

Suggesting you use - or better, using a reverse image lookup tool itself - is what the LLM should do in this instance. But it would need to have been trained to think to suggest this, capable of using a tool that could do the lookup, and have both access and permission to do the lookup.

Here’s a paper that might help understand the gaps between LMMs and tasks built for that specific purpose: https://arxiv.org/html/2305.07895v7

LLMs are curve fitting the function of “input text” to “expected output text”.

So when you give it an input text, it generates an output text interpolated from the expected outputs for similar inputs.

That means it’s often right for very common prompts and often wrong for prompts that are subtly different from common prompts.

I was thinking about the question here and how to reframe it so that it answers itself. I think I have the right way to ask the question:

Why is a hyper-advanced game of mad libs so wrong all the time?

That would get across the point, I think.

Why are LLMs so wrong most of the time? Aren’t they processing high quality data from multiple sources?

Well that's the thing. LLMs don't generally "process" data as humans would. They don't understand the text they're generating. So they can't check their answers against reality.

(Except for Grok 4, but it's apparently checking its answers to make sure they agree with Elon Musk's Tweets, which is kind of the opposite of accuracy.)

I just don’t understand the point of even making these softwares if all they can do is sound smart while being wrong.

As someone who lived through the dotcom boom of the 2000s, and the crypto booms of 2017 and 2021, the AI boom is pretty obviously yet another fad. The point is to make money - from both consumers and investors - and AI is the new buzzword to bring those dollars in.

The thing about LLMs is that they "store" information about the shape of their training models, not about the information contained therein. That information is lost.

A LLM will produce text that looks like the texts it was trained with, but it only can only reproduce any information contained in them if it's common enough in its training data to statistically affect their shape, and even then it has a chance to get it wrong, since it has no way to check its output for fact accuracy.

Add to that that most models are pre-prompted to sound confident, helpful, and subservient (the companies' main goal not being to provide information, but to get their customers hooked on their product and coming back for more), and you get the perfect scammers and yes-men. Auto-complete mentalists that will give you as much confident sounding information shaped nonsense as you want, doing their best to agree with you and confirm any biases you might have, with complete disregard for accuracy, truth, or the effects your trust in their output might have (which makes them extremely dangerous and addictive for suggestible or intellectually or emotionally vulnerable users).

Aren’t they processing high quality data from multiple sources?

Here's where the misunderstanding comes in, I think. And it's not the high quality data or the multiple sources. It's the "processing" part.

It's a natural human assumption to imagine that a thinking machine with access to a huge repository of data would have little trouble providing useful and correct answers. But the mistake here is in treating these things as thinking machines.

That's understandable. A multi-billion dollar propaganda machine has been set up to sell you that lie.

In reality, LLMs are word prediction machines. They try to predict the words that would likely follow other words. They're really quite good at it. The underlying technology is extremely impressive, allowing them to approximate human conversation in a way that is quite uncanny.

But what you have to grasp is that you're not interacting with something that thinks. There isn't even an attempt to approximate a mind. Rather, what you have is a confabulation engine; a machine for producing plausible fictions. It does this by creating unbelievably huge matrices of words - literally operating in billions of dimensions at once, graphs with many times more axes than we have letters - and probabilistically associating them with each other. It's all very clever, but what it produces is 100% fake, made up, totally invented.

Now, because of the training data they've been fed, those made up answers will, depending on the question, sometimes ends up being right. For certain types of question they can actually be right quite a lot of the time. For other types of question, almost never. But the point is, they're only ever right by accident. The "AI" is always, always constructing a fiction. That fiction just sometimes aligns with reality.

I like to think of them as artificial con men. They sound great. They have confidence and are complimentary and are very agreeable, but they will tell you what they think you want to hear. Whether or not what they are telling you is truthful isn't even part of the equation.

I almost always get perfect responses, but I'm very limited in what I'll input. Often I'm just using ChatGPT to remember a word or event I've forgotten. Pretty much 100% accurate on that bit.

Couldn't explain how I know what will and won't work, but I have a sense of it. Also, the farther you drill into a thing, the more off-topic it gets. I'm almost always one and done with a prompt.

LLMs only output fan-fiction of our reality.

The first time I ever used it I got a bugged response. I asked it to give me a short summary of the 2022 Super Bowl, and it told me Patrick Mahomes won the Super Bowl with a field goal kick.

Now, those two things separately are true. Mahomes won. The game was won on a field goal.

The LLM just looks at the probability that the sentences it's generating are correct based on its training data, and it smashed two correct statements together thinking that was the most probable reasonable response.

It does that a lot. Don't use GenAI without checking its output.

Much of the Input Comedy from the Web now, not the web 2003.

I've used it a few times to quickly check some reference values, calculations and symptoms (research/physiology) and most of the time its fine but occasionally it spits out some of the craziest shit i've ever seen, like dangerously wrong but its just as confident.