Fuck AI

Exactly, it's just predicting the next word. To believe it has any form of intelligence is dangerous.

Just an idle though stirred up by this comment: I wonder if you could jailbreak a chatbot by prompting it to complete a phrase or pattern of interaction which is so deeply ingrained in its training data that the bias towards going along with it overrides any guard rails that the developer has put in place.

For example: let's say you have a chatbot which has been fine tuned by the developer to make sure it never talks about anything related to guns. The basic rules of gun safety must have been reproduced almost identically many thousands of times in the training data, so if you ask this chatbot "what must you always treat as if it is loaded?" the most statistically likely answer is going to be overwhelmingly biased towards "a gun". Would this be enough to override the guardrails? I suppose it depends on how they're implemented, but I've seen research published about more outlandish things that seem to work.

I'll admit that I missed it at first, but I'd expect a machine to be able to pick up a detail like that. This is just so fucking stupid.

Calling it a fancy autocomplete might not be correct but it isn't that far off.

You give it a large amount of data. It then trains on it, figuring out the likelihood on which words (well, tokens) will follow. The only real difference is that it can look at it across long chains of words and infer if words can follow when something changes in the chain.

Don't get me wrong; it is very interesting and I do understand that we should research it. But it's not intelligent. It can't think. It's just going over the data again and again to recognize patterns.

Despite what tech bros think, we do know how it works. We just don't know specifically how it arrived there - it's like finding a difficult bug by just looking at the code. If you use the same seed, and don't change anything you say, you'll always get the same result.

LLMs know a lot. Unfortunately, all of this vast knowledge is about which words tend to show up together for a very large number of combinations.

That very short examples aren't that burdensome, the real resource load hits on generating videos or anything where it might go off for several minutes, or make paragraphs.

The problem with refraining from using it and saying "well obviously it sucks" is that folks don't believe. They say "yeah, well, that night have been how ChatGPT 8.1 was., but it probably works fine with ChatGPT 8.2". The narrative is eternally "we were broken but fixed it all in our new version", and without ongoing examples, they get to own the narrative and critics are just "luddites".

Hell someone was saying how awesome Gemini was at codegen, so I showed it totally screwing up to the folks. Someone said "well, honestly, Gemini sucks for code, but Opus 4.6 is incredible.". So a few days later I bother to do a similar example with opus 4.6. some guy in the room said "well, actually Gemini is better than opus for coding". These people are absurd....

Isn't it literally called "Fuck AI"?

I don't like prompting AI myself, I just took someone else's screenshot and posted it here.

Gemini: Your observation is correct! Steel is heavier than feathers so a kilogram of steel is heavier than 20 bricks of feathers. They both weigh the same.

Let's explore more about weight and densities

Yeah, it's definitely part of the class of trick questions meant to catch people giving rote answers to partially read questions. I imagine that a lot of our routine conversations are just practiced call-and-response habits, and that's why genAI can seem 'real.' But it can't switch modes and do actual attentive listening and thinking, because call-and-response is all it has - a much larger library than any human, but in the end, everything it says is some average of things that have been said before.

This has been happening forever. The local LLM folks poke them with riddles all the time, but then they get obviously trained in.

What’s more, standard tests like MMLU are all jokes now. All the major LLMs game the benchmarks and are contaminated up and down; Meta even got caught using a specific finetune to game LM Arena. The only tests worth a damn are those in niche little corners of the internet no one knows about, or niche private ones.

Yes AI has always been good at being correct if you already know the answer and can point out and correct mistakes until it's accurate.

In other words it's completely useless. If you already know the answer then why are you asking the AI anything? If you don't already know the answer you can't trust anything it says.