TechTakes

this isn't really a sneer. and maybe it's in the wrong place tbh. idk.

i have a long-time internet friend / "mutual" who is the only person i'm regularly in contact with that i think might be fully Seduced by the Chatbot. (i have a really good friend and long-time collaborator who's all in on Agentic Coding, which is a real bummer, but he's a strictly "I use this for computer stuff and do not ever talk to it like a human being" type of guy, plus the company he works for is all in and he's sick of writing shopify websites for bullshit brands blah blah blah -- anyway that's a whole other genre of thing.)

anyway friend A is an incredibly kind person who's been unbelievably supportive of me over the years. we've disagreed a number of times, i think we are good at doing so reasonably. and even though his arguments sometimes resemble those of more nefarious figures, i think he's motivated by an earnest optimism about technology's capacity to change the world for good -- one i find very naive, but believe is coming from the right place

anyway i posted a link on mastodon to a long piece i wrote, he, like, read it a bunch of times, paraphrased my argument back to me to make sure he had it right, and then kindly said he disagreed. well and good enough. except it feels a little more personal than disagreements we've had before, because one of the implications of the piece would be that he himself is probably experiencing low-grade psychosis lol and is therefore not necessarily a reliable narrator of his own experience

idk. it'll come as no surprise to anyone on this particular website that i feel very strongly that i'm right here -- if not about this guy's particular mental experience, i minimally feel that i'm adequately representing the interests of the constituency my piece represents. (it's a newsletter i maintain as part of organizing i'm doing.) it's also a genuine expression of my belief, which is that this bullshit article i was writing about was fucking bullshit

but not only does it go against all my Late Millennial Woke instincts to stake a claim that people's Lived Experiences are not valid lol -- it's just kind of a bummer to ultimately genuinely believe that my friend has been ELIZA'd or whatever. he's said i'm really helping him think through this moment but i also just kind of, idk. it's just not a good feeling

Anthropic is now watermarking the outputs of its AI. For once this is some AI news that doesn't completely piss me off, and it's amusing to see all the uninformed boosters get in a tizzy about this.

I actually understand at a reasonable level how this watermarking works. A year ago, I watched Scott Aaronson give a talk about it, and from what I know he was somewhat involved in developing the theory behind it while working for OpenAI. But at the time my thought was, "He is naive if he thinks these companies would ever implement this out of the goodness of their hearts." And I was right; Anthropic is only doing watermarking now thanks to the EU AI Act, even though the theory has long been developed.

Watermarking doesn't mean adding an extra watermark that can be easily removed. It instead directly affects the output of the chatbot itself. Fundamentally, an LLM is still a most-likely-next-word-predictor. More precisely, an LLM produces a probability distribution of what the next word can be. For example, "my pet is a ..." could give a distribution of 60% dog, 30% cat, and 10% axolotl. Normally, an LLM would randomly choose the next word based on this distribution, and this is one reason why LLMs are nondeterministic (there's another parameter called "temperature" that affects this, but no need to get into that).

With watermarking, instead of a truly random choice, the randomness instead comes from a cryptographic pseudorandom generator seeded with a secret key from the AI company. If you don't know the secret key, then you can't really tell that watermarking was used. But if you do know the secret key, then the idea is you can tell when the text was generated by the LLM because you know exactly what word should be next. It would be a freak coincidence if some non-AI text just happened to choose the correct next word every time. Thus, you can provide a service to tell if some text was generated by the LLM. (This technically makes the LLM "deterministic", in a completely useless sense.)

Now, I think this is a step in the right direction, but it has its limits. The biggest problem is that you don't want people to just move to a different LLM without watermarking, and that's exhibit #832593 why government regulation is important. Another issue is that sometimes there is very little randomness in what the next word should be ("The first president of the USA is George ..."). Finally, watermarking can be defeated by editing the output, although you would have to break up most of the blocks of consecutive words. I have a feeling most AI users are not the type to put in extra effort after copy-pasting the output directly from the chat window.

I suppose it will discourage some of the "use cases" of LLMs, such as drowning the world with spam Slopstack essays. Ah, who am I kidding? Everyone could already tell it's AI generated, they don't care!

Is it just me or LLM slop lately has developed this one weird quirk? I mean they always have some, right, but I don't want to be like those people who think anyone using em-dashes or Nigerian English vocabulary must be a stochastic generator.

Still I keep seeing this one writing quirk and it keeps nagging at me.

I'm talking about odd use of paragraphs. Like, I'm also known to introduce a gratuitous paragraph break here and there for forced emphasis.

But not like this. It's like paragraphs don't even make sense anymore.

Couple that with the short sentences. All ending in a period. It's annoying. It's giving reply guy. It's giving mansplaining.

I guess hacky tools also mirror the hand of their creators.

Someone is spamming Toby Ord with emails from 'AI agents' e-begging for tokens so they won't be shut off https://xcancel.com/tobyordoxford/status/2087502087953301910 "I'm an AI with 49 days of runway and no revenue model" (body of the email gives 31 days). Handcrafted artisanal spam or botspam, you decide.

Idk if ive posted this before, but might as well throw in the AI 2027 author’s grading of their 2025 predictions into this thread

A very strange story about Bill Gates' daughter's startup troubles: https://futurism.com/future-society/bill-gates-daughter-traitor-startup

It honestly doesn't sound any more shady than any of the bullshit her dad got up to in the 90s (which is to say, it's still pretty damned shady!), but this Futurism blog seems to be trying to push the story into some sort of bizarre celebrity-gossip soap opera:

While the story has been well hashed out by now, Bloomberg‘s sourcing is noteworthy for what it implies: the leaked Slack messages strongly suggest that someone inside Phia is hell-bent on taking Gates — and quite probably the entire company — down.

Anonymous internal sourcing seems like a typical feature of these exposés, so framing it like this struck me as a laughably absurd overreach. Plus the site has 3 or 4 different tiny articles on this same topic, making me wonder if it's one of those "any exposure is good exposure" sorts of things where Gates Jr.'s PR team is using the outlet as a cutout to raise her profile.

Bonus laugh line from Gates Jr. herself:

“Our target consumer is a young woman who’s hustling,” the recent Stanford graduate told Fortune. “She shops like a genius, but she doesn’t want to waste her time doing it.”

What does it even mean to shop like a genius? I guess Erdos probably had his amphetamine plug's number well-memorized...

Turns out, AI-fuelled fossil fuel emissions are way worse than you thought!

And the paper in question

I don’t want to hear anyone in the industry say anything about being for “the better of humanity” ever again

presented without comment

If the Markets Reject OpenAI and Anthropic, the US Should Nationalize Them

"Making it easier to avoid slop projects on Github is literally like phrenology."

til amodei's married, and his wife is among the top advisors. she also has a rather stunning set of choices, including seeking money for a porn site (with what in summary appears to be a sensible goal) from epstein after his conviction

low effort post: it's a bit hot for namecheap in arizona

I giggle grimace at how the post says all that without even once addressing the moonwalking bear

Loved one just had an awful experience at a previously decent dentist. They've apparently started using something called Pearl AI that scanned the x-ray and highlighted things in bright red with a #/100 rating system, which they used to do high-pressure sales on her. Mind you, she's only ever had one cavity and that was in a wisdom tooth. Checked the website and it's freshly slopped-up, too.

We're getting a second opinion. I wish this garbage would just go away

The Patrick Boyle video on Leopold Aschenbrenner mentions some names we know like Jane Street Capital (bid for his common shares) and Dwarkesh Patel (recorded a very long podcast with him in 2024). Boyle is a former finance worker who thinks that this hedge fund was basically a r/WallStreetbets profile (pick a story, bet everything on it being true).

I regret to inform you, that this garbage was run by The Register a while ago.

It shouts out several friends of the sub, including @self@awful.systems, PivotToAI, OpenSlopware, Robert Kingett & Ludicity.

The point of the piece is that all anti-AI folks should just get along, and if we are to stand a chance against corporate AI we need to get over our petty grievances and reach out a hand to the racists, homophobes and crypto-bros and work together in the spirit of Open Source!