This alarm's being rung for over a year now, so "calling it now" means finally reading the writing on the wall
this post was submitted on 16 Jun 2026
1 points (100.0% liked)
Web Development
5713 readers
4 users here now
Welcome to the web development community! This is a place to post, discuss, get help about, etc. anything related to web development
What is web development?
Web development is the process of creating websites or web applications
Rules/Guidelines
- Follow the programming.dev site rules
- Keep content related to web development
- If what you're posting relates to one of the related communities, crosspost it into there to help them grow
- If youre posting an article older than two years put the year it was made in brackets after the title
Related Communities
- !html@programming.dev
- !css@programming.dev
- !uiux@programming.dev
- !a11y@programming.dev
- !react@programming.dev
- !vuejs@programming.dev
- !webassembly@programming.dev
- !javascript@programming.dev
- !typescript@programming.dev
- !nodejs@programming.dev
- !astro@programming.dev
- !angular@programming.dev
- !tauri@programming.dev
- !sveltejs@programming.dev
- !pwa@programming.dev
Wormhole
Some webdev blogs
Not sure what to post in here? Want some web development related things to read?
Heres a couple blogs that have web development related content
- https://frontendfoc.us/ - [RSS]
- https://wesbos.com/blog
- https://davidwalsh.name/ - [RSS]
- https://www.nngroup.com/articles/
- https://sia.codes/posts/ - [RSS]
- https://www.smashingmagazine.com/ - [RSS]
- https://www.bennadel.com/ - [RSS]
- https://web.dev/ - [RSS]
founded 3 years ago
MODERATORS
yeah I was kinda like. calling it?
Let it be known that the first person to call it was actually Sam Altman when OpenAI's paper on AI Scaling Laws in 2020 subtly showed that the diminishing returns will stop showing improvement with infinite power, compute time, and data before 94% accuracy is reached.
No one paying the bills cares
Agents cannot program
This is just factually incorrect. Difficult to get past a false assertion of this magnitude.
They are a highly sophisticated statistical model designed to mimic the distribution of programming.
I thought we had got over the stochastic parrot nonsense by now.
You can totally have objections about the ability of AI to program - how good it is, poor failure modes, high cost, technical debt, knowledge debt, broken social contracts, etc. All valid.
But if you're still in the "It's just a next word predictor! It can't really think!" stage of denial, even now... Sorry you're an idiot.
Thank you for letting us know where you stand. Those who tag users can act accordingly to this public declaration of robo-handjob-giving.
Um... but it is just a sophisticated statistical model... that's literally what the math underpinning machine learning models is... and all it can do is make associations based on correlations within the field of the training data. That's what it does.
Um… but it is just a sophisticated statistical model…
The mistake has been thinking this implies LLMs can never do X task, and using it as a catch-all argument for any value of X, but it isn't a good argument because it has been wrong for most of those.
The mistake has been thinking this implies LLMs can never do X task
As this article points out, an LLM can spit out chunks of regurgitated code that it scraped from the internet, but that does not make the LLM a programmer. The resulting output is an attempt to find an existing pattern in the database which fits with what the user has asked for, but it is not a product of actually understanding the use case for the code. It is just statistical correlation.
So, sure, an LLM can be set up to generate output related to X task. If you can collect and clean data that can be used to train the kind of output you want, it should be able to produce an approximate facsimile of the results you want. Is that valuable for your use case? Maybe.
We're still just talking about what is essentially a complex search function. The statistical model returns results from its database that correlate most closely to your input. That does not mean it returns the right answer. If there is no good correlation, it will still return a result.
As long as you understand that the result you get is just a correlation based on your input and may or may not be relevant to your specific problem, and you are not fooled into believing that the LLM actually understands what you're asking and produced a result by "thinking" about it, then you might be able to use an LLM as an effective tool - to search a large collection of information for something that is relevant(ish) to what you're asking for.
The real mistake has been broad misunderstanding of what LLMs actually do, and trying to use them as general-purpose problem solving tools (or worse, as accurate and reliable sources of information).
load more comments
(3 replies)
This is like saying "but it is just a sophisticated network of neurons. All it can do is transform input signals into output signals. That's what it does."
Not really.
A machine learning model is a computer program. It is fundamentally a math equation, which we understand completely.
A living brain is not fundamentally a math equation, and is not purely a statistical model, at least not in any empirically demonstrable way. We don't understand completely how it works, but we do know that it's more complex than what you're trying to imply.
The comparison is not valid. Machine learning models are not an equivalent to a biological brain.
Tbf, LLMs may be comparable in complexity to their specific brain.
A machine learning model is a computer program. It is fundamentally a math equation, which we understand completely.
Lol you couldn't be more wrong about this. One of the most widely commented things about DNNs is that we don't really understand them completely. I don't know how you would miss that if you knew anything about AI at all.
A living brain is not fundamentally a math equation
It is. A very complicated one, sure. Which part of the brain do you think is impossible to simulate with maths?
load more comments
(1 replies)
Our apologies. Let us issue a correction:
"Agents should never be allowed to program"
There.
if it's broken in a way that can't be detected, is it actually broken?
all software is broken in some way. if the rate of bugs generated by llm and the severity of those bugs drops below the rate you would expect from a human programming team, then llm is offering something competitive.
It will eventually be detected, but it passes tests before hitting production, that is the problem.
broken in a way that can't be detected
Is not what anyone said and you're lying when you pretend they did.
It's so nice to see some people speaking reason. If only any of those people ran multibillion dollar companies.
Let them fail and then scramble to rehire.
You know the feeling that you want to rewrite a project? But you know that most rewrites are a bad idea.
Be it your own, old code. Or code you inherited.
There is a small chance that the world realizes that they went in the wrong direction and nothing can get fixed. That will be the time of rewrites.
No, I don't expect this to be very likely. The agent code will remain, and human programmers get yelled at for not fixing it fast enough.
This is very obvious unless you are in tech leadership, in which case your job is now to push this at all costs and suppress dissenting voices.
In tech leadership. I don't have to push it. My talented engineers took to it immediately.
They learned quickly that it is a tool. Instead of using a shovel and a wheelbarrow, they have a backhoe now. If you don't know how to dig a hole, the backhoe is just a way to make a mess faster. It doesn't replace intelligence.
They can use it to do the scutwork while they focus on the important stuff.
The duds are still typing shit into spreadsheets and emailing them as attachments while their coworkers are getting stuff done.
It is a tool. You can learn to use it or you can just be mad that it exists. In either case it isn't going away. Like the telephone, the car, the computer, the internet, it is here to stay.
If you’re letting your engineers find uses for it instead of constantly demanding that they generate lengthy “user stories” and decision documents and deferring thinking to agents instead of quickly planning stuff out using their experience then you’re probably quite an outlier by now.
I am a very lazy man, micromanagement takes so much fucking energy. Hire talented people, give clear and unambiguous guidance and trust them to do the work. It is amazing how easy management can be when you don't get in the way.
Being in leadership is way easier than being an engineer and the pay is better too. Some people really overcomplicate shit.
What's fascinating about this conversation is ... how do people think software used to be made ? With talented and knowledgeable developers who would never "hallucinate" an API or a library function ? With cybersec experts who would never put their user's data in jeopardy ? With performance investigators checking the computational complexity of each function ? Bitch please...
Software engineering is not the kind of mystical cathedral building these people have in mind, it's more like a musty workshop in Pakistan where they make tractor tires with no safety equipment and a cigarette in their mouth. We've been throwing imperfect humans in various states of lucidity at every problem known to man for 30 years but suddenly people start believing that their bog standard CRUD software should be written by monks having attained cosmic godhood.
Typing shit into spreadsheets isn't something software engineers typically do... And usually shit that's typed into spreadsheets has to be accurate. There are major consequences if not, and AI is not reliable enough for that kind of work.
Don't forget the vibe coders who can't tell that the Ai is doing it wrong
They are not the automated from 0 to 100 coders that some people claim them to be. But they are quite capable, definitely much more capable than what anyone could have imagined ten years ago. Given well defined problems they can excel at even relatively complex tasks. I pointed Claude at a latex file of a somewhat complicated nonparametric statistical estimate calculation to look for any mistakes and it was actually able to find some. I then pointed it at a code that replicates the calculations and it was also able to correctly identify some issues with the code. I think this is the way one should use LLMs, not let it loose on coding tasks. In the former way you won't even be able to burn through your first tier account quota where as in the latter the LLM will likely end up getting in weird loops burning tokens like there is no tomorrow. Also this method of sane usage of LLMs is much more suitable for open local LLMs. I don't think there is any doubt anymore that LLMs can be very useful tools, not just for doing stuff but learning it too. People should move past the stage of invalid criticisms like "they are just stochastic parrots" and move to more serious matters like environmental impact, greedy fucking CEOs pretending LLMs are replacements for humans, degredation of skills, getting lazy at checking AI code and the unsustainable AI bubble that tech companies are pushing for.
invalid criticisms like “they are just stochastic parrots”
That's not a criticism per se, it's a description of how they work.
Sure, and at that level of accuracy it's also a description of how humans work. I didn't invent these words myself, I'm just stringing them together based on a stochastic process my brain was trained into.
Like LLMs, some of my speech is semi-random initialization (dada wawa googoo), some of that is mimicry (some of that is mimicry), some of that is reinforcement learning (downvotes incoming), and some of that is the output of a subprocess that uses the same systems prompted at the meta-level and without verbalization (maybe they won't get the analogy between thinking and LLM scratchpads... how about I use this space to clarify).
Calling an LLM a stochastic parrot has the same social-emotional role as calling a human an animal. Yes, it is correct. But people can infer the connotation.
Humans are animals. LLMs randomly generate text based on the corpus they were trained on and the conversation so far, so stochastic parrot is an accurate description.
LLMs don't learn. Humans do. LLMs generate text randomly using a massive matrix. Humans don't; you lied. An LLM is incapable of lying because it has no understanding of truth. It just bullshits convincingly all the time. It's very very good at it, but it's all hallucinated for the LLM, true or false.
Expecting your random word generator to tell you truths is insane. The training measure is "sounds right" not "is right". It passes if it sounds like the other discourse it read. Just like the confident drunk guy at the pub who thinks he knows everything passes of he convinces the other drunk guys at the pub.
Whereas humans learn at school and on the job and the training measure is "your teacher or supervisor approves". LLMs were not trained on truth or accuracy. Trusting in them and treating them as equivalent to human intelligence, as you and a whole bunch of other folks do, is profoundly unsound, and soon the necessary price rises to pay for the processing costs (let alone the vast, vast, vast, vast, vast debts on the infrastructure) are going to make most slophouses which jettisoned their human talent go out of business. And very, very few people indeed will be sorry at that point.
Meanwhile LLM slop is shitting in github all day long, every day, and shitting on the internet, and it will eat it's own shit and produce crappier shit.
Your analogies don't change the truth, and that is that LLMs don't know the difference between sounds correct and is correct any more than MAGA voters know the difference between sounds good to me and is good for me.
What do you mean LLMs don't learn? How do you think they became capable of stringing a sentence together?
They don't learn during a deployment, but neither do humans; humans only learn during sleep. The behaviors a human exhibits while "learning" in the moment are just stochastic parrot behaviors based on their immediate context window, if the human doesn't sleep in time the event can slip out of their context window and they don't learn despite having acted as if they do.
You seem to be very naive about human learning in general. What makes the "truth" of school lessons greater than the "truth" of an LLM's curated dataset it is reinforcement learned on? Have you ever seen actual evidence that mitochondria exist, or are you just stochastically parroting your biology teacher?
I also oppose LLMs in almost all applications (live translation being an example of a good application). But please oppose it with arguments based in reality.
What do you mean LLMs don’t learn? How do you think they became capable of stringing a sentence together?
You're confusing constructing the LLM, which is done with an actual AI (neural network) and a massive corpus of text (stolen from millions of humans in the greatest intellectual theft in history) and running the LLM, which is done with a random number generator and a massive matrix of probable next words.
They don’t learn during a deployment,
They don't learn. They don't change. They're as random next time as this time.
but neither do humans; humans only learn during sleep.
False and false. Soooo much pseudoscience.
The behaviors a human exhibits while “learning” in the moment are just stochastic parrot behaviors
Wrong again.
if the human doesn’t sleep in time the event can slip out of their context window and they don’t learn despite having acted as if they do.
If that were true, most people would learn very badly first thing in the morning and get better and better later in the day. I think you'll find that most school teachers would vehemently disagree with your nonsense conclusions.
Then again, perhaps by "doesn't sleep in time" you mean stays up all night, then admittedly they might function less well cognitively but (a) we tend not to regularly torture humans that way and (b) you're massively overstating the role of sleep in the learning process.
You seem to be very naive about human learning in general.
No, you seem to be very naive indeed, to extremes, about the intelligence and reliability of LLMs. When I ask them about general things that I know about, I tend to get the right answer about 60%-70% of the time. Why would I believe it when I didn't know the answer. To trust an LLM to tell you the truth about stuff you aren't checking when it clearly blags nonsense so frequently when you are is really really stupid.
What makes the “truth” of school lessons greater than the “truth” of an LLM’s curated dataset it is reinforcement learned on?
Most teachers tend to consistently teach the content of the syllabus rather than randomise what they say to classes based on the preceding conversation. They reinforce and update their prior knowledge by also learning from the mark schemes of the tests and exams their students sit.
Have you ever seen actual evidence that mitochondria exist, or are you just stochastically parroting your biology teacher?
No. I trust my teachers. I am rational to do so. I don't trust LLMs. You are irrational to do so.
But please oppose it with arguments based in reality.
You are utterly deluded and have bought the hype. You seem unable to distinguish between distinct things and are dismissing a large amount of evidence that your "just as good as a human" is a crap-spewing shit machine, no more honest than donald J trump, and with no less sharting.
running the LLM, which is done with a random number generator and a massive matrix of probable next words.
Not true. Inference is done by providing the context to the pre-trained neutral network (technically a transformer network not your daddy's old multilayer perceptron) to generate possible outcomes with logprobs that are then selected based on their likelihood. If it was just frequency-based RNG, they wouldn't have any semantics in the responses and would sound more like traditional Markov chains (like when you mash a button on predictive text and it spits out correct but meaningless gibberish).
If it were just selecting random words from a matrix of probabilities without the network and attentions, it would also be waaay faster and easier to run on a potato.
The stuff about human learning also isn't quite right. There are different types of "learning" and different kinds of memory.
Sleep is generally understood physiologically to be required to formulate long term memory (eg. as described in this paper).
The previous commentator was analogising human short and mid-term memory with LLM context windows (also things like vector databases etc.) and long term memory with retraining/merging/fine tuning of LLMs. It's not totally the same but the analogy is accurate. Brain behaviour is a big influence and inspiration on how machine learning techniques are designed.
Human memory is also notoriously inaccurate and unreliable and tasks done by humans often needs to be double checked and externally verified.
This isn't to say LLMs are trustworthy or reliable. They are not. More that humans think much more highly of themselves than is really warranted.
Brain behaviour is a big influence and inspiration on how machine learning techniques are designed.
I repeat, the LLM is not doing machine learning while users are using it.
This isn’t to say LLMs are trustworthy or reliable. They are not.
We agree here.
More that humans think much more highly of themselves than is really warranted.
And we agree here too, but to trust an LLM to tell you the truth on your question that you don't know the answer is like trusting some random drunk at the pub, because you don't know whether the answer is from an LLM hallucination, a random lie/error on reddit or an expert's contribution to wikipedia.
And to trust an LLM when there's a trained programmer or professional journalist is stupid. Sure, an LLM might even sometimes write as good or better code than an intern, but again, the LLM is not learning from its mistakes as you correct it. The intern gradually becomes an expert. The LLM does not. Paying interns is an investment in future programmers, who get more expensive the more experienced they are.
The LLM is currently cheaper than the intern, but LLM pricing needs to go up by a factor of about ten to cover running costs let alone pay off the vastly more immense debts of buying all that hardware.
Sleep is generally understood physiologically to be required to formulate long term memory (eg. as described in this paper).
Like I said before, humans sleep every night, with rare exceptions. LLMs do not get retrained every night. The human brain adapts to feedback loops during everyday interactions, not just overnight. It's a silly analogy and this is a silly point to defend.
There are plenty of textbooks that say that volatile running RAM is like short term memory and hard disks and SSDs are like long term memory, but it would be silly to reverse the analogy as you are doing and claim that sleep is pressing the save button on the day's learning, or that this makes your word processor the same as your human intelligence because, and this is the central point you've been trying to argue around and about and against, they're doing fundamentally different things, and telling me one was inspired by the other doesn't change that. An LLM is fundamentally a stochastic regurgitator whose training is designed primarily to make it sound right. A human brain just doesn't work that way.
If you truly believe that the LLM is learning like a human or intelligent like a human, you are confusing analogies for reality.
the LLM is not doing machine learning while users are using it
This is a small terminology misconception. The LLM is not doing "training" during inference. It's still a "machine learning" system.
In terms of learning/retaining information in the short/mid term while the user is using it, as the context grows, it retains that information during the current session. In a lot of systems, sections of that context are then summarised and stored, indexed by a vector, to be retrieved into future contexts that have similar semantics. That's why some systems seem to be able to "remember" things from previous "conversations". Your message is vectorised and then that vector used to look up similar past interactions. The model isn't fine tuning on that, so it's not "long term" memory, but the model can take it into account for future interactions.
AI companies do then use that (and full conversation histories) to regularly fine tune the models, as well as train new ones. It might not be fresh trained every day but certainly more often than you might think.
to trust an LLM to tell you the truth on your question that you don't know the answer is like trusting some random drunk at the pub
They're a little more reliable than that and are getting significantly more capable at an alarming rate. We absolutely agree that they shouldn't be trusted and are not very accurate (nor should most humans be trusted or are accurate) but I also think it's dangerous to underestimate them.
They’re a little more reliable than that
Depends which drink guy at the pub you randomly pick. The attribute that they share with the drink guy at the pub is their reluctance to admit that they don't know or have no expertise or can't help you. Clever and experienced people know where their expertise ends and express self doubt when appropriate. LLMs don't. They can't. They're making literally everything they say up. It's probably right, but they are the script kiddie of conversationalists.
it’s dangerous to underestimate them
It's dangerous to underestimate their ability to sound good enough to convince executives to fire humans. It's dangerous to underestimate the scale of substitution of plausibility over knowledge that will only accelerate with further adoption. It's dangerous to assume that the interactions that middle and senior management have with staff that do actual work cannot be replicated already with a suitably trained LLM.
In terms of learning/retaining information in the short/mid term while the user is using it, as the context grows, it retains that information during the current session. ... past conversations ... remember ...
Those slashes are doing a lot of work in that sentence!
Humans learn by generalising from examples. Humans learn when you ask them well-designed questions. Humans learn by practising skills repeatedly. Holland park from their mistakes. Humans learn because they are in a constant state of feedback loop. Humans learn by watching other people. Humans learn by experimentation. Humans learn through playing with new things. Humans learn by talking to each other. Humans learn by sitting and thinking things through. Humans learn through thought experiments. Humans learn through seeing and hearing and reading more quickly than doing any of them alone. Humans learn by explaining things to other people, crystallising their experience into verbal solidity. Humans learn through discussion. Humans learn by learning who to trust and how to weight different input by source. Humans learn by learning how to learn more effectively.
LLMs do none of any of those things.
Machine learning is a very, very narrow form of "learning" and you're conflating the use of a neural network with actual learning, which you then compound by confusing the resulting LLM with the neural network that was used in its creation.
Pulling the wool over people's eyes about what an LLM is is at least as harmful as underestimating the ability of AI to be so plausible as to disrupt absolutely everything about how money moves around society.
load more comments
(5 replies)
part of the issue as well is that when they get something completely broken, people just re roll the output until they get something that’s broken in ways they don’t notice. Or re roll parts of it, or tell the system to judge if the output is broken and re roll the parts that it judges are broken automatically. Or increase the size of the context window to get it closer to that upper limit of accuracy.
All this together can get a more functional output with less effort, and as people find these tricks it gives them the illusion of an upward trend in capability, like this is all solvable issues that will improve as time goes on. Big problem with that though, theses tricks and methods explode the compute cost rapidly. That’s all fine and dandy when everyone is getting their compute costs for these tools subsidized by these model providers, but eventually they will need to charge the real cost of running this. The compute providers that host the model providers are also running at a loss, trying to help grow the market segment and maximize their market share. And then places that have the datacenters in them are giving tax breaks and discount utilities to attract new construction.
Everyone except the people making the chips is selling at a loss, and as people pile on usage to make up for the fundamental limitations of these systems, the demand balloons, validating to the providers at all levels that this is a growing market they should invest more in to.
But eventually… they need to make money. The bill comes due on all the debt and investment. What happens to the people who have fully embraced these to run their businesses? Or to all the people who have built their skill set around using these systems? It’s a crisis, a series of crisis, each time a debt wall gets hit by someone in the supply chain. A half decade of technical debt that just got really expensive to deal with, and not enough experienced people to handle it, since all the grey beared retired and not enough new people got brought in to replace them because the entry level work was automated.
It's going to be a wonderful time to be a Freelance Senior Developer and above in a few years.
remember, when you interview for a job and they ask you, "do you have any questions?", you ask;
- has AI ever been used to develop your product?
- what percentage of your product has been written by agenetic AI?
- is the use of AI tracked as a performance indicator?
The output is broken, but in a way that’s getting harder and harder to detect.
Introducing code homeopathy: "Yes those two programs are consisting of the exact same bytes, but this one contains pure virgin artisan essence (C20 distilled)"
geohot
Now there's a name I haven't heard in a long time. George Hotz was the guy who first jailbroke iOS and the PlayStation 3 and made the towelroot exploit for early versions of Android, before legal threats drove him out of the scene.
load more comments
(1 replies)
Calling them a statistical model is misrepresenting their functional understanding of concepts.
This guy does not get it.
He's bang on. LLMs have no understanding of anything: they are literally just statistical models.
Unpopular prediction: AI agents are going to get better at coding. Not great, but halfway decent at cranking out basic features. Once everything levels out in like 3-5 years, AI agents will be a cherished part of the toolbox most software developers. It will be useful for skimming code, it will be useful for tedious parts of tasks that are just a degree off from boilerplate.
People are definitely gonna try to use it for things more complicated than that, and that'll be a mistake, and it will be costly, but the far side of it could be pretty cool actually. Admittedly I have an optimistic disposition.
~~I have yet to be impressed.~~ I’m not very convinced*. I asked for format type mappings between Pipewire, WebGpu, and Vulcan and both ChatGPT and Gemini failed very badly only providing the most common type mappings. This should be a wildly easy task, something any programmer or even beginner programmers can complete. It’s just very boring, mundane, buffer shifting like work. It almost feels like pencil pushing.
Why couldn’t they do it?
Hence the "are going to get better" and "3-5 years".
Are you using real agents or the free chats on the web, because the latter ones are really dumb. Even when you ask them to search the web for basis you don't get much success.
load more comments
(2 replies)
But they do work, maybe not as a full replacement but my god the amount of boilerplate I can avoid in creating unit tests from scratch. Extracting and finding information in the code base is also useful, not everything is an easy text search of tracing a few code paths. It's an incredible tool for these kinds of work.
If it becomes harder to tell the difference then it also means it's closer to matching reality. And todays AI can do very impressive "reasoning", managing to debug complex issues I have had.
The most important part is that you as developer is fully responsible and can stand behind what they do and deliver using AI agents.
load more comments
(1 replies)
view more: next ›