this post was submitted on 02 Nov 2026
17 points (94.7% liked)
TechTakes
2777 readers
167 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Well, they've also reached model sizes where scrapes of the entire internet aren't sufficiently large enough to train the model, so they also need more data. But yeah, they compute cost of actually training the models are prohibitively large.
yeah they're far past public-only sources, there's more than a few non-public sources going into this shit too. and still it's not enough to work and sort out the issues! almost as tho the whole thing is a bit of a silly idea
I'm glad there's finally some financial reckoning coming, I just hope I'm wrong about how utterly fucking bad it will be
If that paper about astra being able to make sense of other models' weights has any merit, at some point they became so desperate they actually started adding serialized open weight models to the training data.