Who cares? Might as well so how many organic grass fed Nazis there are. FOSS doesn't clean off the stink of AI
this post was submitted on 01 Sep 2026
0 points (NaN% liked)
LocalLLaMA
5323 readers
1 users here now
Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
Rules:
Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.
Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.
Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.
Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.
founded 3 years ago
MODERATORS
Your brain on Lemmy
If the dataset is ethical and can be inspected, and the model is self-hosted, what is your objection
I don't think this is an easy debate to have. Even those kinds of models have effects on society, on people, on the environment. A lot of resources are needed for research and training. Same for inference even if you own the computer. And we'd need to agree on what's ethical. Usually that means treat the entire world like a self-service outlet and just take everything unless someone called you and explicitly said no. At that point it'll become a nasty technical problem because there's no easy way to get rid of it after the fact. They could in theory be ethically sourced, but used for evil. Which is gonna be a problem to society. And all of the debate is hypothetical because that's not even what's in use out there...
My post was more concerned with the facts. What kinds of models we do have. Not what we should do and an universally accepted definition of ethical.
Looks like they go over their data sets here:
https://www.sciencedirect.com/science/article/pii/S1877050925006210
https://huggingface.co/togethercomputer
Hopefully someone else knows more, I don't really care, I just want Europe to catch up to China/America otherwise it won't matter how ethical the LLM is because everyone will be using Chinese/American anyway like 99% of people already are
LLM 360 probably deserves a mention there, trying to be open, reproducible and publishing intermediate training checkpoints.
meaning they’re not made up of unethically sourced training material
I am old enough to remember the days before 'piracy' was considered unethical. I still stand by that. Authors need to be compensated but also "publication" is supposed to give some rights to the "public". Sharing non-commercially things you love over internet used to be a gray zone that should have been the subject of a political debate. Instead, right holders decided that the letter of the law should strictly apply to conditions it was never designed for. I have little empathy for them moaning now that AI training, while not formally illegal, should be considered unethical.
A copyright reform is long overdue. Let's discuss it seriously instead of playing the game of "who can lobby the most?". In the meantime, enjoy the gray zone that refusing to update 19th century laws to the century of internet causes.
I was trying to reply to this, bringing up the controversy that erupted when image generators were able to use prompts with artist names to simulate their style, and how that shows how blurry the copyright line is. As I wrote it, I realized how complicated the issue really is, and how little I understand current copyright laws. Some may argue there isn't any vagueness, an artist's work in any format is theirs and shouldn't be copied for other use, training or otherwise. But we all use other people's work in our own, don't we? There is no solid line where one is ethical and one isn't. Plagiarism without proper attribution has always been wrong, but if you take the same content and tweak it a bit, it's not. LLMs can and do that, through statistical randomness, while using someone else's material, just like human brains would.
The compensation issue is definitely a problem and where the ethics come into play. Even if many people freely read a book in a library, it has been bought at some point. But how do you determine who was unfairly "stolen" from? Isn't it basically everyone at this point?
I know I'm way too nitpicky... But I feel I should point out there's some confusion hidden here...
There's stuff which is moral to do. And stuff which is legal to do. Those aren't the same! When talking about who can mention what, what copyright law demands people to do and those blurry lines, we're concerned with legality. That doesn't have anything to do with ethics. (At least not directly.)
Compensation is actually the only issue. If they were fairly paid, most artists would be ok with people enjoying their works in any way they can.
Many of the Chinese LLMs have open source models, even frontier models.
Open source or open weights?
Please read the whole post. OP is asking for the datasets used to train the models.
Uh, that one looks nice. Thanks!
I recall OpenAI saying to a courtroom judge that modern AI requires training off copyrighted material. And obviously the AI labs cannot just release those.
Distlled models (aka models that train themselves off the responses from other AI models) could probably release all the AI responses they used for training, since afaik AI generated content is not copyrightable..
However I suspect that most modern models are built from previous models. For example Claude Fable is probably based on Claude Sonnet or something. And for a downstream model to be fully "open source" and reproducible, you would need to reproduce the upstream model too. So even if a downstream model didn't use copyrighted data (eg it only used distills), one of the upstream models likely did.
They could still release a manifest of hashed contents and metadata.
What would be an application for this? More transparency for court cases? Or enable people to search for an alternative source for the data, like a torrent?
It would allow users to audit the dataset for ethical concerns, to interrogate the set to understand what a model does and does not "know", to identify weaknesses and biases, and to ensure that nothing has been inserted specifically to engineer prompt responses. Just examples off the top of my head.
Knowledge should be relatively impossible, though, if you just get the names and hashsums of the files? I mean I guess you could look up if a specific file you already have is in it. Might have some application. It needs to be very detailed, though. If they just call it GitHub-dump-timestamp or Wikipedia-dump-2026 with some overall hash, that doesn't tell us anything.
Most resources have a URI, Universal Resource Identifier. Related to a URL, except that it doesn't necessarily tell you how to locate the resource it identifies, URIs have dozens of schemas (eg isbn: for books) that can identify virtually any kind of media. Ideally they would tell us the URI, all metadata, source/provenance/licensing information, and the hash of the resource. If they truly are training on anonymous unattributed blobs that they just mysteriously found in their dataset, well that would be interesting to learn.
I see Olmo by AllenAI frequently used in research. The publish all the checkpoints of training and all training sources. https://allenai.org/olmo / https://huggingface.co/collections/allenai/olmo-3
Here's the link to the dataset used to train Olmo 3: https://huggingface.co/datasets/allenai/dolma3_mix-6T
It's fairly modern. Olmo 3.1 was released January 2026.
Nemotron is a fairly significant model family in this space, IIRC. e.g. here's one of the recent releases: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
Huh. It doesn't come with all the datasets, though. Does it? The model card contains a long list of "Private Non-publicly Accessible Datasets", both by NVIDIA and by third parties. And some of the stuff just says "Undisclosed".
"Private Non-publicly Accessible Datasets"
Huh. I thought they were fully open. I missed that. Most of the data sets and methodology is open though. (Maybe I'm remembering an older model? Or maybe I just fell for their marketing a while back... 😔️)
NVIDIA's own proprietary stuff looks largely like machine translation examples, and some sort of game logs (Sudoku and Minesweeper 🤨️), plus a few things I'm not quite sure I understand ("Multitool Typewriter Hard"? Is that for OCR maybe? Edit: or maybe they mean 'multitool' in an agentic sense... and that's for tool call reasoning?).
The third-party stuff is these six entries:
- Global Regulation
- TAUS Translation Memory
- Scale HLE
- HackerRank Coding
- RL data for Search
- Mercor SWE-AgentsV1
I wonder how hard it would be to rebuild something equivalent that is fully open. 🤔️
Sudoku and Minesweeper could very well be some internal project names and have nothing to do with what the words mean. I have no clue.
Seems to me their announcements ooze with marketing speech. I bet they put it all in one sentence to mislead about the datasets. Put a comma there and omitted the word "partly". It's technically borderline correct. I'll have to read up on it, whether that's a good amount of data, or just some tiny fraction. It's not looking good, though.
It could be, but using the games as logical puzzle solver/reasoning tests might have value -- I don't know.