this post was submitted on 03 Jun 2026
962 points (99.7% liked)
People Twitter
10365 readers
556 users here now
People tweeting stuff. We allow tweets from anyone.
RULES:
- Mark NSFW content.
- No doxxing people.
- Must be a pic of the tweet or similar. No direct links to the tweet.
- No international politcs
- Be excellent to each other.
- Provide an archived link to the tweet (or similar) being shown if it's a major celebrity, figure or any politician. https://web.archive.org/save is an option. Archive.is is no longer recommended because they remove/alter content.
- If a tweet (or similar) makes a statement of fact, either mark it as "satire" in the body or provide the original source link backing up the statement
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I run 27b at q8 with unquantized KV cache and 256k context on two Instinct MI60 GPUs. Definitely the best model that I have been able to run locally at a reasonable speed. 35b generates tokens as fast as you'd expect from any cloud provider. 27b is slower than 35b, of course, but token generation is still faster than my reading speed and suitable with coding agents.
The wattage is actually relatively low compared to a lot of current gen GPUs (mainly NVIDIA ones). They are software capped to 225W, but the GPUs can handle 300W. Compared to 5090 which is like 600W