this post was submitted on 21 Jul 2026
1 points (100.0% liked)

Technology

1467 readers
10 users here now

A tech news sub for communists

founded 4 years ago
MODERATORS
muad_dibber@lemmygrad.ml
burlemarx@lemmygrad.ml
egs81t@lemmygrad.ml
1
AI Companies Are Terrified. (youtu.be)
submitted 4 weeks ago by Makan@lemmygrad.ml to c/technology@lemmygrad.ml
31 comments fedilink hide all child comments
 

*Western AI companies are terrified

Anyway, what do you all think of the news?

Video is 8 minutes and 30 seconds long if you're interested.

Is it true that the DeepSeek models and that other AI model (I forgot the name already lol) can be run locally? How do they compete with Claude and OpenAI?

Also, what about that other AI thing, OpenClaw?

I am interested in AI but still very much skeptical.

I have only used Duck AI sometimes but try to avoid it.

you are viewing a single comment's thread
view the rest of the comments
[–] Makan@lemmygrad.ml 0 points 4 weeks ago (2 children)

How much VRAM or RAM do you need?

[–] PoY@lemmygrad.ml 0 points 4 weeks ago (1 children)

you need terabytes of vram if you want to run a full foundational model.. otherwise you're running a quantized version that introduces loss of information and accuracy.. some models can be quantized low enough to run on 8gbof vram, and many smaller models can run fine on 16, 24, or 32gb of VRAM

[–] salim@lemmygrad.ml 0 points 4 weeks ago* (last edited 4 weeks ago) (2 children)

Depends on the model, but maybe it's around 16 gigs of vram, and a good gpu, but you'd need a lot more resources to make it perform as good as the one in the web interface, i tried with my nvidia gtx 1660 ti 6 GB vram and the models that were available to me weren't really clever

[–] Makan@lemmygrad.ml 0 points 4 weeks ago (1 children)

So basically needs more power, huh...

[–] salim@lemmygrad.ml 0 points 4 weeks ago (2 children)

Based on the other comments a lot more, the whole ai crash isn't really about regular consumers but big companies that prefer to host their own ai instead of paying open ai and anthropic

[–] Makan@lemmygrad.ml 0 points 4 weeks ago

Honestly, the development of AI (not it happening but HOW it happened) has been very interesting and somewhat hopeful to me.

This is because the companies that control this technology don't have full control and basically are fighting each other for it.

Really fascinating ngl

[–] Faux@lemmygrad.ml 0 points 4 weeks ago

I'd say it's even more about medium sized and small companies (and also companies that aren't focused on IT primarily) being able to have an actual choice of a provider and providers being forced to compete for a price.

[–] Faux@lemmygrad.ml 0 points 4 weeks ago* (last edited 4 weeks ago) (1 children)

You need 128GB of VRAM for aggressive quantization of DeepSeek Flash. 16GB might be enough for aggressive quantization of smaller models. In theory you can run Qwen3.6 27B in 2-bit quantization (or something like https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf ) and have some space for context but it's a toy. You need at least 24GB of VRAM to use reasonable quant of Qwen3.6 27B (or 32GB of shared VRAM to handle Qwen3.6 35BA3B which is faster but takes more VRAM than 27B) in actual tasks.

[–] jmo@lemmygrad.ml 0 points 4 weeks ago

I've had great luck running qwen2.5-coder:7b on my work laptop. It's great for a wide range of dev and operational tasks with opencode.