this post was submitted on 21 Jul 2026
1 points (100.0% liked)
Technology
1467 readers
10 users here now
A tech news sub for communists
founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
you need terabytes of vram if you want to run a full foundational model.. otherwise you're running a quantized version that introduces loss of information and accuracy.. some models can be quantized low enough to run on 8gbof vram, and many smaller models can run fine on 16, 24, or 32gb of VRAM
noted