Selfhosted

59976 readers

108 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

338

GPU prices are coming to earth just as RAM costs shoot into the stratosphere - Ars Technica (arstechnica.com)

submitted 6 months ago by otters_raft@lemmy.ca to c/selfhosted@lemmy.world

69 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] hoshikarakitaridia@lemmy.world 6 points 6 months ago (2 children)

AI or servers probably. I have 40gb and that's what I would need more ram for.

I'm still salty because I had the idea of going cpu & ram sticks for AI inference literally days before the big AI companies. And my stupid ass didn't buy them in time before the prices skyrocketed. Fuck me I guess.

[–] NotMyOldRedditName@lemmy.world 6 points 6 months ago* (last edited 6 months ago) (2 children)

It does work, but it's not really fast. I upgraded to 96gb ddr4 from 32gb a year or so ago, and being able to play with the bigger models was fun, but it's not something I could do anything productive with it was so slow.

[–] possiblylinux127@lemmy.zip 5 points 6 months ago (1 children)

Your bottle necked by memory bandwidth

You need ddr5 with lots of memory channels for it to he useful

[–] hoshikarakitaridia@lemmy.world 1 points 6 months ago

I always thought using ddr5 average speeds with like 64gb in sticks on consumer boards is passable. Not great, but passable.

[–] tal@lemmy.today 3 points 6 months ago* (last edited 6 months ago) (1 children)

You can have applications where wall clock tine time is not all that critical but large model size is valuable, or where a model is very sparse, so does little computation relative to the size of the model, but for the major applications, like today's generative AI chatbots, I think that that's correct.

[–] NotMyOldRedditName@lemmy.world 3 points 6 months ago* (last edited 6 months ago)

Ya, that's fair. If I was doing something I didn't care about time on, it did work. And we weren't talking hours, it it could be many minutes though.

[–] panda_abyss@lemmy.ca 2 points 6 months ago (1 children)

I’m often using 100gb of cram for ai.

Earlier this year I was going to buy a bunch of 1tb ram used servers and I wish I had.

[–] hoshikarakitaridia@lemmy.world 1 points 6 months ago (1 children)

Damn

Yeah used ram is probably where it's at. Maybe you get them used later on from data centers...

[–] kossa@feddit.org 2 points 6 months ago (1 children)

Yep, used ECC server RAM DDR3 or DDR4 is basically thrown out. Unfortunately most consumer mainboards do not support ECC.

[–] hoshikarakitaridia@lemmy.world 1 points 6 months ago

This is exactly the reason I'm about to order a dell poweredge r630 with Intel xeon 2680 v4 from alibaba.

Also I've never ordered from alibaba before so we'll see if I get scammed xd