TheMightyCat

joined 10 months ago
[–] TheMightyCat@ani.social 2 points 1 week ago

That user’s Linux is in bed with a US private corpo

Everyone's linux is in bed witu US corpos if you just look at the commit history of the kernel

[–] TheMightyCat@ani.social 8 points 2 weeks ago

The block user button is your friend

[–] TheMightyCat@ani.social 5 points 3 weeks ago* (last edited 3 weeks ago)

I'm running 2x4090, the 35B fits very comfortable in that.

For large models like the 397B without a ton of money there are several ways, ive seen posts of people using arrays of used 3090s with good results.

The other option is CPU inference although with current RAM prices that is less cost effective.

I was looking at maybe an array of Milk-V JUPITER2 since vllm added riscv support which could be very cost effective.

[–] TheMightyCat@ani.social 12 points 3 weeks ago* (last edited 3 weeks ago) (4 children)

Depending what OP was using before but going from something like GPT5.2 to LLama 3 8B will be a massive difference (Although OP says to use it only for basic tasks so that does offset it)

LLama 3 already being a very old model doesn't help either

I run Qwen3.5-35B-A3B-AWQ-4bit which while leagues ahead of LLama 3 8B still is a very noticeable difference.

This is not to say open source is bad, if one had the resources to run something like Qwen3.5-397B-A17B it would also be up there.

[–] TheMightyCat@ani.social 18 points 1 month ago* (last edited 1 month ago) (3 children)

Losing a vital plane now when you are at war then it doesn't really matter if it might get replaced sometime far in the future

[–] TheMightyCat@ani.social 3 points 1 month ago* (last edited 1 month ago)

You don't have to be a tankie or a wankie to recognize the fact that a loss ratio of 0% is very good for an aircraft.

If it was a Russian or Chinese aircraft doing this then that would also be a fact.

It's stupid to say that a single aircraft being damaged makes the entire type worthless like many here are saying when the F-35 is the only aircraft flying these missions and come back with only damage to a single plane.

That is not a political statement or approval of the strikes. That's just a statement on the plane.

[–] TheMightyCat@ani.social 8 points 1 month ago* (last edited 1 month ago) (4 children)

For the all the probably hundreds of sorties flown with a result of zero losses and only a single aircraft being damaged they might aswell be invisible.

[–] TheMightyCat@ani.social 27 points 2 months ago (5 children)

any tips?

My laptop has this option in the bios settings Not sure if it's only for laptops but you could check there.

[–] TheMightyCat@ani.social 18 points 2 months ago (4 children)

I'm running wayland with nvidia-open and nvidia-utils packages, and have never encountered any driver issues in both graphics and compute.

[–] TheMightyCat@ani.social 0 points 2 months ago

Let them cook

[–] TheMightyCat@ani.social 0 points 3 months ago

why the industry is still not ready to move away from it.

But why should the industry move away from it?

Am i the only person on earth who doesn't think C++ is a liability and actuallly works faster in it?

At work i have to develop a backend api in TypeScript and it severly lacks in features compared to C++, again maybe its just me but i don't find how one is expected to magically program faster in TypeScript then C++.

Not to mention the idiocy of compiling a static typed language to a dynamic typed language which is then interpreted.

[–] TheMightyCat@ani.social 4 points 3 months ago

I switched from github to forgejo and that was really easy, they have an migrate button.

 

cross-posted from: https://ani.social/post/16779655

GPU VRAM Price (€) Bandwidth (TB/s) TFLOP16 €/GB €/TB/s €/TFLOP16
NVIDIA H200 NVL 141GB 36284 4.89 1671 257 7423 21
NVIDIA RTX PRO 6000 Blackwell 96GB 8450 1.79 126.0 88 4720 67
NVIDIA RTX 5090 32GB 2299 1.79 104.8 71 1284 22
AMD RADEON 9070XT 16GB 665 0.6446 97.32 41 1031 7
AMD RADEON 9070 16GB 619 0.6446 72.25 38 960 8.5
AMD RADEON 9060XT 16GB 382 0.3223 51.28 23 1186 7.45

This post is part "hear me out" and part asking for advice.

Looking at the table above AI gpus are a pure scam, and it would make much more sense to (atleast looking at this) to use gaming gpus instead, either trough a frankenstein of pcie switches or high bandwith network.

so my question is if somebody has build a similar setup and what their experience has been. And what the expected overhead performance hit is and if it can be made up for by having just way more raw peformance for the same price.

 
GPU VRAM Price (€) Bandwidth (TB/s) TFLOP16 €/GB €/TB/s €/TFLOP16
NVIDIA H200 NVL 141GB 36284 4.89 1671 257 7423 21
NVIDIA RTX PRO 6000 Blackwell 96GB 8450 1.79 126.0 88 4720 67
NVIDIA RTX 5090 32GB 2299 1.79 104.8 71 1284 22
AMD RADEON 9070XT 16GB 665 0.6446 97.32 41 1031 7
AMD RADEON 9070 16GB 619 0.6446 72.25 38 960 8.5
AMD RADEON 9060XT 16GB 382 0.3223 51.28 23 1186 7.45

This post is part "hear me out" and part asking for advice.

Looking at the table above AI gpus are a pure scam, and it would make much more sense to (atleast looking at this) to use gaming gpus instead, either trough a frankenstein of pcie switches or high bandwith network.

so my question is if somebody has build a similar setup and what their experience has been. And what the expected overhead performance hit is and if it can be made up for by having just way more raw peformance for the same price.

view more: next ›