cross-posted from: https://piefed.ca/post/152656
Funny thing is 'Local ML' tinkerers largely can't afford GPUs in the US either.
The 5090 is ludicrously expensive for its VRAM pool. So is the 4090, which is all but OOS. Nvidia will only sell you a decent-sized pool for $10K. Hence non-techbros here have either been building used RTX 3090 boxes (the last affordable compute GPU Nvidia ever sold), EPYC homelabs for CPU offloading, or have been trying to buy those modded 48GB 4090s back.
The insane supply chain is something like this:
-
Taiwan GPUs -> China
-
China GPU boards -> US
-
US GPU Boards -> Smuggled back into China
-
Deneutered GPU Boards -> Sold back to US
All because Nvidia is playing VRAM cartel and AMD, inexplicably, is uninterested in competing with it when they could sell 48GB 7900s basically for free.