I’d be interested in buying one as a headless server, but a lot of the FLOSS software I’d like to run seem to prefer x86 processors.

What exactly do you think a cloud is? It's built exactly like that, computers connected and orchestrated to run a client's arbitrary services.

Now the interesting thing here is to choose many different types of smallscale hardware. Must be some fun to administrate.

Ah, I see you’ve stumbled across the real reason for the “AI boom”.

Hint: it’s not because the models are SO incredible. It’s the fascists taking computers away from us so we can’t stifle them. So we can’t talk shit on billionaires anymore. So we can’t fundraise for a better world.

Just like our cars and our traffic lights watch our every move. Just like our phones have 100 sensors that profile us from the moment we wake up.

Demons took the wheel a long time ago.

I overbuilt built my NAS/container server for $1262 excluding the price of the rack chassis, PSU, cooler and two 1TB NVMe SSDs I cannot recall the price I paid for. B550 board, Ryzen 5, four 10TB Red Plus HDDs, Intel Arc A380, 64GB of DDR 5 ECC. Those listed components are now $3381. When or if you can even find them. The Arc is unobtanium as far as I can see. Of course the RAM and HDDs went up the most in cost, but it's still like $65 and $75 more for the mobo and chip.

Whoop, decided to check and I found the SSDs: I paid $60 a pop for Samsung 980 Pros and they're $240 now. So $1382 then vs $3861 now. Almost triple. The bubble can't pop soon enough.

That same thought just occurred to me: hype a McGuffin and get investors to lend you insane amounts of money which you then use to buy up a marker, causing artificial scarcity, then sell at massive profits.

And that's really why they're hoarding them.

Yeah no way they will allow any of this hardware to go back onto the market. Anything they dont use anymore will be destroyed.

https://github.com/AtomicBot-ai/atomic-llama-cpp-turboquant

I'm running gwen 3.6 with 131k context window on a 3090, it's fast enough and about as good as pay to play Claude at work.

Any 27b Model you can currently recommend for a 16gb AMD ? Mostly coding tasks but not exclusively.

Rosetta 2 is irrelevant here because they'd just compile their shit for ARM for better performance.

At least unlike B300s, these can one day be used if OpenAI dumps them on the used market. IF they do that instead of trashing them to make sure everything stays expensive.

Apple‘s little gambit to be immune to AI price fixing savagery

I don't think it was ever a strategy. If it was they would've stopped mac's from ever reaching OpenAI's datacenters. Or atleast they'd have prevented them from hoarding so much of the M5 production.

Rosetta 2 has literally nothing to do with this.

Macs, in particular, are likely to be the best option at the moment, unless you absolutely need the largest Frontier models.