336
OpenAI Hoarding Tens Of Thousands Of Apple Mac mini And Mac Studio Devices, As ASUS And MSI Burn Through Their Entire First Batch Of NVIDIA RTX Spark Chip And Beg For More. (wccftech.com)
submitted 1 month ago by to c/technology@lemmy.world
79 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 13 points 1 month ago (14 children)

Has the computers for local AI become that good?

  • permalink
  • fedilink
  • source
  • hideshow 14 child comments
    •
  • [–] [S] 19 points 1 month ago (6 children)

    Macs, in particular, are likely to be the best option at the moment, unless you absolutely need the largest Frontier models.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 5 points 1 month ago (5 children)

    Just wait for Mac m5 ultra

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] [S] 13 points 1 month ago (4 children)

    The Mac Studio already has it. And from October, it’ll also be available with 512GB. It does cost around 18K, though.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 6 points 1 month ago (1 child)

    M5 ultra as indeed up to 512gb of unified memory. So you can run very large models. At the same time it's much much faster memory speed and bandwidth, almost simular to the nvidia rtx pro 6000.

    Sure the price is also high. But you get much better value for this setup compared to what nvidia is delivering you today. I still want to see even more competition.. Not just from Apple.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 1 month ago (6 children)

    No, no, they have not.

    I did the math the other day, and the cost of the mac versus the token rate that you get off of it for a decent model would take you over 70 years of 100% 24/7/365 usage to pay for itself versus just using hosted inference.

    They are wildly poor options for local AI models, at least on a cost basis. If you don't care about spending $10,000 to run a halfway okay model at 100x the cost, then go for it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 1 point 1 month ago* (1 child)

    Well local models are getting better. Open weight models are a real thread to their business. OpenAI's prices certainly wouldn't stay that low if they were the only player. Their prices are not profitable at the moment anyway.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] -1 points 1 month ago*

    They really aren't.

    Have you used open weight models for serious jobs? Ones that you can actually run effectively on a Mac studio? (Not deepseek v4 pro, not Kimi K3, not GLM 5.3)

    They work alright, at best, sometimes.

    And the ones you can't run effectively, locally, like Kimi K3 are considerably better at demanding tasks like software engineering. But even then they still suck at that job compared to frontier Anthropic and OpenAI models. I've been building with all of the above, and the open weight models just overall suck at serious high demand software workloads.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 1 month ago (2 children)

    Does that include energy usage (which I would think Apple being in the top 5 of being energy efficient) and just for AI? I’m sure most would be doing more than just hosting and running a single model.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 month ago* (1 child)

    You could include energy usage in the equation, but if you're buying a $10,000 Mac Studio to edit your PDFs and watch YouTube, then that's your own decision. That's also not what we're talking about here.

    The Mac is still going to be incredibly energy efficient.

    Let's break it down for Qwen 3.5 MoE.

    • M4 Ultra Mac Studio:
      • 22t/s
      • ~ 210Wh
      • 9.55 watts per token/s
    • B300 GPU:
      • ~ 2160t/s per GPU
      • ~ 1750Wh (14Kwh 8 GPU System)
      • 0.88 watts per tokens/s

    The Mac Studio comes out using ~ 11-12x more energy per token. Making it extremely energy inefficient at this task in comparison.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 month ago

    Not saying it wasn’t implied but didn’t offer it in my original comment but yeah, that’s what I’m talking about. It’s a computer, PC, desktop, workstation…… mostly like someone would be doing other things instead of just AI. You might be talking about just AI on the Mac Studio. To be fair, neither of us clarified our standings, till I ask further questions based on on my original comment. So you’re not wrong, just not what I was getting at. Thanks for sharing though.

  • permalink
  • fedilink
  • source
  • parent