308
Sam Altman says OpenAI will not go public this year as AI fears grow (www.sfchronicle.com)
submitted 2 weeks ago by to c/technology@lemmy.world
82 comments fedilink hide all child comments
 

Archive link: https://archive.ph/U1zEK

you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 2 weeks ago (1 child)

Thats is pretty decent alright, I had heard 24 was what was needed for decent local AI.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 2 weeks ago

    24 is ideal, 16 works fine especially with some MoE offloading. If you use llama-swap, a fast PCIe bus and fast storage is ideal. Takes about 20 seconds to swap between two models on my X570 board.

  • permalink
  • fedilink
  • source
  • parent