1249
"LLMs can do anything" except make positive cashflow. (europe.pub)
submitted 6 days ago by to c/funny@sh.itjust.works
135 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 5 days ago (2 children)

LLMs run on matrix math and probabilities

Yes, and the silicon to handle that math already exists. It's the "GPUs" being pumped out by nVidia. Those are basically now highly specialized matrix math machines.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 5 days ago (1 child)

    Then why do NPUs exist?

    What I'm talking about exists on the smaller scale. Just a matter of time before its done on a larger.

    A general purpose GPU still contains silicon for display, and video en/decoding. Those don't contribute in any meainiful way.

    Not to mention it doesn't cover all of matrix math for LLM needs....

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 4 days ago

    NPUs are low-power hardware devices for running very small AI models.

    So-called "GPU"s are the devices used for similar things for the very large models that are the "hotness" right now.

    A general purpose GPU still contains silicon for display, and video en/decoding

    No it doesn't. The modern datacenter-style "GPU"s that nVidia has been making and selling for use in AI datacenters have stripped out all the video related elements. They're purpose built for doing only AI work these days. They can't even do standard matrix multiplication work anymore because they're so narrowly designed for AI models where they use a specialized form of lossy matrix multiplication.

    The name has stuck around, but the things nVidia has been building and selling to the AI companies no longer has anything to do with graphics.

  • permalink
  • fedilink
  • source
  • parent