23
Any external NPU that can be hooked to a proxmox miniPC? (lemmy.world)
submitted 3 weeks ago by to c/selfhosted@lemmy.world
27 comments fedilink hide all child comments
 

Hi! So I'm considering...maybe having an NPU or something similar to be hooked to my proxmox server, which runs in a mini PC. It's a EliteDesk 800 micro form factor. It has a Core i5 8500 CPU, which at the moment of purchase was good enough for live encoding HEVC video on Jellyfin...that was my main concern back then. But I'd like to consider the possibility of hooking maybe some docker instances or other containers to some local-only AI acceleration. Is there any NPU or cheap GPU I could hook on USB to this proxmox server to run? Has it been done before?

Thanks!

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 3 weeks ago* (5 children)

You should probably have a look at the software you intend to run and see what kind of external NPU they support. I'm aware of Frigate for example which leverages hardware acceleration to classify camera surveillance footage. Guess their community knows which stick to get. Or you'd have a look at the frameworks which package machine learning models for inference on some NPU and have a look at their table of supported hardware.

  • permalink
  • fedilink
  • source
  • hideshow 5 child comments
    •
  • [–] 1 point 3 weeks ago (4 children)

    Inference from GPU, even QuickSync, is better now than what TensorFlow was able to do on a Google Coral TPU.

    Plus, the Coral drivers are untouched for the last 4 years.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 3 weeks ago* (3 children)

    I think you mean SyCL or Vulkan, OpenCL or oneDNN... QuickSync is just video de- and encoding.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 3 weeks ago (2 children)

    QuickSync is the GPU hardware used to infer from vino models.

    https://docs.frigate.video/configuration/hardware_acceleration_video/#via-quicksync

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 weeks ago* (1 child)

    Thx! That's the video decoding page, though. The thing that decodes the video stream and makes individual images from the stream, for the actual processing. Seems the thing that does inference / object detection is called OpenVINO.

    https://docs.frigate.video/configuration/hardware_acceleration_enrichments

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •