Selfhosted

62,626 readers

6,042 active

373 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

Be civil.
No spam.
Posts are to be related to self-hosting.
Don't duplicate the full text of your blog or readme if you're providing a link.
Submission headline should match the article title.
No trolling.
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

moderators

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

Any external NPU that can be hooked to a proxmox miniPC? (lemmy.world)

submitted 3 weeks ago by iturnedintoanewt@lemmy.world to c/selfhosted@lemmy.world

27 comments fedilink hide all child comments

Hi! So I'm considering...maybe having an NPU or something similar to be hooked to my proxmox server, which runs in a mini PC. It's a EliteDesk 800 micro form factor. It has a Core i5 8500 CPU, which at the moment of purchase was good enough for live encoding HEVC video on Jellyfin...that was my main concern back then. But I'd like to consider the possibility of hooking maybe some docker instances or other containers to some local-only AI acceleration. Is there any NPU or cheap GPU I could hook on USB to this proxmox server to run? Has it been done before?

Thanks!

you are viewing a single comment's thread
view the rest of the comments

[–] hendrik@palaver.p3x.de 3 points 3 weeks ago* (last edited 3 weeks ago) (3 children)

Aren't diffusion models and LLMs (ollama) too big for an external NPU? As far as I know something like a Coral runs specific models only. And it's limited to the 1 or maybe 2GB of memory on it. It'd do tasks like voice recognition, or image classification. But not generate images or text.

If you want to run arbitrary AI models and generative AI, I think you should be looking for a graphics card?!

permalink

fedilink

source

parent

hideshow 3 child comments

[–] frongt@lemmy.zip 0 points 3 weeks ago (2 children)

Agreed. OP should probably upgrade to a bigger case and discrete graphics card.

permalink

fedilink

source

parent

hideshow 2 child comments

[–] iturnedintoanewt@lemmy.world [S] 2 points 3 weeks ago (1 child)

Thanks! I was seeing if there was a way to do it while avoiding a full blown desktop PC case :(

permalink

fedilink

source

parent

hideshow 1 child comment

[–] dan@upvote.au 2 points 3 weeks ago

For small options, there's mini PCs with the AMD Ryzen AI Max+ 395, and the Nvidia DGX. Their memory bandwidth is quite low compared to a GPU though, so expect slower performance.

permalink

fedilink

source

parent