Does anyone run their own AI service?

If so,

Which one?

What hardware?

Do you have a way to access it from outside or from a phone?

And, if you're open to sharing, what do you do with it?

I know AI-enabling hardware is insanely expensive right now, but some of the smaller (<32GB) more efficient models (think deepseek- or qwen-based) don't absolutely need specialized hardware to run and perform well enough for many purposes.

I am so aware of the absolutely abysmal privacy practices of basically all AI companies that I'm not comfortable using them to help me with web searches and the such, because there's no way that all this information won't be going into a profile. This makes me want to have my own "helper" service.

you are viewing a single comment's thread
view the rest of the comments
[–] 17 points 6 days ago (13 children)

Yes, were all on c/LocalLLaMa

I'm running qwen3.8-next-flash on a rx7900xtx plus 128gb of ram. It serves my hermes agent. I direct it through matrix. I also forward an endpoint to my VPS to use for anything. Mostly opencode.

  • source
  • hideshow 13 child comments
  • [–] 2 points 6 days ago (12 children)

    Is that the only community? I see the last activity was 2 months ago.

  • source
  • parent
  • hideshow 12 child comments
  • [–] 8 points 6 days ago (2 children)

    I see the last activity was 2 months ago.

    that means no one from your instance is subscribed, if you subscribe then you'll get new posts

    you can see that there are new posts here https://sh.itjust.works/c/localllama

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 6 days ago (8 children)

    Posts from 16 minutes and 2 days ago here. Maybe your instance is not federating with theirs.

  • source
  • parent
  • hideshow 8 child comments
  • [–] 3 points 6 days ago* (6 children)

    oh yea feddit blocks shitjustworks https://feddit.online/instance/sh.itjust.works/posts

    maybe @xSikes@feddit.online could ask their admins to convert the defederation to a "silenced" or whatever that feature is on PieFed

  • source
  • parent
  • hideshow 6 child comments
  • [–] 1 point 6 days ago (4 children)

    Thank you for confirming. I didn’t know I was even missing out on posts. Your follow up, I don’t even understand. Maybe I should go back to Lemmy (I think my account is on sh.itjust.works) and do my own filtering.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 6 days ago (3 children)

    PieFed allows admins different levels of blocking instances, they can do defederation but there are levels softer than that

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 6 days ago (2 children)

    Are you saying that the admins of feddit.online can do that only to my account or is this a instance as a whole?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 6 days ago

    https://piefed.social/comment/13343280

    PieFed has the concept of silencing another instance. A silenced instance is not defederated from but their posts do not show in the Popular or All feeds and their communities are not shown during onboarding. Their communities can still be found in the communities list (if you unblock the instance first) and joined in the normal way. Once joined, posts in there show up in the Subscribed feed of the joiner as usual.

    Since I added silencing functionality I feel like there’s almost no need for (more) defederation. I can offer protection to newbies who don’t yet know which places to avoid and they can explore the dark corners when they’re ready. It’s a low-drama way to put a bit of distance between groups that rub each other the wrong way.

  • source
  • parent
  • [–] 1 point 6 days ago

    I think that’s it. Even though I’ve been on Lemmy then PieFed for over a year. I still end up not understanding any of this but I think this is one of my core issues here. I would have never know.

  • source
  • parent