I'm interested, primarily in the idea that some parts could be useful longer term... I'm an old school Internet janitor, and appreciate the idea that enshittification is a moving target.
I'd ask up front though if you've seen or considered Veilid? It might be wise to evaluate working with that as a baseline so a stronger fight against the shit can be had.
Let's fuck up some fascists.
Ugh that is extra shitty. Yeah eBay is absurd sometimes with the risks.
For anyone skimming, my cards are all based around the ancient but great LSI 9211-8i chips.
I flash my own, so I can disable BIOS and efi. I suppose if someone gets to the larger hoarding, they should be comfortable flashing their own cards too.
..... Glory you say?
I personally have dedicated machines per task.
8x SSD machine: runs services for Arr stack, temporary download and work destination.
4-5x misc 16x Bay boxes: raw storage boxes. NFS shared. ZFS underlying drive config. Changes on a whim for what's on them, but usually it's 1x for movies, 2x for TV, etc. Categories can be spread to multiple places.
2-3x 8x bay boxes: critical storage. Different drive geometric config, higher resilience. Hypervisors. I run a mix of Xen and proxmox depending on need.
All get 10gb interconnect, with critical stuff (nothing Arr for sure) like personal vids and photos pushed to small encrypted storage like BackBlaze.
The NFS shared stores, once you get everything mapped, allow some smooth automation to migrate things pretty smoothly around to allow maintenance and such.
Mostly it's all 10 year old or older gear. Fiber 10gb cards can be had off eBay for a few bucks, just watch out for compatibility and the cost for the transceivers.
8 port SAS controllers can be gotten same way new off eBay from a few vendors, just explicitly look for "IT mode" so you don't get a raid controller by accident.
SuperMicro makes quality gear for this... Used can be affordable and I've had excellent luck. Most have a great ipmi controller for simple diagnostic needs too. Some of the best SAS drive planes are made by them.
Check BackBlaze disk stats from their blog for drive suggestions!
Heat becomes a huge factor, and the drives are particularly sensitive to it... Running hot shortens lifespan. Plan accordingly.
It's going to be noisy.
Filter your air in the room.
The rsync command is a good friend in a pinch for data evacuation.
Your servers are cattle, not pets... If one is ill, sometimes it's best to put it down (wipe and reload). If you suspect hardware, get it out of the mix quick, test and or replace before risking your data again.
You are always closer to dataloss than you realize. Be paranoid.
Don't trust SMART. Learn how to read the full report. Pending-Sectors above 0 is always failure... Remove that disk!
Keep 2 thumb drives with your installer handy.
Keep a repo somewhere with your basics of network configs... Ideally sorted by machine.
Leave yourself a back door network... Most machines will have a 1gb port. Might be handy when you least expect. Setting up LAGG with those 1gb ports as fallback for the higher speed fiber can save headaches later too...
OP, check if the default format of your pictures is HEIF (Samsung I know does this). It can be found under the camera app settings typically.
Mastodon and some other servers choke on this format frequently unless all the backend code is updated... Not sure about the Lemmy servers yet. It drove me nuts for days until a log item in my mastodon server I run brought my attention to it.
Unfortunately if you confirm this, you can't fix it client side except to convert your pictures first or disable that format for saving.
Yessss.