"Mindfactory's sales information in repeated articles over several years, and its Ryzen 7 5800X, 5600X, and 5900X RMA rates are 0.58 percent, 0.52 percent, and 0.33 percent, respectively."
Based on some very rough estimates using mindfactory and amazon sales data, the 9800x3d is in line with or below 5000 series RMA rates.
If there was real cause for concern, the RMA rate should be much higher.
None of your listed use cases will even come close to taxing the 6600k. It's going to probably sit happily in idle states most of the time.
Proxmox also has great snapshotting and backup features. Makes it easier to mess around with your containers/VMs without worrying too much.
Most people go with either FreshRSS or Miniflux but there's lots of options to choose from.
Any low power device is going to struggle with video playback unless you're doing direct streaming. The exception being the new Intel N line of processors that have capable enough GPUs to do the transcoding. If all you're after is decent transcoding performance for H264/H265 video, anything newer than Intel's 7-8th gen CPUs will do the job. $100-$200 used optiplexes and thinkcentres can easily handle this type of workload.
For reference: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video#Hardware_decoding_and_encoding
Can you be more specific about your performance objectives? The Pi5 can handle jellyfin and nextcloud relatively well, so if you're having issues with them, identifying the bottlenecks can help narrow down the hardware recommendations.
When I bought my B650 Lightning, Wendell from Level1Techs recommendations over the years played a big part. I haven't found a single thing to complain about.