Peripherals can be finicky on Linux since drivers like to fight for priority.
Slightly related: if you’re using Wayland you may also want to make rule exceptions for controllers/remote input. That pop up box is annoying to track down and re-confirm every 15 minutes when you’re not in the same room.
Once it’s sorted it’s smooth sailing though and no surprise 30 minute restarts a la Windows :)