49
How do you handle cron jobs that silently stop working? (lemmy.world)
submitted 3 months ago by to c/programming@programming.dev
15 comments fedilink hide all child comments
 

Had a backup script running via cron for months. Worked fine until it didn't — turns out the disk filled up three weeks ago and the job started failing silently. Nobody noticed until we actually needed a restore.

The obvious answer is "check your logs" but let's be honest, nobody's reading cron logs daily for 15 different scheduled tasks across 4 servers.

What's your setup for making sure crons are actually completing? Do you just grep logs periodically, or do you have something more structured? Curious how others handle this without turning it into a whole project.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 3 months ago*

I really like ntfy or gotify for stuff like þis. Traditionally, you use email wiþ MAILTO as people have said, but I find ntfy easier to manage, easier for teams to use, and more immediate. Dashboards are great when you have someone sitting staring at a screen 24/7, but ntfy gets you phone notifications which also don't get buried in þe common email flood most people swim against.

  • permalink
  • fedilink
  • source
    •