24
What do you check first when a CI pipeline passes locally but fails in CI? (lemmy.world)
submitted 3 months ago by to c/programming@programming.dev
11 comments fedilink hide all child comments
 

Been dealing with this more often lately. Tests pass on my machine, I push, and CI blows up. Usually it's one of these:

  • Different Node/Python/whatever version
  • Missing env vars that exist in my .env but not in CI secrets
  • File system case sensitivity (macOS vs Linux)
  • Some flaky test that depends on timing

My current debugging flow is pretty basic: check the logs, compare versions, run the exact same Docker image locally if I can. But it still eats 20-30 minutes each time before I figure out the actual problem.

Anyone have a more systematic approach? Like a quick checklist you run through before you even look at the logs?

Also curious — do you replicate your CI environment locally with something like act (for GitHub Actions) or just trust the remote runner?

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 3 months ago* (last edited 3 months ago)

If you have a known good (local) and known bad (remote), the first thing I'd probably do is run diff on the local and remote logs. Use a regex or something to strip timestamps or similar that will always differ.

If your tests take 20–30 minutes to run, I think that I'd look into having a way to run a subset of the tests, so you can just run the minimal amount to do the failing one.

If you have a lot of environment problems, I think I'd focus on getting an automated build of the environment to a standard state. If that takes too long, setting up caching or whatever of downloaded packages.

  • permalink
  • fedilink
  • source
    •