Elevated GIT/Application Builder Errors

Incident Report for balena.io

Postmortem

Starting around March 11, some cloud builds began failing intermittently with no such image errors. The failures were non-deterministic and affected all architectures. At peak, some users saw around 50% failure rates.

We identified and fixed several bugs in the builder's image garbage collector that caused it to over-count freed disk space and run too aggressively, eventually deleting images that in-progress builds still needed. Fixes were deployed between March 19 and April 14, with build failure rates dropping to near-zero after the final deploy.

We're continuing to monitor and working on additional safeguards to prevent the garbage collector from targeting images that active builds depend on.

Posted Apr 21, 2026 - 17:06 UTC

Resolved

This incident has been resolved.
Posted Apr 21, 2026 - 16:30 UTC

Update

We are continuing to monitor for any further issues.
Posted Apr 08, 2026 - 19:51 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Apr 08, 2026 - 19:51 UTC

Identified

We're experiencing an elevated level of errors in our application builder infrastructure and are currently looking into the issue.
Posted Mar 31, 2026 - 12:55 UTC
This incident affected: Application Builder.