Elevated GIT/Application Builder Errors

Incident Report for balena.io

Postmortem

An error was introduced in our builder firewall rules that dropped all outbound traffic from the build context. During this time users would see builds start but fail when a RUN instruction included a command that required internet access.

We rolled the builders back to a previous release while we investigated the root cause of the bug.

A fix is now being reviewed and tested to catch this edge case that takes time to manifest and was missed in original testing.

Posted Oct 30, 2024 - 17:14 UTC

Resolved

This incident has been resolved.
Posted Oct 30, 2024 - 12:01 UTC

Update

We are continuing to monitor for any further issues.
Posted Oct 30, 2024 - 11:18 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Oct 30, 2024 - 11:18 UTC

Investigating

We're experiencing an elevated level of errors in our application builder infrastructure and are currently looking into the issue. Some users may experience DNS resolution errors during builds using the balenaCloud builders.
Posted Oct 30, 2024 - 10:42 UTC
This incident affected: Application Builder.