Elevated API Errors
Incident Report for balena.io
Postmortem

Our database was overloaded by too many concurrent device registration requests. As a temporary measure, we upgraded database capacity and implemented a simple registration queue. To prevent future incidents, we will implement a more robust rate-limited queue to handle sudden spikes in registration.

Posted Jun 21, 2023 - 19:44 UTC

Resolved
This incident has been resolved.
Posted Jun 20, 2023 - 17:57 UTC
Update
Device registrations now succeeds. We're still monitoring the situation.
Posted Jun 20, 2023 - 15:24 UTC
Monitoring
We're still experiencing an elevated level of API errors and are currently looking into the issue.

Most requests are successful, but device registrations might still fails.
Posted Jun 20, 2023 - 14:22 UTC
This incident affected: API.