Elevated Device URLs/VPN Errors
Incident Report for Balena.io
Postmortem

Starting around 09:00 UTC, one of the nodes where a VPN service pod was running encountered memory pressure causing VPN connections to drop. The amount of reconnecting devices started overloading the other VPN service pods. This made the VPN connections to intermittently drop as the connections were being redistributed to recovering VPN service pods. The devices were still able to communicate with the backend as soon as a VPN tunnel was established.

We increased the cluster capacity and added additional VPN service pods to handle surges of new VPN connections.

Posted Jun 17, 2021 - 17:20 UTC

Resolved
This incident has been resolved.
Posted Jun 17, 2021 - 14:51 UTC
Investigating
We're experiencing an elevated level of errors in our VPN infrastructure and are currently looking into the issue.
Posted Jun 17, 2021 - 14:13 UTC
This incident affected: Cloudlink (VPN).