Monitoring - We have rolled out improvements to the restart process which lets us pre-reserve capacity before a restart/compute resize operation, with this improvement we are able to abort the restart process if there isn't sufficient capacity available, hence avoiding downtime.
This does not resolve the ongoing capacity issue present in eu-west-3b, but it will negate any type of restart or compute resize-related outage which might be encountered and inform users if this didn't succeed due to a capacity-related issue.
This additionally allows the platform to act more dynamically, allowing these operations to either execute successfully or fail gracefully, based on how AWS capacity fluctuates, and reducing the need for us to institute blanket guards at an availability zone or regional level.
All project operations have been restored in previously affected availability zones:
- ap-south-1a
- eu-central-1a
- eu-west-3c
- eu-west-2b
All project operations except database version upgrades have been restored in the following availability-zone:
- eu-west-3b
Feb 26, 2025 - 05:09 UTC
Update - We have rolled out improvements to the restart process which lets us pre-reserve capacity before a restart/compute resize operation, with this improvement we are able to abort the restart process if there isn't sufficient capacity available, hence avoiding downtime.
This does not resolve the ongoing capacity issue present in eu-west-3b, but it will negate any type of restart or compute resize-related outage which might be encountered and inform users if this didn't succeed due to a capacity-related issue.
This additionally allows the platform to act more dynamically, allowing these operations to either execute successfully or fail gracefully, based on how AWS capacity fluctuates, and reducing the need for us to institute blanket guards at an availability zone or regional level.
All project operations have been restored in previously affected availability zones:
- ap-south-1a
- eu-central-1a
- eu-west-3c
- eu-west-2b
- eu-west-3b
Feb 26, 2025 - 05:09 UTC
Update - Project restarts, as well as compute and database version upgrade operations have been disabled in the following availability zones:
- eu-west-3b
We are continuing to work with our cloud provider to in order to address compute capacity issues.
All project operations have been restored in previously affected availability zones:
- ap-south-1a
- eu-central-1a
- eu-west-3c
- eu-west-2b
Feb 10, 2025 - 09:20 UTC
Update - Project restarts, compute operations and database version upgrades have been disabled in eu-west-2b and eu-west-3b availability zones.
Project restarts, as well as compute and database version upgrade operations have been re-enabled in the previously impacted ap-south-1a, eu-central-1a, and eu-west-3c availability zones. These operations are currently disabled in the still-impacted eu-west-3b and eu-west-2b availability zones.
Feb 07, 2025 - 15:57 UTC
Update - We are continuing to work with our cloud provider to in order to address compute capacity issues in the eu-west-3b availability zone.
Project restarts, as well as compute and database version upgrade operations have been re-enabled in the previously impacted ap-south-1a, eu-central-1a, and eu-west-3c availability zones. These operations are currently disabled in the still-impacted eu-west-3b availability zone.
Project creation has been re-enabled in the eu-west-3 region.
Jan 17, 2025 - 22:47 UTC
Update - We are continuing to work on a fix for this issue.
Jan 10, 2025 - 17:19 UTC
Identified - We have identified an issue with our cloud provider regarding insufficient compute capacity in the following availability zones:
- ap-south-1a
- eu-central-1a
- eu-west-3b
- eu-west-3c
We have disabled the ability to restart projects, compute or database version upgrades within these availability zones.
Project creation has been disabled for the eu-west-3 region.
Jan 10, 2025 - 17:07 UTC
Investigating - We are currently investigating this issue.
Jan 10, 2025 - 17:00 UTC