No components marked as affected
Write-up published
Resolved
Between 09:26 UTC and 09:55 UTC on November 24th 2025, customer projects returned HTTP 556 and 500 errors for 90% of requests to multiple products.
Timeline
Time
Description
09:26 UTC
API Gateway deployment. Customer impact starts.
09:30 UTC \(\+04 mins\)
Multiple product teams report monitoring anomalies and an influx of support tickets indicating problems with the Management API.
09:38 UTC \(\+12 mins\)
Incident is declared internally.
09:40 UTC \(\+14 mins\)
Status page post is created.
09:43 UTC \(\+17 mins\)
A recent deploy was rolled back as a precautionary step.
09:43 UTC \(\+17 mins\)
The team continued investigating various causes to the issue. We saw varied reports of Management API downtime, as well as errors manifesting in the Product services.
09:52 UTC \(\+26 mins\)
Root cause identified as a recent API Gateway release.
09:55 UTC \(\+29 mins\)
Rollback of API Gateway release. Customer impact ends.
Who was affected?
90% of requests via our API Gateway resulted in errors, which includes requests for our Data API, Auth, Storage, Realtime and Functions products - affecting customer projects in all regions.
Direct Database connections and Supavisor connections were not impacted.
Additionally our Management API was affected, and customers may have experienced errors when taking actions to manage their Supabase organization or projects.
Service
Impact
Data API
Impacted
Auth
Impacted
Storage
Impacted
Realtime
Impacted
Functions
Impacted
Management API
Partially Impacted
Direct Database connections
Not impacted
Supavisor connections
Not impacted
What happened?
At 09:26 UTC we deployed a release to our API Gateway service, which uses feature flags: configuration settings that allow us to enable or disable features without redeploying code.
In this deployment there was a missing feature flag: the value was undefined in production and the code did not gracefully handle undefined feature flag values.
The release had been tested extensively in our staging environment since November 20th 2025 with no issues, however the testing in staging did not take into account how an undefined feature flag value in production could have affected the release. Due to this testing overconfidence, we enabled an immediate global release, rather than a gradual rollout. The immediate global rollout caused the issue to have larger impact that it otherwise should have had.
The team rolled-back a recent deploy relating to the Management API as precautionary step while they continued to investigate. This did not solve the issue. Once the offending commit was identified the API Gateway release was rolled back, resolving the issue.
Fixing the root cause
To guarantee that our code properly supports undefined feature flags, we now apply TypeScript's Partial<> utility type to the FeatureFlags interface. This ensures that every feature flag property is automatically marked as optional, eliminating the risk of runtime errors from missing values:
// Make all feature flags optional: enforced by type system
type FeatureFlagPartial = Partial<FeatureFlags>
// Internal type that includes feature flags: only used by getFeatureFlags
type EnvironmentInternal = Config & FeatureFlagPartial
What will we do to mitigate problems like this in the future?
Feature Flag Resilience - we have improved our API Gateway to ensure that we gracefully support undefined feature flag values. We are making the same changes to all systems across the platform.
Improved Deployment Strategy - we are enforcing changes to our Release Management process, so that global rollouts are not possible except in the explicit case case of an override due to a hotfix or security concern. All services will follow a gradual staged rollout.
Enhanced Observability - we are integrating additional API Gateway metrics and alerts into our monitoring stack. This work has been prioritized to ensure similar incidents are detected immediately. This will bring the API Gateway in-line with our standard for service monitoring.
Fixing second order-effects - during the incident, there were reports of BigQuery errors. This was a downstream effect: as more customers were logging in to check their logs and observability, we hit rate limits on our logging service. We are already in the process of migrating to a new logging backend that will prevent these errors in the future.
To our customers
To our customers: I'm sorry if this affected you. Supabase is the core infrastructure for millions of developers. This outage was avoidable, and the relaxed processes were unacceptable.
Our focus as a company is to become the most reliable service available. This outage is our responsibility and we’ll use it as an important lesson for several areas of improvement.
On behalf of the Supabase team - we’re sorry, and we’ll do better.
Paul, CEO & cofounder
Resolved
This incident has been resolved.
Monitoring
We rolled back a deployment to our API Gateway and we can see requests recovering. We will continue to monitor.
Identified
We are continuing to work on a fix for this issue.
Identified
We have identified issues with failing requests across a range of services and customer projects. Our Engineering team is working to resolve this now.
Investigating
We are currently investigating this issue.