Railway's Outage Was a Breach: Rotate Every Secret You Stored There

As of October 22, 2026, 20:29 UTC.

If you stored a secret in a Railway environment variable, treat it as exposed and rotate it. That is Railway's own advice to every customer, not only the ones it thinks were hit.

The outage and the breach were one event. Railway paused builds while it investigated, so for most of a day customers could revoke a leaked key but could not deploy the new one.

What happened

All times are UTC, from Railway's status page unless noted.

• October 21, 07:24. Railway starts investigating a "Railway Platform Outage."
• 09:28. The issue is identified. The dashboard, API and CLI are unavailable, and new builds are paused in all regions. The support site is down too.
• 20:30. Railway invalidates OAuth sessions "due to impact from adjacent cache systems," so users are signed out.
• 20:44. Railway publishes a security disclosure. The status page links it at 21:59.
• 22:10. Railway starts rotating every storage bucket credential on the platform.
• October 22, 01:22. Railway posts a set of recovery guides, one per credential type.
• 03:32. Builds return for Enterprise workspaces.
• 04:17. Railway invalidates all Railway API tokens.
• 06:29. Builds return for Pro workspaces. A deployment backlog clears by 13:53.
• 17:00. The API is fully restored.
• 19:06. Railway pauses new signups "so we can focus on making sure everything is working well for existing customers."
• 20:29. Hobby builds are still being enabled. Free and Trial come after.

The status page lists builds in all four regions (US West, US East, EU West and Southeast Asia), the API at backboard.railway.com and the support site as affected.

What Railway says was exposed

From the disclosure: Railway "detected anomalous activity involving systems that store customer environment variables," and an unauthorized third party "may have accessed encrypted environment variables and, for some of them, obtained the decrypted contents."

Railway says those variables can hold API keys and database and registry credentials. It believes the intrusion is "part of a series of attacks recently targeting platform companies with the aim of compromising AI inference providers," and says it told those providers to revoke keys. Its initial analysis "suggests the impact may be limited to a subset of users," and it will email affected customers directly.

Railway has not published a root cause; it says a full incident report will follow once the incident is resolved. It has not said how the attacker got in, how many customers were affected, or when the access began.

Outside reporting adds one claim Railway has not made. The Hacker News reports, citing security researcher Taylor Monahan, that the attackers drained cryptocurrency wallets whose private keys were stored on Railway. Railway's own posts do not mention wallets.

What Railway reset for you, and what it did not

Railway rotated some credentials itself. Per its bucket and GitHub guide, it regenerated the S3 credentials for every bucket in every environment, invalidated the GitHub tokens it held for account connections, and removed GitHub tokens stored as registry credentials. The status page adds that all Railway API tokens and Project Tokens created before the incident are dead.

Three of those still need you:

• Services read variables only at startup. A service keeps using the old bucket key until you redeploy it.
• Removed is not revoked. The ghcr.io tokens Railway deleted "still work on GitHub until you revoke them," per the same guide.
• Everything else is yours. Railway cannot rotate your OpenAI, Anthropic, Stripe, AWS or database credentials for you. It has posted guides for PostgreSQL, MySQL, MongoDB, Redis, Stripe, GitHub, AWS, GCP, OpenAI, Anthropic, Grok and OpenRouter, all linked from the status page.

What to do now

1. Rotate every secret stored in Railway. Revoke at the issuer first (OpenAI, Stripe, AWS, your registry), then set the new value in Railway and redeploy. Prioritize keys that spend money or reach production data.
2. Regenerate database passwords the supported way. Railway's PostgreSQL guide says to use Regenerate in the Database view, not ALTER USER, because hand edits leave the variable and the database out of sync. Turn off public access on databases that do not need it.
3. Redeploy every service that reads a rotated value, including bucket consumers. Replace pasted values with reference variables so the next rotation reaches them on its own.
4. Revoke the old GitHub tokens on GitHub, then reconnect. Create fresh Railway API tokens for CI and scripts.
5. Check for misuse in the exposure window. Usage and billing on AI providers grouped by key, Stripe and cloud audit logs, unknown database sessions and roles, objects you did not upload. Railway's audit log marks its own resets as Railway Staff, so any other change in that window needs an owner.
6. Watch your inbox. Railway says it will email the customers it confirms were affected.

What managed-platform teams should take from this

Railway's customers learned in one day that their host held three things at once: the running app, the secrets, and the only path to deploy a fix. When builds paused, the secrets were already exposed and the path to replace them was closed.

Know your single points of failure. List what stops if the provider's API, dashboard or build pipeline goes dark: deploys, secret changes, rollbacks, scaling, support. Railway's support site was down for most of the first day.

Keep secrets reproducible outside the platform. The source of truth for each secret should be the issuer and a secrets manager you control, with the platform holding a copy. That way you can say which keys a provider held and rotate them from a list, without reconstructing it from a dashboard you cannot reach. Do not solve this by copying secrets into the repo.

Keep deploy artifacts portable. A container image in a registry you control, plus infrastructure config in git, is what lets you stand the app up somewhere else when the build pipeline is the thing that broke.

Subscribe to the status page. Railway posted updates through the night and called the status page "the fastest place for updates." Route it to the channel your on-call reads.

Write the failover runbook before you need it. Decide in advance what read-only mode looks like, who can flip DNS, and which secrets a second provider needs on day one.

Restore a backup at least once. Railway's bucket guide notes that buckets have no automatic backups. A backup you have never restored is a guess.

Rotating in a hurry is how the next leak starts

A breach like this sends many teams to mint new keys at once, often onto a laptop, often into a .env file, sometimes into a new provider stood up that afternoon. That is the moment a fresh key lands in a commit. We have written about how exposed .env files get found and what to do in the first hour after a leak; the Railway response follows the same steps at platform scale, much like the Vercel breach in April.

The open-source Rafter CLI helps with that part and nothing upstream of it. rafter secrets . scans a directory for secrets locally, with no account or API key, and rafter agent install-hook adds a pre-commit hook that blocks commits with secrets in staged files. It cannot tell you whether Railway leaked your keys. It can keep the replacement keys out of git while you rotate.