The Outage Nobody Saw Because Nothing Crashed

Coin Droppa is a Polygon airdrop arcade — real contracts, real MATIC, a signed jackpot roll dressed up as a retro cabinet floor. In May, the security pass landed: EIP-712 typed-data signing to replace a plain personal_sign, a CSPRNG jackpot roll to replace Math.random(), per-claim nonces with a five-minute expiry, generic error responses to kill the side-channel that told scrapers exactly why a request failed. A SECURITY.md documented all of it as a public threat model, attacker taxonomy included. Canary tripwires went in too — decoy cabinet addresses that don't exist on-chain, honeypot paths that look like a misconfigured deploy, a ledger endpoint borrowed straight from the Thinkst Canarytokens playbook: silent to the attacker, loud to the operator.

None of it ever shipped.

The site that looked fine

The production site had been serving a build from May 7 since May 7. Every deploy attempted after that — four of them, spread across four months — failed. Nobody noticed, because a failed deploy doesn't take a site down. It just leaves the last successful one running. The homepage loaded. The arcade animated. The wallet button worked. There was no error page, no downtime alert, nothing that would make a person check.

The tell, when I went looking, wasn't visible from the site at all. It was in the response headers. Every path — the homepage, a canary status page, a completely made-up URL I typed to see what would happen — came back HTTP 200 with the identical etag and the identical content-length. A real 404 has a different shape than that. A working set of serverless functions has a different shape than that. What I was looking at was a single static file, index.html, catching every route through an SPA fallback rule, including the ones that were supposed to hit real backend functions. The signing endpoint that issues jackpot signatures — the one actual piece of infrastructure a live wallet claim depends on — was one of the routes getting swallowed.

Reading the actual failure

The build log named it precisely, once I pulled it:

Scanning for secrets in code and build output.
Scanning complete. 51 file(s) scanned. Secrets scanning found 1 instance(s)
of secrets in build output or repo code.

Secret env var "BADGE_CONTRACT_ADDRESS"'s value detected:
  found value at line 13 in OPERATIONS_GUIDE.md
  found value at line 125 in README.md
  found value at line 6 in app.js
  found value at line 40 in netlify/functions/orchestrator.js

BADGE_CONTRACT_ADDRESS is a deployed contract's address on Polygon mainnet. It's meant to be public — it's printed in the README's own deployment-addresses section, on purpose, so anyone can verify the contract independently. Netlify's secret scanner had it registered as a sensitive environment variable, and its scanner does something specific: it checks whether the value of every configured env var shows up anywhere in the files about to ship. A contract address is, structurally, indistinguishable from a leaked API key to that kind of check — it's just a string that matches. The scanner did exactly what it was built to do. It was checking the wrong thing.

An AI-generated build-failure summary I was also handed made this worse before it made it clearer. It read the same log, saw a "Resolved config" block listing every environment variable configured for the build — ALLOWED_ORIGINS, CANARY_HMAC_KEY, RPC_URL, SIGNER_PRIVATE_KEY, the works — and reported all of them as flagged secrets. Only one line in the actual log said found value at line X. The rest of that block is just Netlify echoing its own configuration back for debugging, not a list of leaks. Trusting the summary instead of the source would have meant chasing five phantom secrets and never finding the real one.

The fix, and the part that matters more

[build.environment]
  SECRETS_SCAN_OMIT_KEYS = "BADGE_CONTRACT_ADDRESS"

One line, scoped to exactly one variable. Every other configured secret — the platform signer key, the canary HMAC key, the RPC URL — stays fully scanned. The instinct to reach for SECRETS_SCAN_ENABLED = "false" and make the whole problem disappear was right there, and it's the wrong fix: it would have solved this deploy by turning off the thing that would catch a real leak next time.

Once that shipped, the rest came back in one deploy. The signing endpoint started responding to malformed requests with the generic REQUEST_DECLINED the code was always supposed to return, instead of silently vanishing into a fallback page. The canary ledger started serving real tripwire events from Netlify Blobs instead of a 404 dressed up as a 200. Four months of hardening went live in the time it took to push one commit.

What actually failed here

Not the contracts. Not the signing logic. Not even, really, the secret scanner — it was configured correctly for every variable except one, and flagging a value that matches a configured secret is the entire point of the tool. What failed was the feedback loop: a deploy can fail silently, repeatedly, for months, while the thing sitting in front of users looks completely fine. The gap between "the site is up" and "the site is running what I think it's running" is exactly where this kind of drift lives, and it doesn't announce itself. You have to go looking, the same way you'd audit anything else you didn't personally watch ship.

---

GhostInThePrompt.com // A build can fail silently for four months and the site never once looks down. Uptime was never the metric that mattered.