# Failure modes that actually happened

Each of these cost real time or real blindness. They are listed because the
symptom pointed somewhere other than the cause every single time.

## A bot running in two places

**Symptom:** duplicate alerts, and config edits that appear to do nothing.

`crash-bot` existed on both the Mac and the droplet with the same Telegram
token. The Mac copy was updated to 9 wallets; the droplet copy kept running
the old 3-wallet config. Both posted as `@crash_accum_bot`. Three wallets
double-alerted, six new ones were never watched, and the Mac edit looked
correct on inspection.

**Rule:** confirm the host before editing. Fleet home is the droplet.

## A launchd agent that was never installed

**Symptom:** "I restarted it and nothing happened."

`launchctl kickstart -k gui/$(id -u)/com.benkorsa.crashbot` fails silently to
a reader who is not looking, because no plist existed in
`~/Library/LaunchAgents/`. The bot's last life had been a manual foreground
run that died with the terminal 44 minutes later.

**Rule:** `launchctl list | grep <label>` before trusting any restart
command. Absent from the list means the service does not exist, not that it
is stopped. Moot now that nothing runs on the Mac.

## Silence read as death

**Symptom:** a healthy bot restarted 51 times in one night.

`xalert` holds a websocket and logs nothing for hours when its accounts are
quiet. The watchdog judged it by log age and killed it repeatedly, each
restart costing ~95s of genuine blindness while twitterapi.io held the slot.

**Rule:** liveness needs a per-service probe. Log chatter is not liveness.
The watchdog now uses `max_silence 0` for poll-loop bots and a live `:443`
socket probe for `xalert`.

## A budget that could never balance

**Symptom:** seven straight hours of GitHub `403 rate limit exceeded`, while
the rate-limit endpoint reported `used: 0`.

`gramwatch` cost 18 GitHub calls per cycle and polled every 600s - 108
calls/hour against an unauthenticated ceiling of 60. It burned the budget
before it could ever finish a first clean pass, so it never stored ETags and
never set `baselined`, which guaranteed full price on every subsequent cycle.
A self-sustaining deadlock. The `used: 0` reading was real: each window's
failures had happened in the previous window.

The code's own docstring claimed a 304 does not consume quota. Measured
unauthenticated: 9×304 + 1×200 moved the budget 28 → 18. **A 304 costs a full
call.** The no-token design rested on a false premise.

**Fix:** split the clocks. Channels every 300s (free, and the tier that
matters), repos every 1800s via a new `repo_poll_seconds` gate - about 20
calls/hour against 60. Verified: two channel-only cycles moved the budget
11 → 11.

**Rule:** a GitHub PAT in `github_token` raises the ceiling to 5,000/hour and
makes all of this irrelevant. The code already reads it, and
`os.environ["GITHUB_TOKEN"]` as a fallback. Ben has not created one.

## A threshold of zero

**Symptom:** would have been a firehose.

`crashbot` shipped with `min_usd 0` and `min_sol_move 0.05` - every dust
movement across nine wallets, when the stated requirement was $1,000. Set to
`1000` / `5`.

**Rule:** read the thresholds before concluding a bot is broken, and before
concluding it is quiet.

## A whale mistaken for a funder

**Symptom:** a wallet-linking inference built on nothing.

`iGdFcQoy…` looked like the shared funder of two Crash wallets. It holds
~438,886 SOL and turns over ~425 tx/hour - an exchange hot wallet. Sharing a
CEX withdrawal address is not evidence of common ownership.

**Rule:** before treating a shared funder as a link, check its balance and
transaction rate. Exchange-scale means it links nothing.

## The wrong local shell

**Symptom:** confident conclusions drawn from another machine's state.

`device_bash` runs in an isolated Linux VM with the user's folders mounted -
**not** on the Mac. It has no network and cannot see Mac processes, so
`ps aux` there returns the sandbox's processes and `urllib` to
`api.telegram.org` raises `URLError`. Desktop Commander
(`mcp__remote-devices__Desktop_Commander__*`) runs on the real Mac with real
network and real process visibility.

**Rule:** for anything about what is running or reachable, use Desktop
Commander. `device_bash` also cannot delete files - move unwanted files into
a `_to_delete/` folder instead.