---
name: trench-model-2
description: Continuation of trench-model.md from 13 Sep 2026 21:35 UTC, after that file hit its size cap. The three error-sweep passes - the survivorship bug in the labeller, reconstructed training rows built from klines and wallet entry times, and the calibration correction. Read with trench-model.md before any work on the GMGN runner-prediction backend.
sources: [cowork]
aliases: [trench model part 2, trench-model-2, recon rows, reconstructed training rows, rebuild.py, trench audit]
---
Continues [[trench-model]], which hit its 48KB cap. Everything there still applies; this is the next three passes.

# THREE ERROR-SWEEP PASSES, 13 Sep 2026 21:35 UTC. One fatal bug, one big missing piece.
- [stated] Ben: "after this run 3 more iterations to make sure there's no errors and nothing crucial missing"

## THE FATAL BUG: 1,099 SNAPSHOTS AND 2 OUTCOME ROWS
`/opt/trench/audit.py` (new, read-only, checks that every thread has actually WRITTEN recently rather than that the process is alive) found the system was collecting data it could never use. In `label_one`:
```
after = [x for x in pts if x[0] > sts]
if not after: continue          # silently drops the snapshot
```
`pts` comes from `token_kline`, and **a candle only exists where a trade happened**. A token that dies stops producing candles, so every token that went nowhere hit `continue` and produced no outcome row. **That is not a data gap, it is the answer**, and it is the most common outcome on this chain. The training set could only ever have contained survivors: a survivorship filter on the LABEL, which is worse than one on the sample.
Fix: merge our own `obs` rows (written every poll whether or not anyone traded) into the point list; if there is still nothing after the snapshot and more than an hour has passed, record the token as DEAD (peak after = its own snapshot market cap, rug=1); only skip when the snapshot is under an hour old and genuinely unjudgeable. **Gate checks 22-25 replay it.**
The identical bug existed a second time in `rebuild.py` (`len(bars) < 6` rejected tokens with 0-4 candles, which was eleven of twelve pending tokens). Fixed the same way. **It cost twice in one evening because I fixed the symptom and not the habit: absence of data is not missing data when the silence is itself the measurement.**

## THE MISSING PIECE: RECONSTRUCTED TRAINING ROWS, `/opt/trench/rebuild.py`
A live training row needs a token watched from birth and then aged past the horizon, so the system had **six** eligible tokens and would have had a usable sample in about a week, while holding 131 tokens that actually ran and learning nothing from any of them.
**The insight: two of our sources are RETROACTIVE FACTS, not current state.** `token_kline` gives the whole price and volume history, so what the market cap was at minute five is recoverable today. `token_top_traders` gives `start_holding_at`, so WHICH WALLETS WERE IN at minute five is also recoverable. An honest point-in-time vector can therefore be rebuilt for any token that ever existed and scored against what actually happened next.
- **29 features** (`RECON_FEATS`): price and volume shape up to the cutoff (mcap, off-high, volume, volume acceleration, bars traded, **dead share**, returns, volatility, slope, acceleration) plus who was in by the cutoff (count, arrival rate, how many have a measured track record, their mean and max lift, first entry minute).
- **Deliberately NOT used, because it would leak**: everything GMGN reports as current state (holder counts, dev flags, taxes, wallet tag stats, socials), and from the roster ONLY the entry timestamp. Not `buy_volume_cur`, not `profit`, not `end_holding_at`: those are lifetime figures that already contain the future.
- **Resolution decides the age**: `limit` is the only paging parameter, so 1m candles reach back 25h and no further. An older token cannot have its first five minutes rebuilt at minute resolution, so each token is reconstructed at the ages its history honestly supports (`pick_res`). A coarse row about a real $300M runner beats no row.
- Trained by `recon_training_rows` through the SAME `fit_and_promote` gate as the live models (beat the base rate by 2%, beat the incumbent by 1%, on a holdout the model never saw). Named `rc-<target>-r<age>`.
- **`recon_live_vector()` scores LIVE tokens with it**, from their klines and rosters, so the history-trained model touches real predictions instead of being a statistic. Its prediction is stored alongside the live one, never replacing it, so the report can show which is calibrated.
- `reconfill.py` backfills in bulk; the `reconer` thread keeps it current.

## THE CALIBRATION TRAP THAT CAME WITH IT
The reconstructed sample was winner-heavy (52 of 101 rows at the 6h age reached $1M, against a true rate of 1 in 1,846) because `reconfill` ordered "biggest peak first" and the winners were discovered BECAUSE they ran. Read as a probability, that would have told Ben half the board was about to do a million dollars. **Same error as the base rate, one layer down.**
- `reconer` rebuilds in **creation order, not peak order**, continuously, so the sample converges on the population.
- Every model records `sample_rate`, the base rate it was fitted on.
- **`M.prior_shift(p, sample_rate, true_rate)`** converts a score from sample odds to population odds. It preserves RANKING exactly (which is what makes it safe to apply) and changes only what the number means. `M.TRUE_RATE` holds the measured per-launch rates. **Gate checks 26-30** assert the shift lands near the true rate, preserves order, and leaves a model with no recorded sample rate alone.

## OTHER REAL FIXES THIS PASS
- **`pick_threshold` returned None for every rung**, so the `run` model never trained at all, in any band. The old rule was two separate gates ("at least 12 positives" AND "at most 60% positives") and a skewed sample fell straight through the gap between them: $1M had 69% positives, $5M had 9. Replaced with one question: **the rung that maximises the smaller side**, with the floor scaled to the sample. It cannot return None when a usable rung exists.
- **The holdout day was too thin to fit on.** `fit_and_promote` split on the newest calendar day and required 25 rows in it; reconstructed rows spread over weeks of launch dates. Now falls back to a time-ordered last quarter, which is an equally honest holdout and always has enough rows.
- **The 5m reconstruction age could almost never settle**: 1m candles need the token under 25h old, and the shortest horizon was 24h, so a row was usable for a fifty-minute window in a token's life (4 usable rows of 263). Added a **6h horizon** to `recon`, widening it to between 6h and 25h. 5m went from 4 usable rows to 111.
- **patch48 deleted two module constants by slicing to the next `def`.** `HORIZONS` and `MIN_SETTLED` lived in that gap, and `horizon_for` then raised NameError on every training pass. **Anchor a patch on the exact text being replaced, never on "everything up to the next boundary".** Same family as the schema regex that missed `tok.late`.
- **`seed.py`**: pulls the full historical universe from GMGN rank (every interval x sort order) plus GeckoTerminal pools, all as `late=1`. Old tokens are what the trainer needs, because only a CLOSED horizon teaches anything.
- `audit.py` now scopes errors to since-the-last-restart (it was reporting tracebacks already fixed by the restart that followed them) and reports the recon table.

## STATE, 21:35 UTC. FIRST LEARNED CHAMPIONS PROMOTED.
`trench.service` active. **Gate 30 of 30. `audit.py` ALL CLEAR.**
- **1,182 reconstructed rows over 457 tokens**: 5m 276, 15m 276, 1h 356, 6h 274. At the 6h age, 102 rows reached $1M and 61 reached $5M.
- **First two learned models promoted**, both from reconstructed history: `rc-run-r21600` at threshold $1M, **Brier 0.1433**, and `rc-rug-r21600`, Brier 0.2848, each beating its base rate on a holdout it never saw. The live bands are still on `rules-v0` and will be until tonight's tokens age past their horizons.
- Wallet layer: 29,038 roster rows, 16,572 wallets, 631 tokens.
- 923 tokens total, 369 late-discovered, 222 older than 7 days (horizons closed).

## THE TOOLS TO RUN FIRST NEXT SESSION, in this order
1. `python3 /opt/trench/audit.py` — liveness per thread, errors since last restart, data consistency, what is settled, the recon table. Should say ALL CLEAR.
2. `python3 /opt/trench/traintest.py` — forces a training pass and says for every band and age exactly WHY a model did or did not fit ("n=45 needs 120"), then lists the models and their Brier scores.
3. `python3 /opt/trench/gate.py` — 30 checks, every past mistake replayed. Run before any change ships.
4. `python3 /opt/trench/peek.py` — read-only live status. Never an ad-hoc query: `store.db()` takes a write lock.
Diagnostics kept for reuse: `diag_outcome.py` (where outcome rows are being lost), `diag_recon.py` (why a token produces no reconstructed rows), `diag_thr.py` (why a threshold was or was not picked), `basecheck2.py` (the real base rate), `ratetest.py`.

# THE TABLE WAS INCOHERENT, AND BEN CAUGHT IT AGAIN. 13 Sep 2026 22:05 UTC
- [stated] Ben: "Your table there makes no sense. If it's 11% of 6,500 launches, 6%, those numbers make no sense. 33% run to 5 to 50 million, and that's one in 2,800... I think the large majority stays under 100K, obviously. And so it's basically like a pyramid, right? Most tokens stay very small. Some tokens get medium and so on. A few less tokens get bigger and very few tokens go above 50 million, right?"

**He is right, and his pyramid is exactly the right shape.** The fault was mine and it was not a wrong number, it was **two different denominators printed side by side without saying so**:
- "among tokens that trade" was really **"among the ~326 tokens big enough to appear on GMGN's rank list"**, which is a TOP-N list, not a population. Saying "only 11% of those are under $100k" is just saying a list of the biggest tokens is not full of small ones.
- "per launch" used all daily launches.
Read together they look self-contradictory. **This supersedes the two-column table in [[trench-model]]'s correction section.**

## THE MEASURED PYRAMID, one denominator, every launch (`/opt/trench/pyramid.py`)
Launches per day measured from the collector's own window (2.51h watched, 339 tokens born in it): **~3,241 a day**, so ~97,000 in 30 days.

| out of every launch | count | odds | share |
|---|---|---|---|
| stays under $100k | 96,815 | | **99.56%** |
| reaches $100k | 428 | 1 in 227 | 0.440% |
| reaches $1M | 238 | 1 in 409 | 0.245% |
| reaches $5M | 134 | 1 in 726 | 0.138% |
| reaches $50M | 22 | 1 in 4,420 | 0.023% |

**And the number that actually matters once something is already moving**: of the 428 tokens that ever touched $100k, **56% went on to $1M, 31% to $5M, 5% to $50M**. That is the real shape of this chain: **getting off the floor is the hard part, and after that the drop-off is gentle.** A token at $100k is not a long shot for $1M, it is better than a coin flip.

## THE CAVEAT, kept attached to the numbers
GMGN's rank list returns 100 rows a call and surfaces the BIGGEST first. So the **$50M count is close to complete** (only a couple of dozen tokens have ever been that big and all of them are on the list) while the **$100k count is a FLOOR**: most tokens that briefly touched $100k are long gone and never appear. The bottom of the pyramid is at least this steep and probably steeper; the top rows are the trustworthy ones.

## SHIPPED
- `model.TRUE_RATE` rebuilt on this one denominator: 25k 1 in 90 | 100k 1 in 227 | 250k 1 in 300 | 1M 1 in 409 | 5M 1 in 726 | 50M 1 in 4,420. `model.GIVEN_100K` holds the conditional table. `SEEN` rewritten to the same denominator so it reads as a pyramid.
- **Gate checks 31-33**: TRUE_RATE must be monotonic (every tier rarer than the one below, which is what a pyramid IS and what a mixed-denominator table fails), SEEN must sum to a whole population, and the conditional table must agree with the per-launch one (rate of $100k x share reaching $1M must land on the measured rate of $1M). **Gate now 33 of 33.**
- Gate check 30 was still pinned to basecheck2's superseded rates and went red on the corrected ones, which is the gate working. Re-pointed at pyramid.py and pinned by value, so a silent edit to the odds fails there rather than quietly changing every prediction.

## THE LESSON, third time this evening in a different costume
Every one of tonight's three big errors was the same mistake wearing different clothes: **a number computed over whatever set happened to be in front of me, presented as if it described the world.** The blind base rate (collector's sample), the winner-heavy recon set (discovery method), and now a top-N list read as a population. The habit that catches it: **before printing any rate, say out loud what the denominator is, and check that it is the population the reader will assume.** If two rates in one table have different denominators, they do not belong in one table.

# FIRST OVERNIGHT RUN, reviewed 14 Sep 2026 03:25 UTC. Collection excellent, learning loop broken in four places.
- [stated] Ben: "good morning, how did the collection and learn process perform over night?"

## WHAT WORKED, unattended, no restarts, no rate limits
477,448 observations (about 40,000/hour), 1,554 tokens, 3,216 snapshots and predictions, 1,263 labels, **zero errors since the last restart**, zero 429s. Wallet layer grew to **61,074 roster rows over 26,011 wallets and 1,152 tokens, 254 wallets now with 3+ closed tokens**. Reconstruction reached **2,203 rows over 799 tokens**. `risers` found **579 tokens the launch board never showed**. The data side of this is solid.

## FOUR BUGS, all in the LEARNING loop, all found by audit.py + diag scripts
**1. The model could never improve, because it never found its own incumbent.** Models are STORED with the threshold appended (`run@1000000`) and were looked up by the bare name (`run`). `champion()` returned None on every refit, so the "beat the incumbent" half of the promotion gate was vacuous: **eleven identical models, identical Brier 0.1433, on the same 93 rows, each promoted in turn.** Fixed by matching both spellings. Ten duplicates demoted. **Gate 34-36.**

**2. Nothing ever settled, so no live band could ever train.** 1,817 outcome rows, **zero with ready_6h=1 after twelve hours**, with 1,220 rows already past the horizon. Three causes stacked:
   - the relabel query ended `ORDER BY t.created_ts DESC LIMIT 25`, so it picked the 25 newest tokens every pass and never revisited anything;
   - ordering by "most overdue" instead still starved them, because `COALESCE(l.ts,0)` puts every never-labelled token at the front. **Sorting harder does not fix a queue holding two different jobs.** Now two batches, 20 fresh + 40 overdue, neither can starve the other;
   - the queue was then full of **seeded and late-discovered tokens that have a `lab` row but NO SNAPSHOTS**, and an outcome row is written per snapshot, so `label_one` returned None for all of them and they took every slot. Queue now requires `EXISTS(SELECT 1 FROM snap ...)`, dropping it from 822 to 487.
   Throughput also raised from 25 per 5 minutes to 60 per 3 minutes (one kline call each, weight 2, about 0.67 units/s of 20). ready_6h went 0 -> 21 within minutes and is clearing.
   **And the log line was guarded by `if n`, so a labeller processing 40 tokens and writing nothing looked identical to one with no work.** It now logs every pass, including empty ones.

**3. "5-50M" printed on 245 predictions, 7.6% of the board, against 1 in 726.** `peak_dist` reads `lab` to project how high a token that clears the rung goes, and `lab` now holds 579 `rank_ath` rows (the late-discovered winners, average peak $31.9M). Median peak of anything clearing $100k read as **$1.54M** instead of $211k. **Third appearance of the same error: a sample of winners read as a sample.** Fixed: peak_dist reads only watched tokens (`late=0`, source != rank_ath).

**4. THE TIER LADDER WAS STRUCTURALLY INCAPABLE OF SEPARATING TIERS, and this is the interesting one.** After bucketing on odds rather than a projection, the gate asserts (correctly) that the conditional table and the per-launch table AGREE: P($50M) = P($100k) x 5%, and TRUE_RATE[50M]/TRUE_RATE[100k] = 0.051 against a conditional of 5%. **Because they agree, a single multiple-of-base-rate bar is cleared by every tier at once or by none.** Measured: lift 3 gave 8% "huge runner" and zero of everything between; lift 20 gave nothing at all. The ladder was not mis-tuned, it could not work.
   The fix is a reframing. Two different questions were hiding in one word:
   - **does it run at all**, which the model predicts and which varies hugely between tokens;
   - **if it runs, how far**, which is roughly FIXED by the chain (of tokens reaching $100k: 56% reach $1M, 31% $5M, 5% $50M).
   A stronger signal does not move a token up that distribution, it raises the chance it enters it. So the tier now comes from **STRENGTH**: the calibrated probability as a multiple of the population rate. `TIER_BANDS`: under 3x no run, 3-12x could run, 12-40x 1-5M, 40-120x 5-50M, over 120x huge runner. A token already above a tier is named there directly.
   **And p_run had to be calibrated first.** `rules-v0` emits hand-written numbers averaging 0.072 and the learned models are fit on a winner-heavy sample; neither is on the population's scale. `calibrated_p` shifts using the model's recorded `sample_rate`, or for rules-v0 its OWN measured average output, which is by definition its base rate.
   Board now: **no run 79.6%, scam 17.2%, could run 3.1%, nothing higher.** Honest: no model is yet confident enough to claim $1M.
   **Gate 38-42** catch the collapse class directly: a rising signal must never produce a weaker verdict, the verdict must use more than one word across a realistic sweep, and a token already past a tier must be named there. **A verdict function that returns one label for everything passes every correctness test; it needs a behavioural one.**

## GATE NOW 42 CHECKS. `audit.py` ALL CLEAR.

## THE RECURRING LESSON, now four for four
Every significant error in this project has been the same shape: **a number computed over whatever set was in front of me, presented as if it described the world.** Blind collector sample, winner-heavy recon set, top-N list read as a population, and now `lab` polluted with seeded winners. The habit that catches it: **before any rate or projection, say what the denominator is and check it is the population the reader will assume.**
Second recurring lesson, also four for four: **never rebuild a file from a prefix or slice "to the next def".** patch48 ate HORIZONS and MIN_SETTLED; patch61 ate prior_shift, TRUE_RATE and GIVEN_100K. Replace the exact span, then assert every expected symbol still resolves.

# PRODUCT NAME: PRICE PREDICT. Set by Ben 14 Sep 2026.
- [stated] Ben: **"1. The original one, where we give ratings from 1 to 5 stars to the project based on the website... We call that TECH SCORE. 2. What we've built and was running overnight, this is what we call PRICE PREDICT, right, because we're trying to predict if this thing is going to be a runner or not."**
So everything in [[trench-model]] and this file is **Price Predict**. The 1-5 star website rating in [[casig-gmgn-overlay]] is **Tech Score**. Use these names in every reply and every label.
- Tech Score answers "did anyone actually build this". Price Predict answers "is this going to run".
- **Tech Score feeds Price Predict as two features** (the star value, and whether it says scam). Not the other way round.
- Applied: report titled **Price Predict** at http://159.223.88.168/trench.html with the subtitle naming the split; prediction reason strings now read "Tech Score 3" / "no Tech Score".

# ONE OWNER PER DIRECTORY, and the Tech Score handover. 14 Sep 2026.
Two Cowork sessions were editing `/opt/casig` at the same time (a `score` table gained columns this session never defined; `bak-20260914-0402/` and `patch_casig2.py` appeared from the other, which had already written `cohort_of`, `training_rows` and `reference_rows` into `ledger.py`). Rule from here:
- **`/opt/trench` is owned by the Price Predict session.** `/opt/casig` is owned by the Tech Score session. `/opt/cleaner` either may touch, but announce first.
- A session needing a change outside its own directory asks the owner session rather than editing.
- [stated] Ben approved handing the known-outcome Tech Score scoring job to the Tech Score session: "Yes, you can hand it that job." Full handover in [[techscan-2]].
What Price Predict owes that job: the **1,159 tokens with a known peak** (139 above $5M, 118 at $1M-5M, 239 at $100k-1M, 663 dead) are read out of this DB, biggest peak first. `outcome_backfill.py` in `/opt/casig` was stopped mid-run having written zero rows; it is now the Tech Score session's to take over or replace.

# 14 Sep 2026. THE TECH SCORE FEATURE HAS BEEN A CONSTANT ZERO SINCE IT WAS WRITTEN.
Found by the Tech Score session while building the coin-page view (see [[casig-gmgn-overlay]] v2).
`stars_client.stars(addr)` reads `GET /v?k=...&ca=<addr>` on casig. That endpoint contained a live NameError (`fresh(CACHE.get(ca), links)` with no `links` in scope), so **every call raised, `stars()` swallowed it and returned `(0, "")` for every token, always.** The model's `stars` and `star_scam` features were not weak, they were **absent**, and a constant column teaches nothing while looking exactly like a feature that does not matter.
Same shape as the 821-of-1,763 missing star fields the ledger found the day before, and the same lesson again: **a measurement that silently fails reads as a clean zero.** `ask()` worked the whole time, so casig was rating these tokens and Price Predict was throwing every answer away.
- Fixed on the casig side. **`/v` with no links is now a READ**, never a scan: it serves the cached record (marked `stale`) or says `pending`. That mattered more than the NameError fix itself, because a naive fix would have had Price Predict's own lookups scan tokens with nothing to check, conclude "no website", overwrite good cache records and append false ledger rows. **The reader would have corrupted the table it feeds.**
- `deploy.sh` gained a **step 5 smoke test that runs AFTER the restart**, because selftest runs before it and therefore gates the OLD process, and nothing living in the HTTP handler can be tested there at all. It asserts `/v` with no links adds zero ledger rows, and that `/v` with links returns a rating carrying its chain.
- **CROSS-BOUNDARY EDIT, disclosed**: `/opt/trench/stars_client.py` was patched by the Tech Score session (before the one-owner-per-directory rule above was read) so `ask()` sends `created_ts`, `holders`, `mcap` and `view: "predict"`, putting its ledger rows in a measured cohort instead of `unknown`. Backup at `stars_client.py.bak-cohort-*`. trench.service was restarted; `audit.py` ALL CLEAR afterwards, 2,394 recon rows over 863 tokens, 44 models stored. Nothing else in `/opt/trench` was touched.
- **Worth re-checking once real star values start arriving**: every model fitted before 14 Sep 2026 was fitted with this feature dead, so any conclusion about whether Tech Score predicts a runner is currently unmeasured, not negative.

# ASKING THE RIGHT QUESTION. 14 Sep 2026 04:20-04:50 UTC. Patches 68-73.
- [stated] Ben: "What are the best next steps to improve the Price Predict?" then "Okay, run it."

## THE HIDDEN BUG: EVERY PREDICTION HAD BEEN FALLING BACK TO RULES FOR HOURS
`audit.py` showed 23 `freeze-err` since restart, message `KeyError('m')`. The `champion()` rewrite (the fix that made `run@1000000` findable by the bare name `run`) changed the returned dict to `{name, blob, score, target}`, but `predict()` still reads `cr["m"]` and `top_drivers()` reads `cn["m"]`. So **every freeze pass that touched a band WITH a champion threw and aborted the whole pass**: snapshot throughput fell from 1,265 in the 2-6h window to 377 in the last 2h, and no learned model reached a live prediction.
**patch68**: `champion()` returns BOTH spellings, `"blob"` (raw) and `"m"` (parsed). Neither side can break the other again.
**New lesson, a class of its own**: a fix that changes a function's RETURN SHAPE must be checked against every caller, not only the caller that was broken. The "never rebuild from a prefix" rule does not cover a changed contract.
Note this bug and the constant-zero `stars` feature above are the same family: **a failure that is swallowed looks exactly like a working system with nothing to say.**

## THE REAL PROBLEM: THE MODEL WAS LEARNING THE WRONG QUESTION
Live champions were fitted at **run@25000**. `pick_threshold` walks `LADDER` and $25k was the bottom rung, so on a thin sample it always won. $25k is below the pyramid's first real rung ($100k, 1 in 227) and is not a run by any reading. Meanwhile only the **6h horizon ever settled** (24h/72h/7d/30d all zero), because a row could only enter training once its whole window had closed, so the system was learning "does it move a little inside six hours" while Ben's question is "does it reach $1M to $50M", which on this chain takes DAYS.
**patch69/70, the reframing**: right-censoring done properly. Inside a FIXED window,
- a row that has **already reached the threshold is settled YES now**. A peak does not un-happen, so nothing left in the window can change it. That is a fact, not early reading.
- a **dead token is settled NO now**. It makes no candles and it is not coming back.
- a **closed window** settles either way.
- everything else is genuinely pending and stays out.
Every label still refers to the same fixed window, so the question does not drift between rows; the yes-cases simply stop waiting out a clock already decided. `LADDER` floored at **$100k** (`RUN_FLOOR`), new module function `settled(peak_hz, ready_hz, rug, thr)`.

## THEN I MADE THE SAME ERROR TWICE MORE IN ONE HOUR, IN NEW COSTUMES
**Fifth and sixth appearance of "a sample of whatever was in front of me".** Both were self-inflicted by the fix above, and both were caught by `traintest.py` within minutes, which is the argument for running it after every change.
1. **Applying the monotone rule to RUG inverted the sample.** A rug is OBSERVED early; "not a rug" is only known when the window closes. So admitting early rugs does not settle rows early, it **filters the sample down to rugs**: 1h read 187 rows and 187 positives, and `rc-rug-r3600` promoted at **brier 0.0006 on 436 of 450 positives**, a model that always says rug. **patch71**: rug trains on CLOSED windows only. The asymmetry is the whole point: the monotone rule is safe when the early-settling class is the RARE one and destructive when it is the common one.
2. **The horizon chooser then selected for winners.** `horizon_for` counted early-settled rows as evidence a horizon was usable, so it walked out to 72h/7d where nothing had closed, and the only rows able to settle were ones already past the bar: recon 5m read 46 rows and **46 positives**, 15m 40 of 40, 1h 84 of 86. **patch72**: a horizon is usable when enough windows have CLOSED at it, and early settling never votes for its own horizon. Inside the chosen horizon, **if the early-settled positives are the MAJORITY of the positives they are discarded** and the fit falls back to closed windows only. A top-up that outnumbers what it tops up is not a top-up, it is the sample.

## PATCH73: BOTH RULES ARE NOW NAMED FUNCTIONS WITH GATE CHECKS
Standing rule is that a repeated mistake gets code and a gate, not another paragraph. `settled()` and new `usable_sample(n, pos)` (refuses a sample with under 12 or under 8% in the minority class, which is what let the always-says-rug model through). **Gate now 53 checks, all passing**, including: the ladder has no rung below $100k; a bar already cleared settles yes with the window open; a dead token settles no without waiting; a 436-of-450 sample is refused; the early top-up cannot outnumber the positives it tops up; and no horizon chooser counts an early-settled row as a closed window.

## STATE AFTER THE WORK
- `audit.py` **ALL CLEAR**, zero freeze-err.
- **Three run@25000 champions demoted**, plus the degenerate `rc-rug-r3600`. Champions left: `rc-run-r21600` (run@$1M) and `rc-rug-r21600`. The live bands are back on rules-v0 **on purpose**: at $100k they hold under 12 positives, so the honest answer is that they cannot fit yet. They will as watched tokens accumulate.
- Class balance restored everywhere: recon run 18/60 at 5m, 16/54 at 15m, 29/54 at 1h, 58/76 at 6h.
- **Reconstruction is now the binding constraint, not the labels**: bands need 120 rows and sit at 40-86. `/opt/trench/reconall.sh` runs `reconfill.py` in passes until under 20 tokens remain unreconstructed, because **reconfill orders biggest-peaks-first and a half-finished run leaves a winner-heavy sample**, which would be the same error a seventh time. Recon grew from 2,394 to 2,817+ rows over 1,019 tokens during the session; 820 tokens were still unreconstructed when it started.
- Wallet layer: **467 wallets with 3+ closed tokens** (was 393). Still the largest untested feature and the next job: score each wallet, score a token by who is in it by minute five, then fit the same model with and without those columns on the same holdout. Ben's hunch gets a measured yes or no rather than an opinion.
- Also still open, and now un-blocked by the casig fix above: re-measure whether Tech Score predicts a runner, since every model before today was fitted with that feature dead.

# BEN'S ATTENTION THEORY, TESTED AND CONFIRMED. 14 Sep 2026. `/opt/trench/attn_test.py`
- [stated] Ben: "on GMGN you can see how many people are watching a coin, there's an eye symbol... I believe that the very first few minutes are going to be predominantly dominated by people who are in the trenches on these pages and not looking on social media. Maybe after 5, 10, 15 minutes, the people who are already positioned in the coins will try to get attention, and they will post about it... in different Telegram groups, and then it will start spreading because they need attention and they need new buyers to ride the coin up."
- [stated] Ben on wallets: "at the very beginning, people have their own categorization process... either sophisticated or not. Some go by gut feeling, some have some checks, some have it automated, and then they will also check WHICH OTHER WALLETS ARE BUYING. Some have that automated also."

**The eye is `visiting_count`, and it was already in the stored payload all along** (one of the 100 trenches fields). So was `callout_count`, `tg_call_count`, `smart_degen_count`, `renowned_count`. `obs` keeps the whole payload on every poll, so an attention time series for every watched token was already on disk and had never been looked at.

## THE MEASUREMENT, 26,053 observations over 560 WATCHED tokens (late=0 only, so no winners-only comparison)
Mean by minutes since launch, split by what the token actually went on to do.

**`visiting_count` (the eye) at minutes 0-2, and this is the strongest single separator found in this project so far:**

| what it went on to do | 0-2m | 2-5m | 5-10m | 10-15m | 15-30m | 60-180m |
|---|---|---|---|---|---|---|
| reached $250k-1M | **219.0** | 140.4 | 60.8 | 124.8 | 86.6 | 42.1 |
| reached $100k-250k | **147.1** | 97.1 | 46.6 | 18.3 | 14.9 | 4.2 |
| reached $30k-100k | 37.0 | 13.4 | 4.5 | 2.3 | 1.5 | 1.1 |
| never left the floor | **4.4** | 1.2 | 0.6 | 0.3 | 0.2 | 0.2 |

**About 50x between a dud and a $100k+ token, visible at minute two.** Sample caveat: only 8 tokens in the $250k-1M band and 1 above $1M, so the top two rows are thin and the shape matters more than the exact numbers.

**The two-phase structure he described is in the data.** The eye peaks in the first two minutes and decays; `callout_count` does the opposite, climbing over the same window for winners (0.0 at 0-2m to 4.6 at 60-180m for the $250k-1M band) and staying flat zero for everything that never left the floor. **Attention arrives on the page first and on social second, exactly as he said.**

**Smart money is in from minute zero, not later**: `smart_degen_count` at 0-2m reads 14.8 for the $250k-1M band against 0.3 for the floor, and `renowned_count` 1.5 against 0.0. This is the wallet thesis showing up as a GMGN counter rather than as our own roster work, and it is direct evidence for his "they check which other wallets are buying" mechanism.

**`tg_call_count` is dead on this chain**: zero in every band at every age, 24 nonzero rows in 3,000. Do not build on it.

**Watchers per holder** (visiting_count / holder_count) is 1.2 at 0-2m for the floor against 2.7-3.4 for everything that went anywhere, but it decays fast and separates far less cleanly than the raw eye count. Keep the raw count as the feature; the ratio is the weaker form.

## THE CAVEAT THAT HAS TO TRAVEL WITH THIS
`visiting_count` is partly GMGN's own surfacing: a token the trench board ranks highly gets looked at, so the eye measures real interest AND the board's decision to show it. That still makes it a legitimate predictor (it is observable at minute two and it precedes the price), but it is not purely organic demand and should never be described as such.

## WHY THIS HAS BEEN WORTH NOTHING SO FAR
The LEVELS are already in `feats.py` (`visiting_count`, `callout_count`, `tg_call_count`, `smart_degen_count`, `renowned_count`, `x_user_follower`). **They have never been learned from**, because no live band has ever fitted a run model at a meaningful rung and the champions that did exist were fitted at $25k. The signal was being recorded and thrown away.
**And it can never come from the recon layer**: klines carry price and volume only, so attention exists solely in the live path. That raises the priority of getting live bands to train, which needs watched tokens, not more reconstruction.

## WHAT TO BUILD NEXT, in order
1. **Attention VELOCITY, not level**: per-minute slope and acceleration of `visiting_count`, its peak so far and distance from it, and the phase marker "callouts rising while the eye is still high", which is his hand-off from page to social.
2. **The scoreboard** ("did we beat a coin flip"): score every stored prediction against what the token actually did, by bucket, so a silent breakage like today's `KeyError('m')` shows up as a flat line instead of going unnoticed for hours.
3. **Wallet identity and co-movement**: which wallets, and whether a wallet's entry follows other wallets' entries, which is the automated-copy behaviour he described.

# ATTENTION VELOCITY, THE SCOREBOARD, AND TWO DEAD COLUMNS. 14 Sep 2026, patches 74-78.
- [stated] Ben: "Everything you build, recheck it three times to make sure it's accurate, efficient, elegant, and fast."

## `/opt/trench/attn.py` — attention as a SPEED, not a level (38 features)
5 counters (`visiting_count`, `callout_count`, `smart_degen_count`, `renowned_count`, `holder_count`) x 6 shapes (now, peak, off_peak, slope/min, slope over the last third, acceleration) plus 8 shape features: the eye at first sight, how early "first" was, when the eye peaked, watchers per holder, callouts per watcher, **`attn_social_phase`** (callouts climbing WHILE the eye is still at half its peak, which is Ben's page-to-social hand-off as one flag), span and observation count.
- `tg_call_count` deliberately excluded: measured dead on this chain. **Gate check asserts it stays out**, because a dead column costs a fit and pays nothing back.
- Every value comes from observations at or **before** the cutoff. Gate proves it on a real token: the peak at an earlier cutoff can never contain a later peak, and `now` equals the last observation on or before the cutoff.
- **FEATS 193 -> 231.** Append only, and that is now load-bearing: `model.apply_logistic` reads the **PREFIX** when the vector is longer than the model, so a model fitted on 193 columns is still fed the right 193 of 231. Without that rule every existing champion would have gone silent (returning None, which looks exactly like working) the moment the vector grew. Gate checks both directions.
- **`attnfill.py`** backfilled the block onto 3,298 existing snapshots from `obs` in 41s, grouped by token so the history decompresses once per token rather than once per band. Without it the whole live sample would have been dropped by the length check. 537 older rows (120 and 168 long, pre-dating the info and wallet blocks) remain unusable and were already dead before today.

## `/opt/trench/score.py` — THE SCOREBOARD. "Did we beat a coin flip?"
One fixed bar for every model, **$100,000**, so models are comparable. Headline is **AUC**, because AUC *is* the coin-flip question: the chance the model scored a token that ran above one that did not. 0.50 is a coin flip exactly, and it judges ORDER only, which is the right test for a board that is read from the top by a model whose probabilities are not yet calibrated. Writes one row per run to `sboard` so the trend is kept. **Hourly cron at :17.**
**First run read AUC 0.965 and a 9x top-decile lift, and that was too good.** Two inflations, both now separated out rather than argued away:
1. **Easy mode vs HARD MODE.** A token already at $90k when we looked is nearly certain to touch $100k, so the board was partly reading back the market cap it was handed. Hard mode keeps only snapshots taken **under $50k**, which is the question Ben actually asks.
2. **2,272 rows is not 2,272 events.** One token appears at up to five bands. The board now counts **distinct winning TOKENS** and refuses a verdict under 10 of them however large n is.
**Honest current state: 8 winning tokens overall, 4 in hard mode. Verdict withheld: "NOT ENOUGH SMALL TOKENS THAT RAN YET."** Worth keeping for the trend: hard-mode AUC 0.919 and a top-decile lift of 8.3x on those 4, and the word "could run" on the board has landed 21.3% against a 0.9% base (24x), while "scam" has a 0.0% hit rate on 401 rows.

## THE DEAD-COLUMN CHECK, and the two columns it found in twenty minutes
**Third time this failure has cost us**: a measurement that silently fails reads as a clean zero, which is indistinguishable from a feature that does not matter (Tech Score stars, the `KeyError('m')` fallback, now these). `audit.py` gained `dead_column_check`: every FEATS column against the last 500 snapshots, naming any that never varies, and **failing outright if a load-bearing one goes flat** (`attn_visiting_now`, `attn_visiting_slope`, `visiting_count`, `holder_count`, `market_cap`, `stars`). It is generic, so it covers feature blocks added after it was written.
It immediately found **`d_volume_per_min` has been a constant zero for the life of the project**: `history()` returned `(ts, mcap, holders, prog, 0.0)` with the volume slot hardcoded, while already SELECTing and discarding the payload that holds `volume_24h`. Fixed inside `history()` (patch78), so no signature change and no refill: the column just starts carrying a number. 29 constant columns remain and most are legitimately flat flags on a clean sample.

## THE THREE RECHECKS, `/opt/trench/recheck.py`, 20 of 20 passing
Not a gate rerun. Accurate: newest 400 snapshots all 231 long, the eye column varies and is populated, leakage measured on a real token, the scoreboard join proven to match each prediction to its own outcome row, zero outcome peaks at or before their snapshot. **Fast**: an attention vector costs well under 40ms cold and 0.25ms warm over 624,684 observation rows; the scoreboard runs in seconds. Elegant: NF matches FEATS, no duplicate names, the block is the tail, a vector built without the block still pads to full length with zeros, an older short model still scores a 231-long vector.
**Gate is now 68 checks, `audit.py` ALL CLEAR, recheck 20 of 20.**

## STATE
Wallet layer 588 wallets with 3+ closed tokens (was 393 this morning). Recon 3,024 rows over 1,085 tokens; `reconall.sh` finished its five passes with 664 tokens still unreconstructable (no supply or too old for minute resolution), which is a floor, not a queue. Live bands still cannot fit a run model at $100k and that remains correct and expected.
**Next: the wallet layer proper.** Score each wallet, score a token by WHO is in it by minute five, and test whether a wallet's entry follows other wallets' entries, which is the automated-copy behaviour Ben described. Then fit with and without those columns on the same holdout so his hunch gets a measured yes or no.

# THE WALLET LAYER, AND THE LEAK IT UNCOVERED. 14 Sep 2026, `/opt/trench/lead.py`.
Ben's mechanism: early buyers are not one population. Some run their own screen (sophisticated, or gut), and some "check WHICH OTHER WALLETS ARE BUYING", some of them automatically. The existing wallet block counts who is in and how they have done; it could not tell an originator from a copier.

**The measure is entry ORDER, normalised.** For every token a wallet has been in, take its rank among that token's entrants as a fraction (0.0 = first in, 1.0 = last). The mean over many tokens is its lead score, the spread is its consistency. Written per wallet per DAY into `wallead`, computed only from tokens launched before that day, so a row frozen last Tuesday reads Tuesday's scores. 31,760 rows over 87 days, 5,165 wallets scored, 5.5s to rebuild. The distribution is a real bell with real tails: **52 wallets are essentially always first in, 42 always last.**

## THE LEAK, and it is the most dangerous one found in this project
The first test looked spectacular: at a fixed crowd size, tokens where originators outnumbered followers reached $100k 22.1% of the time against 2.0%, an 11x. It was wrong.
The tell was a bucket that was arithmetically impossible: inside a 300-second window, tokens where the first buy came more than **600 seconds** before the median buy reached $1M 66.7% of the time and **never died**. That can only happen if some entry timestamps sit BEFORE the recorded launch. They do.

| roster rows with an entry before the recorded launch | tokens | reached $1M | died |
|---|---|---|---|
| none | 663 | **2.6%** | 387 |
| at least one | 156 | **53.8%** | **0** |

**A recorded `created_ts` that postdates real trading means the token MIGRATED, and only tokens that graduate migrate.** So "has pre-launch entries" is very nearly "already succeeded", and any feature measured against `created_ts` on those rows is reading the outcome rather than predicting it. 6.2% of roster rows and **19% of tokens** are affected.
Fixed in patch79: pre-launch entries are dropped, and the cascade measure is bounded by the width of its own window so it can never encode how long the token lived.

## AND THEN THE HONEST RESULT: HIS HUNCH DID NOT SURVIVE THE CONTROL
With the leak removed, at the same crowd size (4 to 12 scored wallets in by minute five):

| composition | tokens | reached $100k |
|---|---|---|
| originators outnumber followers | 114 | 8.8% |
| followers outnumber originators | 52 | 3.8% |
| no scored originator at all | 8 | 12.5% |

The third row being the highest is the giveaway: at this sample size this is noise, not signal. The 11x was the migration tell wearing a wallet costume. **So LEAD_FEATS was NOT wired into the model.** Shipping twelve unvalidated columns would dilute every fit for the sake of a story.
This is not a refutation either: `ld_mean` still separates the $1M band (21.2% under 0.35 against about 1% above) without crowd size controlled, the control group is only 229 tokens, and lead scores need 3+ tokens so they cover 5,165 of 30,297 wallets. **The test is not yet powerful enough to confirm or deny.** The layer stays, keeps accumulating, and gets retested when the sample is bigger.

## WHAT THIS OPENS UP, and it is bigger than the wallet question
`created_ts` is wrong on roughly a fifth of tokens, and wrong in a direction that correlates with winning. `launch_of()` feeds `age_s` and every per-minute rate in FEATS, plus the reconstruction cutoffs. **Every one of those may be inflated on exactly the tokens that ran.** That is the next thing to check, and it outranks adding another feature.