---
name: pricepredict-data
description: The chain data layer this session owns (/opt/hist day DBs + /opt/pricepredict-data): what exists, the 5 Oct 2026 data-fix brief build order, what is done and what is next. Read before touching any RH chain history, bars, supply, pools or save-now collector.
sources: [cowork]
aliases: [pp data, data fix brief, birth table, savenow, venuepick, all venues]
---
# Price Predict data layer (owner: launch / Price Predict session)

- [stated] Ben 5 Oct 2026: this session owns the database; every data collection input comes here because later buying processes build on it, even when not relevant to the launch snipe.
- [stated] Brief: http://159.223.88.168/hx-c9ois/hx/pp-datafix-brief.md (v2, 16:45 BKK). Build order: birth table, all pools all venues (prove on COUPONS), USD per minute per quote, token minute bars 0..14 d in USD all pools, 72 h wallet tape for $100k+ coins, backfill to 3 Jul, save-now collectors. Hands off picker code paths; heavy jobs nice/ionice.

## Done 5 Oct 2026
- ETH price: ethusd.py had a units bug (MAX(t) seconds used as ms), dead since 3 Oct 20:16 BKK; fixed, 3 Binance hosts, ethusd.timer (1 min) + cron (2 min) under flock, sentinel C10 kick + alert at 30 min.
- Supply: /opt/pricepredict-data/supply.py (Multicall3 aggregate3, 150 tokens x 2 calls) filled 90,614 day-DB rows; NULL supply 0 of 127,214 (was 71%). Table tokens.db supply.
- COUPONS 0xf086c53f2780e601f0d005fabb3ee5a220336062 trades in Uniswap v2 pool 0x13bc28ec6a29b1843872750d9af7161402dc25bb (GeckoTerminal), which no collector read.
- venuepick.service (/opt/hist/venuepick.py, venues.db): live paper picker for v3 PoolCreated, v2 PairCreated, Pons V2 curve (factory 0xe33e9e479df8802cb0866d5d05258bec4cf62948, launch topic 0xdcacba5e..., buy 0xec36bf57... [ethIn, tokensOut, fee, fee], sell 0x8113d738... [tokensIn, ethOut, fee, fee]) and repools (token first seen > 24 h earlier, incl. v4); books vx_all (blind buy every launch at 30 s), vx_repool, vx_<model>; exit sell v1 with real liquidity (v2 Sync reserves, v3 liquidity()==0 = pulled). Sentinel C9 restart + alert. First 10 min: v2 47, pons 18, v4_repool 6, v3 1.
- GeckoTerminal top-200 pools by 24 h volume: v4 97 pools $341M, v3 59 pools $337M (26 with $1M+ FDV), Ramses v3, Giga v3, Up v3, Alandale CL, Pons V2 dex (v4 pools in the canonical PoolManager), Bankr (v4).
- savenow.py (timer 1 min + cron 3 min, flock, sentinel C11): rhtrenches.com public API (fomo trader tape /api/tape, /api/tokens, /api/traders) and X posts every 15 min for board coins at $100k+ (caps 3,000 searches and 20,000 posts a day, about $3). fomo trending needs a login (prod-api 431), not collected.
- Mac feed watcher self-heal + coverage check (see trading-edge-hub-5).
- Public RPC limits: topic-only getLogs 30,000 blocks, address-filtered 10,000,000 blocks; Alchemy free = 10 blocks; dRPC = 100 blocks. Chain block 1 = 30 Apr 2026.

- poolscan done 5 Oct: 2,212,688 unique pools since block 0 (v4 965,929, v3 453,952, pons 440,757, v2 352,050); 1,674,136 tokens with a birth row (tokens.db). COUPONS birth 15 Sep 10:30 BKK on its v2 pool, 7 pools; WORM birth 9 Sep 01:56 BKK on Pons (runtrack agrees, day file said 17 Sep).
- birth3 fix: token side = the coin with fewer distinct pairs (v1 dropped pools where both sides had 25+ pairs, so MEME/AI-type quote coins lost their own pools and birth).
- venuepick fix: feature code keyed buyers on the v4 PoolManager address; for other venues the pool is now the hub (v2 had 0 of 130 coins with model features before).
- venuepick first hours: v3 blind buys 55 of 59 at -100% (liquidity() 0 and ~0 WETH left in the pool = pulled, verified on chain); v2 -13% avg, pons -14% avg.
- fomo.family needs a logged-in bearer token for every prod-api call (431 without); its token lists come via /proxy/filterTokens (defined.fi / Codex). Not collected: copying Ben's session token to a server is not acceptable and it expires.
- chainswaps (Mac ~/ppdata/cs, 12 shards, launchd com.kookwatch.csguard every 5 min restarts missing shards): every swap / liquidity / sync log of every pool, 15 Sep to now; public RPC caps getLogs at 10,000 results ("exceeds limit" error, v4 alone ~5 swaps per block in Sep), so windows start at 800 blocks.

- birth4.py (full rebuild, CREATE TABLE AS, ~9 min) replaced the in-place birth3 UPDATE (too slow on this disk). Result: all 2.21M pools have a token side; 1,677,395 birth rows; MEME birth 4 Sep 03:26 BKK on v4 (688 pairs, it is also a quote), TIBBIR first RH pool 30 Sep 01:15 BKK.
- Increments: Mac launchd com.kookwatch.pooltail (every 15 min, poolscan.py tail, ships poolscan.tail.jsonl) + droplet ppinc.timer hourly + cron :41 (flock) run birth_inc.py then supply.py; sentinel C12 kicks at 3 h, alerts at 8 h. supply.py rerun filled 8,528 new day-DB rows (rh_rebuild adds days without supply).
- Two full checks 5 Oct ~16:20 BKK: services up, ETH 4,320 of 4,320 minutes in 3 days, no NULL supply, healthall ALL OK, no failing sentinel checks; alert BACK lines only after a real DOWN (_gback).
- Gotcha: pkill/pgrep -f with a pattern that appears in your own ssh command kills that ssh (hit twice); anchor patterns with ^.

- [stated] Ben 5 Oct: backfill daily RH swaps 30 Apr..15 Sep (traders, volume, buys vs sells), ETH bridged in per day via L1 bridge 0xDf8755334ce7A73cCF6b581C02eA649AE3E864b3, stablecoins on RH per day since launch; fomo trending every minute (pop right after it lands on trending, maybe the pop to sell into).
- chainday.db (/opt/pricepredict-data): bridge_eth (bridgehist.py, eth.blockscout.com balance history, cron 03:17 UTC daily), stables_day (Mac stablescan.py mint/burn rebuild, launchd com.kookwatch.stablesday daily, then stablesday.py). Swaps/traders per day: Mac shards e1..e8 (chainswaps 0..63M blocks) + x1..x8 (xferscan.py: v4 PoolManager token transfers in/out + v2/v3 swap recipients), guards com.kookwatch.csguard / xsguard with MAXRUN 14 workers (public RPC throttles the Mac IP; more workers only add 429 back-off). Aggregator still to write.
- fomotrend (Mac ~/fomotrend, Playwright venv, own profile): `fomotrend.py login` once (Ben), then `run` records WebSocket frames mentioning trending and ships to /opt/pricepredict-data/fomotrend/. FomoScan trending (free) also saved in savenow.db fomo_trending every minute but its snapshot is hours old.
- honest board: /opt/hist/honestboard.py (top of scoreboard.txt) from ppexit.service (/opt/hist/ppexit, copy of sellsys v1 engine, seeded with sell.db), sentinel C13.
- gate5.py in aipick5 (kill list, bundle_flip, rand_30s, _pullable/_fam/_creator in scored feats), serial_creators.json (rebuild with `python3 gate5.py`).

## Built 5 Oct evening (paid key)
- liqlock.db: whole-chain v4 Initialize/ModifyLiquidity/LP-NFT scan; tables v4init, ml, nft, hookfp, poolfam, liq, lock, bundle_sellers, btime (build liqbuild2.py, set-based SQL). Status doc for other sessions: http://159.223.88.168/hx-c9ois/hx/pp-data-status.md
- crew.db (crewmap.py): crewlive.service maps every new v4 deployer (who funded whom, stop at contracts / nonce >2,000); backfill from block 68,958,876 (21 Sep) via cron every 10 min under flock, paused by pickguard priority mode; sentinel C15.
- pons.db (ponstape.py): Pons curve launch/buy/sell from 13 Sep + graduations (v4 Initialize with hook 0xe5e702641ea86f4ae6cc3cdaed2b886f976be044).
- chainlogs paused by /tmp/cl_pause while heavy builds run (auto-resumes after 3 h).
- Hook fingerprint convention = code length + md5 of the "0x..." hex string (gate5 and knowbt fixed 5 Oct 18:50 BKK).

- [stated] Ben 5 Oct ~21:00 BKK: crew map must be done by the next morning; it runs whenever the live picker allows, the other overnight jobs pushed through too.
- nightjobs.sh (cron every 5 min): order liqlock build > crew backfill (nice 5) > ponstape > knowbt replays > crewgrade.py (writes crew_grades.json, checks pond-crews claims) > chainlogs (kept paused via /tmp/cl_pause until liq + crew done). Log /opt/pricepredict-data/nightjobs.log.
- pickguard fix 5 Oct 20:52 BKK: the throughput check counted a picker's first stats window after a restart (pools == inits, warm-up) as lag, so aipick5b restarts kept priority mode (all research paused) on from 20:18 to 20:56; warm-up windows are now skipped.
- gate5 logs the crew root and grade of each launch (_crew, _crew_grade in scored feats; grades "new" until crew_grades.json exists). Logging only, no buy change.
- LP lockers named by the sell session: 0xeff166aaf189323c58dc27ed1206eb2c37faacdf holds 55,285 position NFTs, no owner(); 0x267444d099b10fb5ed7c3cc7b7c767adca574952 holds 10,059 and is Ownable (owner 0x263ed295dafae1d9aadd6e56c4b6f9f38ee019dd, transferOwnership present); 0x35c5c1037eef9eb07b2826d537e4fdae61878852 holds 35. Names not verified (no name()).

## Monitoring layout for the 5 Oct builds (Ben 5 Oct ~21:10 BKK: redundant monitoring and self-fixing for everything new)
- Layer 1 sentinel (cron every minute + timer): C15 crewlive, C16 nightjobs.ok, C17 ponstape live (after backfill), C18 liqinc inc_t, C19 pdguard.ok; C14 fomo parked (PARKED file).
- Layer 2 /opt/pricepredict-data/pdguard.py (cron 5 min): re-adds missing cron lines (nightjobs, cl_run, liqinc, pdguard), restarts crewlive/ponstape/ppexit/venuepick, runs nightjobs/liqinc/cl_run when stale, disk under 8 GB pauses chainlogs, checks sentinel still has C15..C18; writes pdguard.ok.
- Layer 3 guardkit (other session, cron 15 min) restores the pdguard and nightjobs cron lines and keeps crewlive.service enabled/active. healthall.py lists the new services, heartbeats and crons.
- liqinc.py (cron :23 hourly) keeps liqlock.db current (raw scan + poolfam + liq + lock), skips while liqbuild2 runs.
- GOTCHA 5 Oct: aipick5.service runs /opt/hist/aipick5.py, which mk_aipick5.py BUILDS from aipick.py head + aipick5_main.py; patches to aipick5_main.py are not live until `python3 mk_aipick5.py` and a copy to /opt/hist/standby/aipick5.py (aipick5b). rand_gated and crew tags were dead 17:35 to 21:04 BKK because of this.
- pickguard cron now under timeout 120 and pickguard.service TimeoutStartSec=120 (one cron run hung 9 min under load); sentinel cron under flock + timeout 300. NONESSENTIAL (paused in priority mode) now has liqbuild2.py and NOT crewmap backfill (light, network bound, must finish by morning).
- pp-wash-brief v1 (http://159.223.88.168/hx-c9ois/brx/pp-wash-brief.md): /opt/hist/wash.py builds wash.db (sw swaps with trader wallet = end of token transfer chain in the tx, coin, rep repeat wallets, crew_day), crew_wash.json, public http://159.223.88.168/hx-c9ois/brx/crew-wash.md and analysis/wash_rescore.txt (section 6 + 7). Runs from nightjobs after crew + liq done, then daily 06:31 BKK with crewgrade. gate5.lane() tags every scored launch _wash/_lane/_max_eth (logging only, no buy change until Ben approves).

- Hourly learners (5 Oct ~21:45 BKK): /opt/hist/hourtrain.py cron 50 */2 (shadow aimodel5h.pkl, offline cache analysis/hourtrain_off.npz, labels analysis/hourtrain.db, log analysis/hourtrain.log, heartbeat hourtrain.ok; needs scored feats `_b` logged since ~20:40 BKK); gate5.py serial list cron :12 hourly (analysis/serial.log); xsafe mine cron :13 hourly. Watched by sentinel C20, pdguard, guardkit, healthall. Crontab backup /root/crontab.bak-hourly.

- 5 Oct 22:10 BKK: wash.py and crewgrade.py read pulls from liqlock ml (same dl, indexed ml_pool) instead of the derived liq table, and nightjobs runs them after crew_done only, because liqbuild2 sat 2.5 h+ in the liq/owner_at stage. Backups wash.py.bak-liq, crewgrade.py.bak-liq, /root/nightjobs.sh.bak-liq.

- AI spend (5 Oct 23:45 BKK): [stated] Ben recharged the Anthropic API several times and it keeps running out; wants it cheaper without losing quality. Measured: radar3 Opus judge ~$0.036/coin (47k cached system, 2.6k in, 321 out, ~2k cache write avg), deepread Opus card ~$0.026/call but paid 3 times per coin (pass 1, 15 min, 2 h) and logged only the last pass. Fix deployed: deepread fp_skip (passes 2/3 pay only if the coin's own site/app/GitHub/code/X input changed), spend log analysis/aispend.log. Pending once credits are back: 1 h cache TTL test for radar (ttltest.py in /tmp), Sonnet vs Opus agreement test on a sample. Backup deepread.py.bak-cost.

- [stated] Ben 5 Oct ~23:50 BKK: no rush on the project read (site/X bio do not change in 15 min), goal is enough categorised coins to test if it predicts anything; agreed to a cheaper model tested on a sample. Deployed: deepread writes every card with Sonnet, 1 coin in 10 also read by Opus into deep.db table cmp (agreement test); radar3 AI judge off while /opt/laneradar/state/JUDGE_OFF exists (free fast label stays); judgeguard watches deepread api_refused instead (alert key judge_api_refused); lanecheck skips judge retries/alerts with the flag. Backups *.bak-cheap. Expected ~$12/day. Next: Sonnet vs Opus agreement after ~200 cmp rows; Batch API (50% off) since speed does not matter.

- [stated] Ben 5 Oct ~00:00 BKK: many projects are duplicates sharing socials, never pay twice, copy the categorisation; novelty must be judged like a trench veteran who knows existing projects and the accepted limits of each category (ladder: commodity, something new but nothing crazy, interesting/different, completely new and likely huge). Deployed in deepread: keymap table (every site host or launchpad page path + X handle of a judged coin -> its card; any shared key copies the card and nov row, no AI call; launchpad hosts keyed by full page so different coins on one launchpad are not merged, fixing earlier wrong copies on stonkfun.xyz/icetea.fun/agencypad keys); SYSTEM now carries the trench-veteran brief + radar novelty rubric (judge_v3_system.md from "# NOVELTY RUBRIC") and new JSON keys novelty_level 0-4, novelty_class, novelty_score, limit_removed, gap, novelty_why saved to deep.db table nov. Backup deepread.py.bak-nov.

- [stated] Ben 6 Oct ~00:15 BKK: novelty must be judged across all crypto product categories (not just trading): build the category tree from live projects, per subcategory map the normal feature set vs what the leaders do differently, novel = beyond that. Also wants a Robinhood first-mover read: novel for RH but not for crypto is still interesting when it is the first RH project in an important subcategory whose leader elsewhere (Solana, Base, Ethereum) is big.

- [stated] Ben 6 Oct ~00:20 BKK, worked example of how locked-in traders pick early: PAID / UsePaid (Solana, 98kfF7rmsg1QDUEoCqNE7g7M1FdrTt92TEp2CLzypump) was the first to use X Money and sent its trade fees to individual people on X instead of buybacks or holder airdrops; novel AND an attention engine (point money at celebrities or causes, they ask in public where it came from). Deployed: deepread asks for attention_engine 0-3 + attention_why with this example (deep.db nov.attn, nov.attn_why); first live cards 00:2x BKK fill novelty level/class and attention. Backup deepread.py.bak-attn.
- [stated] Ben 6 Oct: another project claims GitHub active 20+ days before launch predicts a coin reaching $1M-$2M; to retest (prelaunch-watch-v1 found no edge on ~215 name-matched coins at $500k/10x).

- [stated] Ben 6 Oct ~00:10 BKK: topped up $20 of Anthropic API credit. After top-up deepread worked through the outage backlog: 273 Sonnet reads $4.53 (avg $0.0166) + 31 Opus checks $1.24 in ~40 min, 391 copies + 101 skips free. Deployed hard daily AI cap: deepread over_cap() reads /opt/hist/analysis/ai_daily_cap (8 dollars per Bangkok day), cards past the cap get err budget_cap (no alert, no fallback). Backup deepread.py.bak-cap. pickguard: standby aipick5b throughput lag no longer arms priority mode (it kept research paused 22:41 to 00:00 BKK), backup pickguard.py.bak-5b. Crew map backfill done 23:40 BKK (roots set, 23,627 creators); wash.py + crewgrade + knowbt started 23:41.

- [stated] Ben 6 Oct ~00:50 BKK: a coin also needs a story holders can spin on X to sell it to new buyers ("is it spinnable?"). deepread now scores four separate things per coin: novelty_level 0-4, significance 0-3 (real progress for sector experts), attention_engine 0-3 (mechanism makes noise by itself), spin 0-3 + pitch (the one tweet a holder would post). deep.db nov columns level, sig, attn, spin, pitch; live 00:5x BKK. Backup deepread.py.bak-spin.

## To do later (not priority)
- [stated] Ben 5 Oct 22:00 BKK: fomo trending parked, keep on the to-do list. Recorder works (POST /proxy/trendingTokens from the logged-in headless page, ~/fomotrend/trend.jsonl, fomopop.py cron, sentinel C14) but the list is Solana only; the "pop after trending" study waits.

## Running / next
- poolscan (Mac ~/ppdata, 8 shards, public RPC): every pool creation since block 0, then /opt/pricepredict-data/birth.py builds tokens.db pools / quotes (coin paired with 25+ tokens) / birth.
- Next: all-venue swap fetch for launch pools (address batched, Mac), token minute bars in USD, 72 h wallet tape, rh_rebuild continues to 3 Jul (14 Sep took ~70 min for stage A).
- [stated] Ben 5 Oct 2026: upgraded dRPC to a paid plan for research and a full history backfill; use it wisely like any limited API. Key stored on the droplet at /opt/common/drpc_paid.json (chmod 600, key "robinhood"); never print it. Tested 5 Oct: archive state works, getLogs accepts 100,000-block ranges and returns 16k+ results per call (free key: 100 blocks; public RPC: 10,000 results cap). Full backfill belongs to this data layer (Price Predict owns the database); other sessions use it only for small targeted pulls.