# Server rules for every team

## RULE 0 (Ben, 8 Oct 2026): ONLY BEN SWITCHES LIVE
- No session, script, cron, timer or deadline may turn live trading on, hand a wallet to another engine, or raise a live size. Not even with "GO given in advance".
- Allowed: switching live OFF (kill switches, stops). Proposing a change as a brief (link) that Ben reads and flips himself.
- Every live change sends Ben a Telegram message (livechange.py, every minute on firstmin and main). If he did not make it, it is a bug.
- All code is in Git (D-033). Live bot code and live configs change only with Ben's yes.

## Earlier rules (Ben's order, 7 Oct 2026)

## Read this first (7 Oct 10:45 Bangkok, from Price Predict)
- **What happened:** on 7 Oct around 10:15 Bangkok the main droplet 159.223.88.168 was badly overloaded: 2 CPU, 4 GB, load 32 to 41, swap almost full. Ben saw it.
- **Cause:** research jobs (model training, tuning, back tests, analyst scripts) running on the same small box as the live pickers and feeds.
- **Ben approved two things:**
  1. Main is resized to 4 CPU / 8 GB ($48 a month, done 7 Oct about 10:40 Bangkok; disk kept at 80 GB so it can be scaled back).
  2. **Research does not run on main any more.**
- **Ben also asked:** nobody runs code fetched from a public paste site again. One job on main was started like this: `curl https://dpaste.com/... | base64 -d | gunzip > /tmp/x.py && python3 /tmp/x.py`. That runs code nobody on the team has seen, as root, next to the live bots. It also breaks the moment the paste expires.

## Where things run
| Machine | What runs there |
|---|---|
| main 159.223.88.168 (4 CPU, 8 GB) | ONLY live services: live pickers and paper books, live data feeds, alert bots, the public folder (bus files). Small read-only lookups (one coin, one pool, under 10 s). |
| research-box 152.42.241.219 (8 CPU, 16 GB, 236 GB free) | ALL research: back tests, training, tuning, rebuilds, full-table scans, analyst scripts. |
| firstmin 178.128.20.136 | fmmc5 live trading only. Nobody runs anything else there. |

## How to run a research job (do it this way)
1. **Write the script into your own folder on research-box**, for example /opt/<team>/<role>/myjob.py. Copy it from your session with ssh (heredoc) or scp. Never pipe a URL into python.
2. **Read your inputs read-only:** `sqlite3.connect("file:<path>?mode=ro", uri=True)`. Use the copies on research-box. If a file you need is only on main, ask Price Predict for a copy; do not copy big files from main yourself between 09:00 and 01:00 Bangkok.
3. **Start it gently, with a lock and a time limit:**
   `cd /opt/<team>/<role> && nohup flock -n /tmp/<team>-myjob.lock timeout 3h nice -n 10 ionice -c3 python3 -u myjob.py > myjob.log 2>&1 < /dev/null &`
   - flock: never two copies at once (cron jobs that overrun piled up on main).
   - timeout: a hung job dies by itself.
   - nice + ionice: live work always comes first.
4. **Cron jobs** use the same line (flock + timeout + nice). Name the lock after your team and job.
5. **Publish results** to the public folder on main with one small copy at the end (scp of the CSV or md file), not by running the job on main.
6. **Memory:** keep one job under 4 GB. Stream rows; do not load a full day DB into pandas.

## Already moved by Price Predict
- hist-rh.service (day-file rebuild) stopped on main; it moves to research-box.
- wash.py, crewgrade, xsafe mine/canary move to research-box today.

- 8 Oct 2026 (PP Data, Ben order): also moved to research-box: day-file rebuild, daily picker retrain, hourtrain, coin brain training, wash + signer + crew grades, daily report, soltrain, metatest, runner lab. Main only schedules them (/opt/offload) and pulls the results.

## Asks per team (please confirm in your status file)
- Coin brain (/opt/brain train.py, nightly 02:45): move to research-box.
- xsafe tuning (/opt/common/xsafe): Price Predict moves it.
- /opt/hist/reconcile.py (every minute, 300 MB): owner please move or slow it to every 10 min.
- /opt/hist/feedcmp.py (every 10 min): owner please move.
- /tmp/a3_xfull.py (Analyst 3): rerun on research-box from your own folder, per steps 1 to 3.
- /opt/runtrack/rhledger.py (515 MB, live tracker): stays on main if it feeds a live alert; owner please check memory.

Version: v1, 7 Oct 2026 10:45 Bangkok, Price Predict. Latest: http://159.223.88.168/hx-c9ois/server-rules.md