# NOTES — TCM stroke network-pharmacology null-model control

## What actually ran

`run.sh` → `run_experiment.py` (raw per-threshold/seed computation, ~157s) →
`summarize.py` (schema-compliant `results.json`) → `make_figures.py` (2 PNGs).
Full sweep: 3 STRING score thresholds (400/700/900) × 3 seeds × 1000 null
repetitions each = 9000 null-arm evaluations, in 157s total on 2 CPU threads.
The plan's 45-minute budget was never under pressure — data download + full
sweep + figures took under 3 minutes end to end.

## Data reality checks (before writing any pipeline code)

- Confirmed all 4 URLs return HTTP 200 and file sizes matching the plan
  before downloading (`data/` dir, ~54MB total).
- CTD gene-disease filter on the 5 stroke MeSH IDs gave exactly **131**
  unique genes — matches the plan's stated number, good sign the MeSH IDs
  were the intended ones.
- Two of the 20 named compounds in the plan have **no human-organism
  interaction record in CTD**: Astragaloside IV (黄芪甲苷) and plain
  paeoniflorin (芍药苷). Substituted with the closest same-family compound
  that does have human records (astragaloside A, benzoylpaeoniflorin).
  Logged in `results.json.deviations`.
- The plan's two null-sampling pool sizes (">=20 targets: 1986", ">=50
  targets: 1045") matched exactly what I computed from the real CTD data
  (1986 and 1045) — confirms the plan's stratified-sampling design was
  already grounded in the actual data, not a guess. Used it as specified:
  real compounds with ≥50 CTD targets get null replacements drawn from the
  ≥50-target pool, everything else (including the 6 real compounds with
  <20 targets — the plan didn't specify a third tier) from the ≥20 pool.

## Design decisions not fully pinned down by the plan

- "STRING 子网" (STRING subnetwork) construction: operationalized as seed
  nodes (compound-target ∩ D_seed) plus their 1-hop neighbors in the
  STRING physical-interaction graph — this is the standard network-pharm
  "PPI subnetwork" construction (Cytoscape/CytoHubba-style), but it's not
  the only possible reading. Did not test 2-hop expansion; out of budget
  and not necessary once the primary result was unambiguous across all 3
  thresholds.
- "Degree" for ranking = degree **within the induced subnetwork**, not raw
  whole-network STRING degree (the induced-subnetwork version is what
  network-pharmacology hub analyses actually report).
- `proposed-null-corrected` arm's z-score background correction: for every
  baseline candidate gene, z = (raw degree − mean degree under 1000 null
  seed draws) / (std + 1e-9), re-ranked by z. This uses the *same*
  candidate-gene universe and D_hold labels as the baseline arm, so the
  three arms are directly comparable.

## What I found (real numbers, in `logs/run.log` / `results.json`)

At the primary STRING threshold (physical-link score ≥700):

| seed | baseline AUROC | null mean AUROC | empirical p |
|---|---|---|---|
| 0 | 0.554 | 0.519 | 0.173 |
| 1 | 0.674 | 0.683 | 0.619 |
| 2 | 0.659 | 0.614 | 0.183 |

0/3 seeds significant (need ≥2/3 with p<0.05 to refute). Same pattern held
at threshold 400 (p = 0.10, 0.41, 0.23) and 900 (p = 0.14, 0.43, 0.28) —
**no threshold, no seed** gave a significant result. Top-K gene overlap
between the real-compound baseline and the random-compound null is high
(median Jaccard 0.4–1.0 depending on seed/K) — the "core hub targets" the
standard pipeline surfaces are largely the same well-connected genes
regardless of which 20 compounds (real TCM actives or random CTD
compounds) you start from.

**Prediction outcome: confirmed** (the skeptical prediction — no
compound-specific signal beyond PPI degree bias — was NOT refuted).
I still flag `negative_result: true` in `results.json`, because the
substantive, practically-relevant finding is negative: the standard
degree-hub network-pharmacology workflow, run on this real TCM-stroke
setup, does not show detectable target specificity over a fair
degree-matched random-compound control. Confirming a skeptical prediction
about a method's specificity is a negative finding about that method, even
though "confirmed" is the technically correct `prediction_outcome` value.

The z-score null-correction (`proposed-null-corrected` arm) does not
rescue this — its AUROC is *lower* than the raw baseline in all 3 seeds
(0.538 vs 0.554, 0.619 vs 0.674, 0.613 vs 0.659). It does produce a very
different top-K gene list (median Jaccard vs null ≈0, by construction,
since it directly subtracts the null expectation) — so it changes *which*
genes get flagged as "hub" without improving *how well* those genes
predict held-out stroke genes. Reported honestly, not tuned away.

## Things I did not do (in `results.json.out_of_scope`, unchanged from plan)

Wet-lab validation, docking, OB/DL-based TCMSP screening, CTD inferred
(non-curated) gene-disease associations — all as scoped out in the
original plan, budget and data-access reasons unchanged.

## Self-check performed

- Every `mean` in `results.json` recomputed from `per_seed` and asserted
  equal to <1e-9 (see `summarize.py`'s assertions, ran clean).
- `headline.baseline_mean` / `proposed_mean` both traced back to arm
  entries (`baseline-degree-hub`, `null-random-compounds`).
- Every number in `results.json` has a corresponding line in
  `logs/run.log` (per-seed AUROC/p-values printed during the run).
- Re-ran the full pipeline via `run.sh` end-to-end (not just the dev
  iteration run) to confirm reproducibility — identical numbers both times
  (deterministic `numpy.random.default_rng` seeding from `(threshold,
  seed[, rep])`).