# NOTES — spatial-domain-smoothing-null-control

## What this ran

Full DLPFC 12-slice sweep: k_neighbors in {0,6,12,18,24} x clusterer in
{kmeans, gmm_full} x seed in {0,1,2} = 360 (sample,k,clusterer,seed) configs,
followed by 2 null-model arms (shuffle-expr, geometry-only) at the
sweep-selected best (k, clusterer), each x 3 seeds x 12 samples = 72 more runs.
Total wall time for `python spatial_domain_experiment.py`: 682.5s (~11.4 min),
well inside the 45-minute budget (data download ~8s, figures ~2s on top).

## Steps taken, in order

1. Verified all three data sources (h5 matrices, tissue_positions_list.txt,
   barcode_level_layer_map.tsv) were reachable with `curl -sI` before writing
   any code — all returned HTTP 200.
2. Installed `h5py` into the shared env with `uv pip install` (not present by
   default) and inspected one sample's h5 structure directly (`f.visititems`)
   to confirm it's the standard 10x CellRanger CSC layout
   (`matrix/{data,indices,indptr,shape}`, shape = [n_genes, n_cells]).
3. Downloaded all 12 samples' h5 + positions files in parallel (curl,
   4-way concurrency) — 8 seconds total, not a bottleneck.
4. Prototyped the full per-sample pipeline (load -> filter -> normalize ->
   HVG -> smooth -> PCA -> cluster) on one sample/config in a throwaway
   script to get real timing numbers (load ~0.9s, smoothing ~0.6s, PCA
   ~0.4s, kmeans ~0.3s, gmm_full ~0.5s) before committing to the full sweep.
   This confirmed the 360+72-run sweep would take ~7-10 min, comfortably
   inside budget.
5. Wrote `spatial_domain_experiment.py`, added a `QUICK_TEST=1` env-var
   smoke-test path (3 samples, 2 k values, 2 seeds) and ran it end-to-end
   (sweep -> best-k/clusterer selection -> null models -> Wilcoxon ->
   results.json) to catch integration bugs (there were none beyond the fix
   below) before running the real thing.
6. Ran the full `run.sh` in the foreground. It exceeded the tool's default
   600s timeout and was auto-moved to a background task by the harness
   (not something I initiated with `&`/`nohup`); I then blocked on
   `TaskOutput` until it actually finished (682.5s), so the run was still
   fully awaited in-session before any results were used.

## Design decisions / interpretation of the plan

- **Same top-2000 HVG genes for baseline and proposed.** The plan's own
  `baseline-nonspatial` arm description says it should share "完全相同的
  预处理/降维/聚类/种子" with the proposed arm, with smoothing as the only
  difference. Taken literally, that means HVG selection should happen once
  (on the unsmoothed log-normalized matrix) and be reused for both arms,
  rather than re-selecting top-2000 genes separately from smoothed vs.
  unsmoothed data (which is what a more literal reading of the smoothing
  arm's own description — "log-normalize -> smooth -> HVG -> PCA" — would
  imply in isolation). I went with the shared-HVG reading because (a) it's
  explicit in the baseline arm's own text, (b) it's the fairer controlled
  comparison (isolates smoothing as the only variable), and (c) it's far
  cheaper: smoothing on 2000 already-selected genes (dense, ~4200x2000) vs.
  smoothing the full >=10-count gene set (~15-16k genes, denses out to
  hundreds of MB per sample) — infeasible in the CPU/memory budget for a
  360-run sweep. Logged as `deviations[0]` in results.json.
- **best_k / best_clusterer selected once, globally**, by mean ARI over all
  12 samples x 3 seeds, restricted to k>0. Result: best_k=24, best_clusterer=
  gmm_full (mean ARI 0.348, vs. kmeans best 0.286 at k=24). Both null models
  and the reported "proposed-smooth-knn" / "baseline-nonspatial" arms use
  this single (k,clusterer) pair, so the baseline-vs-proposed comparison is
  apples-to-apples on clusterer type and isolates the smoothing effect.
- **alpha=0.5 fixed**, the optional alpha-robustness check on 151673 (listed
  as "若时间有余" in the plan) was skipped — budget was spent on the full
  12-slice x 2-clusterer x 5-k sweep instead, which the plan treats as the
  primary requirement. Logged as `deviations[1]`.
- **`ari_seed_std` metric is reported per-slice, not per-global-seed.** Its
  own definition ("同切片同参数下3个种子的ARI标准差") is inherently a
  per-slice statistic (one std value per DLPFC sample, computed across the
  3 seeds), so `per_seed` for this one metric lists the 12 per-slice values
  rather than 3 seed-aggregated values like the other metrics. Documented
  inline in `results.json` under that metric's `note` field.
- **P1/P2 → overall `prediction_outcome` bug caught and fixed after the
  first pass.** My first aggregation logic set `prediction_outcome` to
  `"inconclusive"` when P1 was refuted but P2 was confirmed. That's wrong:
  P1 (does smoothing reach useful absolute ARI) is the primary claim, and
  its own refute condition is binary per the pre-registered spec text ("若
  ...则P1被推翻" — no gray zone). A passing secondary robustness check (P2)
  doesn't rescue a failed primary claim. Fixed the decision rule so
  `prediction_outcome="refuted"` whenever P1 is refuted, regardless of P2,
  and patched the already-written `results.json` accordingly (only the
  derived categorical label changed — no measured number in the file was
  touched; `headline`, `arms`, `selection` are all untouched from the real
  run). Fixed the same logic in `spatial_domain_experiment.py` so a re-run
  reproduces the corrected label directly.

## Result summary (see results.json for full numbers)

- P1 (best-k smoothing baseline reaches mean ARI>=0.40, delta>=+0.10,
  paired Wilcoxon p<0.05): **refuted**. Delta (+0.141) and significance
  (p=0.00049) both cleared their bars, but absolute mean ARI (0.348) fell
  short of the 0.40 threshold.
- P2 (geometry-only null stays <=0.15 ARI): **confirmed**. Null-geometry
  mean ARI = 0.050 — replacing real expression with iid Gaussian noise and
  running the same smooth+cluster pipeline gets nowhere near real
  structure, so the ARI gain from smoothing is not a cheap geometric
  artifact.
- null-shuffle-expr mean ARI = 0.005 (near-zero, as expected): decoupling
  expression from position and then smoothing does not fabricate structure
  either.
- Improvement was consistent, not driven by outliers: proposed > baseline
  on all 12/12 slices (see `figs/per_sample_paired_ari.png`).

## Honest caveat

The 0.40 absolute-ARI threshold in P1 was a pre-registered, somewhat
arbitrary bar; the method clearly does something real (statistically
significant, consistent across all 12 slices, far above both null floors)
but a hand-crafted zero-parameter kNN smoother tops out around ARI=0.35 on
this benchmark with the true K given — short of what's needed to call it a
strong absolute result. No parameters, seeds, or subsets were adjusted after
seeing these numbers.