[11:03:38] Config: grad_budgets=[36, 72] seeds=[0, 1, 2] n_images=128 eps=0.03137 quick=False [11:03:38] Loading models... Using cache found in ~/.cache/torch/hub/chenyaofo_pytorch-cifar-models_master Using cache found in ~/.cache/torch/hub/chenyaofo_pytorch-cifar-models_master Using cache found in ~/.cache/torch/hub/chenyaofo_pytorch-cifar-models_master Using cache found in ~/.cache/torch/hub/chenyaofo_pytorch-cifar-models_master Using cache found in ~/.cache/torch/hub/chenyaofo_pytorch-cifar-models_master [11:03:38] Loaded surrogate=cifar10_resnet20, targets=['cifar10_vgg11_bn', 'cifar10_mobilenetv2_x1_0', 'cifar10_shufflenetv2_x1_0', 'cifar10_repvgg_a0'] [11:03:38] Loading CIFAR-10 test set from data/cifar10_test.parquet ... [11:03:39] Pool size: 10000 [11:03:39] === grad_budget=36 seed=0 === [11:03:46] seed=0: scanned 256 candidates to find 128 images correct on all 5 models [11:03:46] selected batch: x0=(128, 3, 32, 32) y0=(128,) [11:03:50] arm=baseline-mi-equal-steps T= 6 grad_calls= 6 whitebox=1.000 transfer_mean=0.645 ngn=57.8835 per_target={'cifar10_vgg11_bn': 0.25, 'cifar10_mobilenetv2_x1_0': 0.852, 'cifar10_shufflenetv2_x1_0': 0.711, 'cifar10_repvgg_a0': 0.766} time=4.3s [11:04:00] arm=baseline-mi-equal-budget T= 36 grad_calls= 36 whitebox=1.000 transfer_mean=0.656 ngn=59.6769 per_target={'cifar10_vgg11_bn': 0.227, 'cifar10_mobilenetv2_x1_0': 0.875, 'cifar10_shufflenetv2_x1_0': 0.711, 'cifar10_repvgg_a0': 0.812} time=10.1s [11:04:10] arm=pgn-flat-maxima T= 6 grad_calls= 36 whitebox=1.000 transfer_mean=0.709 ngn=76.6313 per_target={'cifar10_vgg11_bn': 0.305, 'cifar10_mobilenetv2_x1_0': 0.867, 'cifar10_shufflenetv2_x1_0': 0.773, 'cifar10_repvgg_a0': 0.891} time=10.2s [11:04:10] === grad_budget=36 seed=1 === [11:04:16] seed=1: scanned 256 candidates to find 128 images correct on all 5 models [11:04:16] selected batch: x0=(128, 3, 32, 32) y0=(128,) [11:04:20] arm=baseline-mi-equal-steps T= 6 grad_calls= 6 whitebox=1.000 transfer_mean=0.656 ngn=57.8672 per_target={'cifar10_vgg11_bn': 0.305, 'cifar10_mobilenetv2_x1_0': 0.82, 'cifar10_shufflenetv2_x1_0': 0.695, 'cifar10_repvgg_a0': 0.805} time=4.0s [11:04:30] arm=baseline-mi-equal-budget T= 36 grad_calls= 36 whitebox=1.000 transfer_mean=0.664 ngn=61.9380 per_target={'cifar10_vgg11_bn': 0.242, 'cifar10_mobilenetv2_x1_0': 0.844, 'cifar10_shufflenetv2_x1_0': 0.734, 'cifar10_repvgg_a0': 0.836} time=10.2s [11:04:40] arm=pgn-flat-maxima T= 6 grad_calls= 36 whitebox=0.969 transfer_mean=0.607 ngn=77.9410 per_target={'cifar10_vgg11_bn': 0.234, 'cifar10_mobilenetv2_x1_0': 0.781, 'cifar10_shufflenetv2_x1_0': 0.656, 'cifar10_repvgg_a0': 0.758} time=10.3s [11:04:40] === grad_budget=36 seed=2 === [11:04:46] seed=2: scanned 256 candidates to find 128 images correct on all 5 models [11:04:46] selected batch: x0=(128, 3, 32, 32) y0=(128,) [11:04:51] arm=baseline-mi-equal-steps T= 6 grad_calls= 6 whitebox=1.000 transfer_mean=0.672 ngn=59.8841 per_target={'cifar10_vgg11_bn': 0.289, 'cifar10_mobilenetv2_x1_0': 0.828, 'cifar10_shufflenetv2_x1_0': 0.742, 'cifar10_repvgg_a0': 0.828} time=5.0s [11:05:01] arm=baseline-mi-equal-budget T= 36 grad_calls= 36 whitebox=1.000 transfer_mean=0.664 ngn=62.2919 per_target={'cifar10_vgg11_bn': 0.242, 'cifar10_mobilenetv2_x1_0': 0.836, 'cifar10_shufflenetv2_x1_0': 0.734, 'cifar10_repvgg_a0': 0.844} time=10.1s [11:05:11] arm=pgn-flat-maxima T= 6 grad_calls= 36 whitebox=0.977 transfer_mean=0.650 ngn=78.5863 per_target={'cifar10_vgg11_bn': 0.266, 'cifar10_mobilenetv2_x1_0': 0.82, 'cifar10_shufflenetv2_x1_0': 0.719, 'cifar10_repvgg_a0': 0.797} time=10.3s [11:05:11] === grad_budget=72 seed=0 === [11:05:17] seed=0: scanned 256 candidates to find 128 images correct on all 5 models [11:05:17] selected batch: x0=(128, 3, 32, 32) y0=(128,) [11:05:23] arm=baseline-mi-equal-steps T= 12 grad_calls= 12 whitebox=1.000 transfer_mean=0.664 ngn=55.6146 per_target={'cifar10_vgg11_bn': 0.234, 'cifar10_mobilenetv2_x1_0': 0.883, 'cifar10_shufflenetv2_x1_0': 0.742, 'cifar10_repvgg_a0': 0.797} time=5.6s [11:05:40] arm=baseline-mi-equal-budget T= 72 grad_calls= 72 whitebox=1.000 transfer_mean=0.660 ngn=60.9987 per_target={'cifar10_vgg11_bn': 0.211, 'cifar10_mobilenetv2_x1_0': 0.867, 'cifar10_shufflenetv2_x1_0': 0.742, 'cifar10_repvgg_a0': 0.82} time=17.6s [11:05:59] arm=pgn-flat-maxima T= 12 grad_calls= 72 whitebox=0.992 transfer_mean=0.764 ngn=69.8639 per_target={'cifar10_vgg11_bn': 0.375, 'cifar10_mobilenetv2_x1_0': 0.938, 'cifar10_shufflenetv2_x1_0': 0.844, 'cifar10_repvgg_a0': 0.898} time=18.9s [11:05:59] === grad_budget=72 seed=1 === [11:06:05] seed=1: scanned 256 candidates to find 128 images correct on all 5 models [11:06:05] selected batch: x0=(128, 3, 32, 32) y0=(128,) [11:06:10] arm=baseline-mi-equal-steps T= 12 grad_calls= 12 whitebox=1.000 transfer_mean=0.660 ngn=57.2245 per_target={'cifar10_vgg11_bn': 0.289, 'cifar10_mobilenetv2_x1_0': 0.844, 'cifar10_shufflenetv2_x1_0': 0.711, 'cifar10_repvgg_a0': 0.797} time=5.5s [11:06:28] arm=baseline-mi-equal-budget T= 72 grad_calls= 72 whitebox=1.000 transfer_mean=0.656 ngn=62.1788 per_target={'cifar10_vgg11_bn': 0.242, 'cifar10_mobilenetv2_x1_0': 0.844, 'cifar10_shufflenetv2_x1_0': 0.719, 'cifar10_repvgg_a0': 0.82} time=17.5s [11:06:45] arm=pgn-flat-maxima T= 12 grad_calls= 72 whitebox=0.992 transfer_mean=0.703 ngn=71.7306 per_target={'cifar10_vgg11_bn': 0.297, 'cifar10_mobilenetv2_x1_0': 0.906, 'cifar10_shufflenetv2_x1_0': 0.734, 'cifar10_repvgg_a0': 0.875} time=17.8s [11:06:45] === grad_budget=72 seed=2 === [11:06:51] seed=2: scanned 256 candidates to find 128 images correct on all 5 models [11:06:51] selected batch: x0=(128, 3, 32, 32) y0=(128,) [11:06:57] arm=baseline-mi-equal-steps T= 12 grad_calls= 12 whitebox=1.000 transfer_mean=0.662 ngn=58.7981 per_target={'cifar10_vgg11_bn': 0.266, 'cifar10_mobilenetv2_x1_0': 0.828, 'cifar10_shufflenetv2_x1_0': 0.711, 'cifar10_repvgg_a0': 0.844} time=5.7s [11:07:14] arm=baseline-mi-equal-budget T= 72 grad_calls= 72 whitebox=1.000 transfer_mean=0.648 ngn=65.2867 per_target={'cifar10_vgg11_bn': 0.219, 'cifar10_mobilenetv2_x1_0': 0.852, 'cifar10_shufflenetv2_x1_0': 0.695, 'cifar10_repvgg_a0': 0.828} time=17.8s [11:07:32] arm=pgn-flat-maxima T= 12 grad_calls= 72 whitebox=0.992 transfer_mean=0.723 ngn=71.4982 per_target={'cifar10_vgg11_bn': 0.305, 'cifar10_mobilenetv2_x1_0': 0.875, 'cifar10_shufflenetv2_x1_0': 0.812, 'cifar10_repvgg_a0': 0.898} time=17.4s [11:07:32] Falsification check: Part A premise failed: equal-steps advantage mean=3.29pp < 5pp, so the literature-bias premise the prediction is built on did not even replicate here. [11:07:32] Wrote results.json [11:07:33] Wrote figs/transfer_and_flatness.png [11:07:33] Done in 234.1s. outcome=inconclusive