diff --git a/docs/k2-aic-exhaustive-study/README.md b/docs/k2-aic-exhaustive-study/README.md new file mode 100644 index 000000000000..2b19912a543b --- /dev/null +++ b/docs/k2-aic-exhaustive-study/README.md @@ -0,0 +1,14 @@ +# K2 AIC Exhaustive Study + +This folder contains epic-scoped artifacts for RL360 epic `#126` only. + +- Narrative: [sections/126-inference-decision.md](./sections/126-inference-decision.md) +- Pairwise data: [data/126_inference_pairwise.tsv](./data/126_inference_pairwise.tsv) +- Profile summary: [data/126_inference_profile_summary.tsv](./data/126_inference_profile_summary.tsv) +- Decision table: [data/126_inference_decision_table.md](./data/126_inference_decision_table.md) +- Figures: + [126_inference_tok_s_user_scatter.png](./figures/126_inference_tok_s_user_scatter.png), + [126_inference_user_margin_vs_gpu_ratio.png](./figures/126_inference_user_margin_vs_gpu_ratio.png) + +These artifacts are self-contained and derived from the local additive-band AIConfigurator catalog. +Reproducibility scripts remain out of scope for this PR and are tracked separately by issue `#182`. diff --git a/docs/k2-aic-exhaustive-study/data/126_inference_decision_table.md b/docs/k2-aic-exhaustive-study/data/126_inference_decision_table.md new file mode 100644 index 000000000000..4dc0727bcfe4 --- /dev/null +++ b/docs/k2-aic-exhaustive-study/data/126_inference_decision_table.md @@ -0,0 +1,28 @@ +# Epic 126 Inference Decision Table + +- Source catalog: `/mnt/weka/home/micah.villmow/nvidia/dynamo/.aic_local_logs/k2moe375b_xllm_h200/aiconfigurator/20260427T172219Z_k2_additive_batch_band_full/catalog/selected_points.tsv` +- Load bands considered: `15` (`load_000001` .. `load_016384`) +- Observation: every profile/node/topology selection is invariant across the 15 load bands in this additive catalog, so the table below collapses to the unique inference frontier. + +| Profile | Context | Verdict | Disagg-winning nodes (`tok/s/user`) | Agg-winning nodes | Missing disagg nodes | User ratio range | GPU ratio range | Cluster ratio range | +|---|---:|---|---|---|---|---:|---:|---:| +| Balanced 128K (`balanced_128k`) | 131072 | disagg | 2,4,8,16,32,64,128,256,512 | - | 1 | 1.659..1.659 | 0.415..0.778 | 0.415..0.778 | +| Balanced 16K (`balanced_16k`) | 16384 | agg | - | 2,4,8,16,32,64,128,256,512 | 1 | 0.929..0.929 | 0.492..0.922 | 0.492..0.922 | +| Balanced 1K (`balanced_1k`) | 1024 | disagg | 2,4,8,16,32,64,128,256,512 | - | 1 | 1.007..1.007 | 0.495..0.743 | 0.495..0.743 | +| Balanced 256K (`balanced_256k`) | 262144 | agg | - | 2,4,8,16,32,64,128,256,512 | 1 | 0.540..0.540 | 0.662..1.242 | 0.662..1.242 | +| Balanced 2K (`balanced_2k`) | 2048 | agg | - | 2,4,8,16,32,64,128,256,512 | 1 | 0.996..0.996 | 0.489..0.856 | 0.489..0.856 | +| Balanced 32K (`balanced_32k`) | 32768 | agg | - | 2,4,8,16,32,64,128,256,512 | 1 | 0.923..0.923 | 0.467..0.875 | 0.467..0.875 | +| Balanced 4K (`balanced_4k`) | 4096 | mixed | 8 | 2,4,16,32,64,128,256,512 | 1 | 0.978..1.017 | 0.471..0.766 | 0.471..0.766 | +| Balanced 64K (`balanced_64k`) | 65536 | disagg | 2,4,8,16,32,64,128,256,512 | - | 1 | 2.140..2.140 | 0.415..0.777 | 0.415..0.777 | +| Balanced 8K (`balanced_8k`) | 8192 | agg | - | 2,4,8,16,32,64,128,256,512 | 1 | 0.939..0.960 | 0.459..0.847 | 0.459..0.839 | +| Chat 4K / 500 (`chat_4k_500`) | 4500 | disagg | 2,4,8,16,32,64,128,256,512 | - | 1 | 1.377..1.412 | 0.618..0.672 | 0.618..0.661 | +| 7K / 128K (`observed_high_7k_128k`) | 138240 | disagg | 2,4,8,16,32,64,128,256,512 | - | 1 | 2.661..2.661 | 0.384..0.720 | 0.384..0.720 | +| 7K / 1K (`observed_high_7k_1k`) | 8192 | disagg | 2,4,8,16,32,64,128,256,512 | - | 1 | 1.169..1.215 | 0.565..0.781 | 0.565..0.774 | +| 0.5K / 128K (`observed_low_0p5k_128k`) | 131584 | disagg | 2,4,8,16,32,64,128,256,512 | - | 1 | 3.495..3.495 | 0.362..0.679 | 0.362..0.679 | +| 0.5K / 1K (`observed_low_0p5k_1k`) | 1536 | agg | - | 2,4,8,16,32,64,128,256,512 | 1 | 0.890..0.942 | 0.500..0.889 | 0.500..0.875 | +| 1.5K / 128K (`observed_mean_1p5k_128k`) | 132608 | disagg | 2,4,8,16,32,64,128,256,512 | - | 1 | 3.498..3.498 | 0.363..0.681 | 0.363..0.681 | +| 1.5K / 1K (`observed_mean_1p5k_1k`) | 2560 | disagg | 2,4,8,16,32,64,128,256,512 | - | 1 | 1.028..1.065 | 0.485..0.728 | 0.485..0.721 | +| 32K / 8K (`practical_32k_8k`) | 40960 | agg | - | 2,4,8,16,32,64,128,256,512 | 1 | 0.944..0.944 | 0.489..0.916 | 0.489..0.916 | +| 64K / 8K (`practical_64k_8k`) | 73728 | agg | - | 2,4,8,16,32,64,128,256,512 | 1 | 0.831..0.831 | 0.469..0.862 | 0.469..0.853 | +| 128K profile (`stress_128k`) | 131072 | mixed | 16,32,64 | 2,4,8,128,256,512 | 1 | 0.745..1.338 | 0.485..0.901 | 0.485..0.887 | +| 256K profile (`stress_256k`) | 262144 | no_disagg_candidate | - | - | 1,2,4,8,16,32,64,128,256,512 | -..- | -..- | -..- | diff --git a/docs/k2-aic-exhaustive-study/data/126_inference_pairwise.tsv b/docs/k2-aic-exhaustive-study/data/126_inference_pairwise.tsv new file mode 100644 index 000000000000..36071263372d --- /dev/null +++ b/docs/k2-aic-exhaustive-study/data/126_inference_pairwise.tsv @@ -0,0 +1,201 @@ +workload_profile summary_label context_budget isl osl node_count agg_status disagg_status agg_tok_s_user disagg_tok_s_user disagg_over_agg_tok_s_user disagg_minus_agg_tok_s_user agg_tok_s_gpu disagg_tok_s_gpu disagg_over_agg_tok_s_gpu agg_cluster_tok_s disagg_cluster_tok_s disagg_over_agg_cluster_tok_s agg_total_gpus disagg_total_gpus winner_tok_s_user load_bands_considered agg_load_invariant disagg_load_invariant agg_config disagg_config +balanced_128k Balanced 128K 131072 65536 65536 1 candidate no_results 36.426 36.311 290.488 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=8 no result +balanced_128k Balanced 128K 131072 65536 65536 2 candidate candidate 36.426 60.439 1.659 24.013 36.311 15.073 0.415 580.976 241.168 0.415 16 16 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_128k Balanced 128K 131072 65536 65536 4 candidate candidate 36.426 60.439 1.659 24.013 36.311 22.610 0.623 1161.952 723.520 0.623 32 32 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_128k Balanced 128K 131072 65536 65536 8 candidate candidate 36.426 60.439 1.659 24.013 36.311 26.378 0.726 2323.904 1688.192 0.726 64 64 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_128k Balanced 128K 131072 65536 65536 16 candidate candidate 36.426 60.439 1.659 24.013 36.311 28.262 0.778 4647.808 3617.536 0.778 128 128 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_128k Balanced 128K 131072 65536 65536 32 candidate candidate 36.426 60.439 1.659 24.013 36.311 28.262 0.778 9295.616 7235.072 0.778 256 256 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_128k Balanced 128K 131072 65536 65536 64 candidate candidate 36.426 60.439 1.659 24.013 36.311 28.262 0.778 18591.232 14470.144 0.778 512 512 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_128k Balanced 128K 131072 65536 65536 128 candidate candidate 36.426 60.439 1.659 24.013 36.311 28.262 0.778 37182.464 28940.288 0.778 1024 1024 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 8p + 120d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_128k Balanced 128K 131072 65536 65536 256 candidate candidate 36.426 60.439 1.659 24.013 36.311 28.262 0.778 74364.928 57880.576 0.778 2048 2048 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 16p + 240d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_128k Balanced 128K 131072 65536 65536 512 candidate candidate 36.426 60.439 1.659 24.013 36.311 28.262 0.778 148729.856 115761.152 0.778 4096 4096 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 32p + 480d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_16k Balanced 16K 16384 8192 8192 1 candidate no_results 15.523 155.105 1240.840 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=80 no result +balanced_16k Balanced 16K 16384 8192 8192 2 candidate candidate 15.523 14.422 0.929 -1.101 155.105 76.308 0.492 2481.680 1220.928 0.492 16 16 agg 15 1 1 tp8pp1dp1etp1ep8, bs=80 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=92 +balanced_16k Balanced 16K 16384 8192 8192 4 candidate candidate 15.523 14.422 0.929 -1.101 155.105 114.463 0.738 4963.360 3662.816 0.738 32 32 agg 15 1 1 tp8pp1dp1etp1ep8, bs=80 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=92 +balanced_16k Balanced 16K 16384 8192 8192 8 candidate candidate 15.523 14.422 0.929 -1.101 155.105 133.540 0.861 9926.720 8546.560 0.861 64 64 agg 15 1 1 tp8pp1dp1etp1ep8, bs=80 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=92 +balanced_16k Balanced 16K 16384 8192 8192 16 candidate candidate 15.523 14.422 0.929 -1.101 155.105 143.078 0.922 19853.440 18313.984 0.922 128 128 agg 15 1 1 tp8pp1dp1etp1ep8, bs=80 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=92 +balanced_16k Balanced 16K 16384 8192 8192 32 candidate candidate 15.523 14.422 0.929 -1.101 155.105 143.078 0.922 39706.880 36627.968 0.922 256 256 agg 15 1 1 tp8pp1dp1etp1ep8, bs=80 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=92 +balanced_16k Balanced 16K 16384 8192 8192 64 candidate candidate 15.523 14.422 0.929 -1.101 155.105 143.078 0.922 79413.760 73255.936 0.922 512 512 agg 15 1 1 tp8pp1dp1etp1ep8, bs=80 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=92 +balanced_16k Balanced 16K 16384 8192 8192 128 candidate candidate 15.523 14.422 0.929 -1.101 155.105 143.078 0.922 158827.520 146511.872 0.922 1024 1024 agg 15 1 1 tp8pp1dp1etp1ep8, bs=80 8p + 120d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=92 +balanced_16k Balanced 16K 16384 8192 8192 256 candidate candidate 15.523 14.422 0.929 -1.101 155.105 143.078 0.922 317655.040 293023.744 0.922 2048 2048 agg 15 1 1 tp8pp1dp1etp1ep8, bs=80 16p + 240d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=92 +balanced_16k Balanced 16K 16384 8192 8192 512 candidate candidate 15.523 14.422 0.929 -1.101 155.105 143.078 0.922 635310.080 586047.488 0.922 4096 4096 agg 15 1 1 tp8pp1dp1etp1ep8, bs=80 32p + 480d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=92 +balanced_1k Balanced 1K 1024 512 512 1 candidate no_results 12.720 1622.193 12977.544 8 0 missing 15 1 1 tp1pp1dp8etp1ep8, bs=128 no result +balanced_1k Balanced 1K 1024 512 512 2 candidate candidate 12.720 12.811 1.007 0.091 1622.193 803.035 0.495 25955.088 12848.560 0.495 16 16 disagg 15 1 1 tp1pp1dp8etp1ep8, bs=128 1p + 1d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=136 +balanced_1k Balanced 1K 1024 512 512 4 candidate candidate 12.720 12.811 1.007 0.091 1622.193 1204.552 0.743 51910.176 38545.664 0.743 32 32 disagg 15 1 1 tp1pp1dp8etp1ep8, bs=128 1p + 3d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=136 +balanced_1k Balanced 1K 1024 512 512 8 candidate candidate 12.720 12.811 1.007 0.091 1622.193 1204.552 0.743 103820.352 77091.328 0.743 64 64 disagg 15 1 1 tp1pp1dp8etp1ep8, bs=128 2p + 6d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=136 +balanced_1k Balanced 1K 1024 512 512 16 candidate candidate 12.720 12.811 1.007 0.091 1622.193 1204.552 0.743 207640.704 154182.656 0.743 128 128 disagg 15 1 1 tp1pp1dp8etp1ep8, bs=128 4p + 12d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=136 +balanced_1k Balanced 1K 1024 512 512 32 candidate candidate 12.720 12.811 1.007 0.091 1622.193 1204.552 0.743 415281.408 308365.312 0.743 256 256 disagg 15 1 1 tp1pp1dp8etp1ep8, bs=128 8p + 24d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=136 +balanced_1k Balanced 1K 1024 512 512 64 candidate candidate 12.720 12.811 1.007 0.091 1622.193 1204.552 0.743 830562.816 616730.624 0.743 512 512 disagg 15 1 1 tp1pp1dp8etp1ep8, bs=128 16p + 48d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=136 +balanced_1k Balanced 1K 1024 512 512 128 candidate candidate 12.720 12.811 1.007 0.091 1622.193 1204.552 0.743 1661125.632 1233461.248 0.743 1024 1024 disagg 15 1 1 tp1pp1dp8etp1ep8, bs=128 32p + 96d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=136 +balanced_1k Balanced 1K 1024 512 512 256 candidate candidate 12.720 12.811 1.007 0.091 1622.193 1204.552 0.743 3322251.264 2466922.496 0.743 2048 2048 disagg 15 1 1 tp1pp1dp8etp1ep8, bs=128 64p + 192d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=136 +balanced_1k Balanced 1K 1024 512 512 512 candidate candidate 12.720 12.811 1.007 0.091 1622.193 1204.552 0.743 6644502.528 4933844.992 0.743 4096 4096 disagg 15 1 1 tp1pp1dp8etp1ep8, bs=128 128p + 384d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=136 +balanced_256k Balanced 256K 262144 131072 131072 1 candidate no_results 91.351 22.757 182.056 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=2 no result +balanced_256k Balanced 256K 262144 131072 131072 2 candidate candidate 91.351 49.321 0.540 -42.030 22.757 15.073 0.662 364.112 241.168 0.662 16 16 agg 15 1 1 tp8pp1dp1etp1ep8, bs=2 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_256k Balanced 256K 262144 131072 131072 4 candidate candidate 91.351 49.321 0.540 -42.030 22.757 22.610 0.994 728.224 723.520 0.994 32 32 agg 15 1 1 tp8pp1dp1etp1ep8, bs=2 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_256k Balanced 256K 262144 131072 131072 8 candidate candidate 91.351 49.321 0.540 -42.030 22.757 26.378 1.159 1456.448 1688.192 1.159 64 64 agg 15 1 1 tp8pp1dp1etp1ep8, bs=2 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_256k Balanced 256K 262144 131072 131072 16 candidate candidate 91.351 49.321 0.540 -42.030 22.757 28.262 1.242 2912.896 3617.536 1.242 128 128 agg 15 1 1 tp8pp1dp1etp1ep8, bs=2 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_256k Balanced 256K 262144 131072 131072 32 candidate candidate 91.351 49.321 0.540 -42.030 22.757 28.262 1.242 5825.792 7235.072 1.242 256 256 agg 15 1 1 tp8pp1dp1etp1ep8, bs=2 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_256k Balanced 256K 262144 131072 131072 64 candidate candidate 91.351 49.321 0.540 -42.030 22.757 28.262 1.242 11651.584 14470.144 1.242 512 512 agg 15 1 1 tp8pp1dp1etp1ep8, bs=2 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_256k Balanced 256K 262144 131072 131072 128 candidate candidate 91.351 49.321 0.540 -42.030 22.757 28.262 1.242 23303.168 28940.288 1.242 1024 1024 agg 15 1 1 tp8pp1dp1etp1ep8, bs=2 8p + 120d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_256k Balanced 256K 262144 131072 131072 256 candidate candidate 91.351 49.321 0.540 -42.030 22.757 28.262 1.242 46606.336 57880.576 1.242 2048 2048 agg 15 1 1 tp8pp1dp1etp1ep8, bs=2 16p + 240d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_256k Balanced 256K 262144 131072 131072 512 candidate candidate 91.351 49.321 0.540 -42.030 22.757 28.262 1.242 93212.672 115761.152 1.242 4096 4096 agg 15 1 1 tp8pp1dp1etp1ep8, bs=2 32p + 480d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +balanced_2k Balanced 2K 2048 1024 1024 1 candidate no_results 13.522 861.699 6893.592 8 0 missing 15 1 1 tp1pp1dp8etp1ep8, bs=64 no result +balanced_2k Balanced 2K 2048 1024 1024 2 candidate candidate 13.522 13.462 0.996 -0.060 861.699 421.522 0.489 13787.184 6744.352 0.489 16 16 agg 15 1 1 tp1pp1dp8etp1ep8, bs=64 1p + 1d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=68 +balanced_2k Balanced 2K 2048 1024 1024 4 candidate candidate 13.522 13.462 0.996 -0.060 861.699 632.283 0.734 27574.368 20233.056 0.734 32 32 agg 15 1 1 tp1pp1dp8etp1ep8, bs=64 1p + 3d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=68 +balanced_2k Balanced 2K 2048 1024 1024 8 candidate candidate 13.522 13.462 0.996 -0.060 861.699 737.663 0.856 55148.736 47210.432 0.856 64 64 agg 15 1 1 tp1pp1dp8etp1ep8, bs=64 1p + 7d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=68 +balanced_2k Balanced 2K 2048 1024 1024 16 candidate candidate 13.522 13.462 0.996 -0.060 861.699 737.663 0.856 110297.472 94420.864 0.856 128 128 agg 15 1 1 tp1pp1dp8etp1ep8, bs=64 2p + 14d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=68 +balanced_2k Balanced 2K 2048 1024 1024 32 candidate candidate 13.522 13.462 0.996 -0.060 861.699 737.663 0.856 220594.944 188841.728 0.856 256 256 agg 15 1 1 tp1pp1dp8etp1ep8, bs=64 4p + 28d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=68 +balanced_2k Balanced 2K 2048 1024 1024 64 candidate candidate 13.522 13.462 0.996 -0.060 861.699 737.663 0.856 441189.888 377683.456 0.856 512 512 agg 15 1 1 tp1pp1dp8etp1ep8, bs=64 8p + 56d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=68 +balanced_2k Balanced 2K 2048 1024 1024 128 candidate candidate 13.522 13.462 0.996 -0.060 861.699 737.663 0.856 882379.776 755366.912 0.856 1024 1024 agg 15 1 1 tp1pp1dp8etp1ep8, bs=64 16p + 112d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=68 +balanced_2k Balanced 2K 2048 1024 1024 256 candidate candidate 13.522 13.462 0.996 -0.060 861.699 737.663 0.856 1764759.552 1510733.824 0.856 2048 2048 agg 15 1 1 tp1pp1dp8etp1ep8, bs=64 32p + 224d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=68 +balanced_2k Balanced 2K 2048 1024 1024 512 candidate candidate 13.522 13.462 0.996 -0.060 861.699 737.663 0.856 3529519.104 3021467.648 0.856 4096 4096 agg 15 1 1 tp1pp1dp8etp1ep8, bs=64 64p + 448d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=68 +balanced_32k Balanced 32K 32768 16384 16384 1 candidate no_results 15.754 78.691 629.528 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=40 no result +balanced_32k Balanced 32K 32768 16384 16384 2 candidate candidate 15.754 14.544 0.923 -1.210 78.691 36.741 0.467 1259.056 587.856 0.467 16 16 agg 15 1 1 tp8pp1dp1etp1ep8, bs=40 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=44 +balanced_32k Balanced 32K 32768 16384 16384 4 candidate candidate 15.754 14.544 0.923 -1.210 78.691 55.112 0.700 2518.112 1763.584 0.700 32 32 agg 15 1 1 tp8pp1dp1etp1ep8, bs=40 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=44 +balanced_32k Balanced 32K 32768 16384 16384 8 candidate candidate 15.754 14.544 0.923 -1.210 78.691 64.297 0.817 5036.224 4115.008 0.817 64 64 agg 15 1 1 tp8pp1dp1etp1ep8, bs=40 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=44 +balanced_32k Balanced 32K 32768 16384 16384 16 candidate candidate 15.754 14.544 0.923 -1.210 78.691 68.890 0.875 10072.448 8817.920 0.875 128 128 agg 15 1 1 tp8pp1dp1etp1ep8, bs=40 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=44 +balanced_32k Balanced 32K 32768 16384 16384 32 candidate candidate 15.754 14.544 0.923 -1.210 78.691 68.890 0.875 20144.896 17635.840 0.875 256 256 agg 15 1 1 tp8pp1dp1etp1ep8, bs=40 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=44 +balanced_32k Balanced 32K 32768 16384 16384 64 candidate candidate 15.754 14.544 0.923 -1.210 78.691 68.890 0.875 40289.792 35271.680 0.875 512 512 agg 15 1 1 tp8pp1dp1etp1ep8, bs=40 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=44 +balanced_32k Balanced 32K 32768 16384 16384 128 candidate candidate 15.754 14.544 0.923 -1.210 78.691 68.890 0.875 80579.584 70543.360 0.875 1024 1024 agg 15 1 1 tp8pp1dp1etp1ep8, bs=40 8p + 120d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=44 +balanced_32k Balanced 32K 32768 16384 16384 256 candidate candidate 15.754 14.544 0.923 -1.210 78.691 68.890 0.875 161159.168 141086.720 0.875 2048 2048 agg 15 1 1 tp8pp1dp1etp1ep8, bs=40 16p + 240d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=44 +balanced_32k Balanced 32K 32768 16384 16384 512 candidate candidate 15.754 14.544 0.923 -1.210 78.691 68.890 0.875 322318.336 282173.440 0.875 4096 4096 agg 15 1 1 tp8pp1dp1etp1ep8, bs=40 32p + 480d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=44 +balanced_4k Balanced 4K 4096 2048 2048 1 candidate no_results 13.922 611.690 4893.520 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=352 no result +balanced_4k Balanced 4K 4096 2048 2048 2 candidate candidate 13.922 13.619 0.978 -0.303 611.690 288.276 0.471 9787.040 4612.416 0.471 16 16 agg 15 1 1 tp8pp1dp1etp1ep8, bs=352 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=368 +balanced_4k Balanced 4K 4096 2048 2048 4 candidate candidate 13.922 13.619 0.978 -0.303 611.690 432.415 0.707 19574.080 13837.280 0.707 32 32 agg 15 1 1 tp8pp1dp1etp1ep8, bs=352 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=368 +balanced_4k Balanced 4K 4096 2048 2048 8 candidate candidate 13.922 14.163 1.017 0.241 611.690 364.968 0.597 39148.160 23357.952 0.597 64 64 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=352 1p + 7d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=32 +balanced_4k Balanced 4K 4096 2048 2048 16 candidate candidate 13.922 13.619 0.978 -0.303 611.690 468.331 0.766 78296.320 59946.368 0.766 128 128 agg 15 1 1 tp8pp1dp1etp1ep8, bs=352 3p + 13d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=368 +balanced_4k Balanced 4K 4096 2048 2048 32 candidate candidate 13.922 13.619 0.978 -0.303 611.690 468.331 0.766 156592.640 119892.736 0.766 256 256 agg 15 1 1 tp8pp1dp1etp1ep8, bs=352 6p + 26d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=368 +balanced_4k Balanced 4K 4096 2048 2048 64 candidate candidate 13.922 13.619 0.978 -0.303 611.690 468.331 0.766 313185.280 239785.472 0.766 512 512 agg 15 1 1 tp8pp1dp1etp1ep8, bs=352 12p + 52d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=368 +balanced_4k Balanced 4K 4096 2048 2048 128 candidate candidate 13.922 13.619 0.978 -0.303 611.690 468.331 0.766 626370.560 479570.944 0.766 1024 1024 agg 15 1 1 tp8pp1dp1etp1ep8, bs=352 24p + 104d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=368 +balanced_4k Balanced 4K 4096 2048 2048 256 candidate candidate 13.922 13.619 0.978 -0.303 611.690 468.331 0.766 1252741.120 959141.888 0.766 2048 2048 agg 15 1 1 tp8pp1dp1etp1ep8, bs=352 48p + 208d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=368 +balanced_4k Balanced 4K 4096 2048 2048 512 candidate candidate 13.922 13.619 0.978 -0.303 611.690 468.331 0.766 2505482.240 1918283.776 0.766 4096 4096 agg 15 1 1 tp8pp1dp1etp1ep8, bs=352 96p + 416d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=368 +balanced_64k Balanced 64K 65536 32768 32768 1 candidate no_results 18.201 45.442 363.536 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=20 no result +balanced_64k Balanced 64K 65536 32768 32768 2 candidate candidate 18.201 38.943 2.140 20.742 45.442 18.842 0.415 727.072 301.472 0.415 16 16 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=20 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +balanced_64k Balanced 64K 65536 32768 32768 4 candidate candidate 18.201 38.943 2.140 20.742 45.442 28.262 0.622 1454.144 904.384 0.622 32 32 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=20 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +balanced_64k Balanced 64K 65536 32768 32768 8 candidate candidate 18.201 38.943 2.140 20.742 45.442 32.973 0.726 2908.288 2110.272 0.726 64 64 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=20 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +balanced_64k Balanced 64K 65536 32768 32768 16 candidate candidate 18.201 38.943 2.140 20.742 45.442 35.328 0.777 5816.576 4521.984 0.777 128 128 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=20 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +balanced_64k Balanced 64K 65536 32768 32768 32 candidate candidate 18.201 38.943 2.140 20.742 45.442 35.328 0.777 11633.152 9043.968 0.777 256 256 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=20 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +balanced_64k Balanced 64K 65536 32768 32768 64 candidate candidate 18.201 38.943 2.140 20.742 45.442 35.328 0.777 23266.304 18087.936 0.777 512 512 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=20 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +balanced_64k Balanced 64K 65536 32768 32768 128 candidate candidate 18.201 38.943 2.140 20.742 45.442 35.328 0.777 46532.608 36175.872 0.777 1024 1024 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=20 8p + 120d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +balanced_64k Balanced 64K 65536 32768 32768 256 candidate candidate 18.201 38.943 2.140 20.742 45.442 35.328 0.777 93065.216 72351.744 0.777 2048 2048 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=20 16p + 240d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +balanced_64k Balanced 64K 65536 32768 32768 512 candidate candidate 18.201 38.943 2.140 20.742 45.442 35.328 0.777 186130.432 144703.488 0.777 4096 4096 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=20 32p + 480d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +balanced_8k Balanced 8K 8192 4096 4096 1 candidate no_results 14.969 329.047 2632.376 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=176 no result +balanced_8k Balanced 8K 8192 4096 4096 2 candidate candidate 14.969 14.267 0.953 -0.702 329.047 150.968 0.459 5264.752 2415.488 0.459 16 16 agg 15 1 1 tp8pp1dp1etp1ep8, bs=176 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +balanced_8k Balanced 8K 8192 4096 4096 4 candidate candidate 14.969 14.267 0.953 -0.702 329.047 226.452 0.688 10529.504 7246.464 0.688 32 32 agg 15 1 1 tp8pp1dp1etp1ep8, bs=176 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +balanced_8k Balanced 8K 8192 4096 4096 8 candidate candidate 14.969 14.267 0.953 -0.702 329.047 264.195 0.803 21059.008 16908.480 0.803 64 64 agg 15 1 1 tp8pp1dp1etp1ep8, bs=176 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +balanced_8k Balanced 8K 8192 4096 4096 16 candidate candidate 14.969 14.372 0.960 -0.597 329.047 258.294 0.785 42118.016 30995.280 0.736 128 120 agg 15 1 1 tp8pp1dp1etp1ep8, bs=176 1p + 14d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp4ep2, p_bs=1, d_bs=168 +balanced_8k Balanced 8K 8192 4096 4096 32 candidate candidate 14.969 14.372 0.960 -0.597 329.047 258.294 0.785 84236.032 61990.560 0.736 256 240 agg 15 1 1 tp8pp1dp1etp1ep8, bs=176 2p + 28d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp4ep2, p_bs=1, d_bs=168 +balanced_8k Balanced 8K 8192 4096 4096 64 candidate candidate 14.969 14.372 0.960 -0.597 329.047 258.294 0.785 168472.064 123981.120 0.736 512 480 agg 15 1 1 tp8pp1dp1etp1ep8, bs=176 4p + 56d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp4ep2, p_bs=1, d_bs=168 +balanced_8k Balanced 8K 8192 4096 4096 128 candidate candidate 14.969 14.050 0.939 -0.919 329.047 275.996 0.839 336944.128 278203.968 0.826 1024 1008 agg 15 1 1 tp8pp1dp1etp1ep8, bs=176 9p + 117d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp8ep1, p_bs=1, d_bs=184 +balanced_8k Balanced 8K 8192 4096 4096 256 candidate candidate 14.969 14.050 0.939 -0.919 329.047 275.996 0.839 673888.256 556407.936 0.826 2048 2016 agg 15 1 1 tp8pp1dp1etp1ep8, bs=176 18p + 234d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp8ep1, p_bs=1, d_bs=184 +balanced_8k Balanced 8K 8192 4096 4096 512 candidate candidate 14.969 14.267 0.953 -0.702 329.047 278.711 0.847 1347776.512 1130451.816 0.839 4096 4056 agg 15 1 1 tp8pp1dp1etp1ep8, bs=176 39p + 468d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +chat_4k_500 Chat 4K / 500 4500 4000 500 1 candidate no_results 9.716 387.410 3099.280 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=320 no result +chat_4k_500 Chat 4K / 500 4500 4000 500 2 candidate candidate 9.716 13.405 1.380 3.689 387.410 239.400 0.618 6198.560 3830.400 0.618 16 16 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=320 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp2ep4, p_bs=1, d_bs=320 +chat_4k_500 Chat 4K / 500 4500 4000 500 4 candidate candidate 9.716 13.383 1.377 3.667 387.410 239.400 0.618 12397.120 7660.800 0.618 32 32 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=320 2p + 2d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=328 +chat_4k_500 Chat 4K / 500 4500 4000 500 8 candidate candidate 9.716 13.718 1.412 4.002 387.410 252.928 0.653 24794.240 16187.392 0.653 64 64 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=320 3p + 5d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=32 +chat_4k_500 Chat 4K / 500 4500 4000 500 16 candidate candidate 9.716 13.383 1.377 3.667 387.410 239.400 0.618 49588.480 30643.200 0.618 128 128 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=320 8p + 8d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=328 +chat_4k_500 Chat 4K / 500 4500 4000 500 32 candidate candidate 9.716 13.383 1.377 3.667 387.410 239.400 0.618 99176.960 61286.400 0.618 256 256 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=320 16p + 16d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=328 +chat_4k_500 Chat 4K / 500 4500 4000 500 64 candidate candidate 9.716 13.383 1.377 3.667 387.410 239.400 0.618 198353.920 122572.800 0.618 512 512 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=320 32p + 32d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=328 +chat_4k_500 Chat 4K / 500 4500 4000 500 128 candidate candidate 9.716 13.718 1.412 4.002 387.410 260.155 0.672 396707.840 262236.240 0.661 1024 1008 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=320 45p + 81d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=32 +chat_4k_500 Chat 4K / 500 4500 4000 500 256 candidate candidate 9.716 13.718 1.412 4.002 387.410 260.155 0.672 793415.680 524472.480 0.661 2048 2016 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=320 90p + 162d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=32 +chat_4k_500 Chat 4K / 500 4500 4000 500 512 candidate candidate 9.716 13.718 1.412 4.002 387.410 260.155 0.672 1586831.360 1048944.960 0.661 4096 4032 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=320 180p + 324d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=32 +observed_high_7k_128k 7K / 128K 138240 7168 131072 1 candidate no_results 39.255 39.249 313.992 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=8 no result +observed_high_7k_128k 7K / 128K 138240 7168 131072 2 candidate candidate 39.255 104.443 2.661 65.188 39.249 15.073 0.384 627.984 241.168 0.384 16 16 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_high_7k_128k 7K / 128K 138240 7168 131072 4 candidate candidate 39.255 104.443 2.661 65.188 39.249 22.610 0.576 1255.968 723.520 0.576 32 32 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_high_7k_128k 7K / 128K 138240 7168 131072 8 candidate candidate 39.255 104.443 2.661 65.188 39.249 26.378 0.672 2511.936 1688.192 0.672 64 64 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_high_7k_128k 7K / 128K 138240 7168 131072 16 candidate candidate 39.255 104.443 2.661 65.188 39.249 28.262 0.720 5023.872 3617.536 0.720 128 128 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_high_7k_128k 7K / 128K 138240 7168 131072 32 candidate candidate 39.255 104.443 2.661 65.188 39.249 28.262 0.720 10047.744 7235.072 0.720 256 256 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_high_7k_128k 7K / 128K 138240 7168 131072 64 candidate candidate 39.255 104.443 2.661 65.188 39.249 28.262 0.720 20095.488 14470.144 0.720 512 512 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_high_7k_128k 7K / 128K 138240 7168 131072 128 candidate candidate 39.255 104.443 2.661 65.188 39.249 28.262 0.720 40190.976 28940.288 0.720 1024 1024 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 8p + 120d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_high_7k_128k 7K / 128K 138240 7168 131072 256 candidate candidate 39.255 104.443 2.661 65.188 39.249 28.262 0.720 80381.952 57880.576 0.720 2048 2048 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 16p + 240d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_high_7k_128k 7K / 128K 138240 7168 131072 512 candidate candidate 39.255 104.443 2.661 65.188 39.249 28.262 0.720 160763.904 115761.152 0.720 4096 4096 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=8 32p + 480d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_high_7k_1k 7K / 1K 8192 7168 1024 1 candidate no_results 11.838 259.386 2075.088 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=176 no result +observed_high_7k_1k 7K / 1K 8192 7168 1024 2 candidate candidate 11.838 13.841 1.169 2.003 259.386 146.611 0.565 4150.176 2345.776 0.565 16 16 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=176 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +observed_high_7k_1k 7K / 1K 8192 7168 1024 4 candidate candidate 11.838 14.382 1.215 2.544 259.386 164.678 0.635 8300.352 5269.696 0.635 32 32 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=176 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp8ep1, p_bs=1, d_bs=136 +observed_high_7k_1k 7K / 1K 8192 7168 1024 8 candidate candidate 11.838 14.382 1.215 2.544 259.386 164.678 0.635 16600.704 10539.392 0.635 64 64 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=176 2p + 6d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp8ep1, p_bs=1, d_bs=136 +observed_high_7k_1k 7K / 1K 8192 7168 1024 16 candidate candidate 11.838 13.841 1.169 2.003 259.386 202.681 0.781 33201.408 21078.824 0.635 128 104 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=176 4p + 9d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +observed_high_7k_1k 7K / 1K 8192 7168 1024 32 candidate candidate 11.838 13.841 1.169 2.003 259.386 202.681 0.781 66402.816 42157.648 0.635 256 208 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=176 8p + 18d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +observed_high_7k_1k 7K / 1K 8192 7168 1024 64 candidate candidate 11.838 13.841 1.169 2.003 259.386 202.681 0.781 132805.632 84315.296 0.635 512 416 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=176 16p + 36d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +observed_high_7k_1k 7K / 1K 8192 7168 1024 128 candidate candidate 11.838 13.841 1.169 2.003 259.386 202.681 0.781 265611.264 189709.416 0.714 1024 936 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=176 36p + 81d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +observed_high_7k_1k 7K / 1K 8192 7168 1024 256 candidate candidate 11.838 13.841 1.169 2.003 259.386 202.681 0.781 531222.528 400497.656 0.754 2048 1976 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=176 76p + 171d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +observed_high_7k_1k 7K / 1K 8192 7168 1024 512 candidate candidate 11.838 13.841 1.169 2.003 259.386 202.681 0.781 1062445.056 822074.136 0.774 4096 4056 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=176 156p + 351d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=184 +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 1 candidate no_results 30.270 41.621 332.968 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=11 no result +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 2 candidate candidate 30.270 105.782 3.495 75.512 41.621 15.073 0.362 665.936 241.168 0.362 16 16 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 4 candidate candidate 30.270 105.782 3.495 75.512 41.621 22.610 0.543 1331.872 723.520 0.543 32 32 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 8 candidate candidate 30.270 105.782 3.495 75.512 41.621 26.378 0.634 2663.744 1688.192 0.634 64 64 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 16 candidate candidate 30.270 105.782 3.495 75.512 41.621 28.262 0.679 5327.488 3617.536 0.679 128 128 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 32 candidate candidate 30.270 105.782 3.495 75.512 41.621 28.262 0.679 10654.976 7235.072 0.679 256 256 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 64 candidate candidate 30.270 105.782 3.495 75.512 41.621 28.262 0.679 21309.952 14470.144 0.679 512 512 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 128 candidate candidate 30.270 105.782 3.495 75.512 41.621 28.262 0.679 42619.904 28940.288 0.679 1024 1024 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 8p + 120d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 256 candidate candidate 30.270 105.782 3.495 75.512 41.621 28.262 0.679 85239.808 57880.576 0.679 2048 2048 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 16p + 240d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 512 candidate candidate 30.270 105.782 3.495 75.512 41.621 28.262 0.679 170479.616 115761.152 0.679 4096 4096 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 32p + 480d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 1 candidate no_results 14.310 1142.375 9139.000 8 0 missing 15 1 1 tp1pp1dp8etp1ep8, bs=80 no result +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 2 candidate candidate 14.310 13.486 0.942 -0.824 1142.375 571.254 0.500 18278.000 9140.064 0.500 16 16 agg 15 1 1 tp1pp1dp8etp1ep8, bs=80 1p + 1d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=92 +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 4 candidate candidate 14.310 13.486 0.942 -0.824 1142.375 856.881 0.750 36556.000 27420.192 0.750 32 32 agg 15 1 1 tp1pp1dp8etp1ep8, bs=80 1p + 3d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=92 +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 8 candidate candidate 14.310 13.486 0.942 -0.824 1142.375 999.694 0.875 73112.000 63980.416 0.875 64 64 agg 15 1 1 tp1pp1dp8etp1ep8, bs=80 1p + 7d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=92 +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 16 candidate candidate 14.310 12.740 0.890 -1.570 1142.375 971.263 0.850 146224.000 77701.040 0.531 128 80 agg 15 1 1 tp1pp1dp8etp1ep8, bs=80 1p + 9d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp8ep1, p_bs=1, d_bs=92 +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 32 candidate candidate 14.310 12.740 0.890 -1.570 1142.375 971.263 0.850 292448.000 233103.120 0.797 256 240 agg 15 1 1 tp1pp1dp8etp1ep8, bs=80 3p + 27d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp8ep1, p_bs=1, d_bs=92 +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 64 candidate candidate 14.310 13.486 0.942 -0.824 1142.375 1015.562 0.889 584896.000 511843.248 0.875 512 504 agg 15 1 1 tp1pp1dp8etp1ep8, bs=80 7p + 56d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=92 +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 128 candidate candidate 14.310 13.486 0.942 -0.824 1142.375 1015.562 0.889 1169792.000 1023686.496 0.875 1024 1008 agg 15 1 1 tp1pp1dp8etp1ep8, bs=80 14p + 112d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=92 +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 256 candidate candidate 14.310 13.486 0.942 -0.824 1142.375 1015.562 0.889 2339584.000 2047372.992 0.875 2048 2016 agg 15 1 1 tp1pp1dp8etp1ep8, bs=80 28p + 224d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=92 +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 512 candidate candidate 14.310 13.486 0.942 -0.824 1142.375 1015.562 0.889 4679168.000 4094745.984 0.875 4096 4032 agg 15 1 1 tp1pp1dp8etp1ep8, bs=80 56p + 448d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=92 +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 1 candidate no_results 30.181 41.497 331.976 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=11 no result +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 2 candidate candidate 30.181 105.574 3.498 75.393 41.497 15.073 0.363 663.952 241.168 0.363 16 16 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 4 candidate candidate 30.181 105.574 3.498 75.393 41.497 22.610 0.545 1327.904 723.520 0.545 32 32 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 8 candidate candidate 30.181 105.574 3.498 75.393 41.497 26.378 0.636 2655.808 1688.192 0.636 64 64 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 16 candidate candidate 30.181 105.574 3.498 75.393 41.497 28.262 0.681 5311.616 3617.536 0.681 128 128 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 32 candidate candidate 30.181 105.574 3.498 75.393 41.497 28.262 0.681 10623.232 7235.072 0.681 256 256 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 64 candidate candidate 30.181 105.574 3.498 75.393 41.497 28.262 0.681 21246.464 14470.144 0.681 512 512 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 128 candidate candidate 30.181 105.574 3.498 75.393 41.497 28.262 0.681 42492.928 28940.288 0.681 1024 1024 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 8p + 120d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 256 candidate candidate 30.181 105.574 3.498 75.393 41.497 28.262 0.681 84985.856 57880.576 0.681 2048 2048 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 16p + 240d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 512 candidate candidate 30.181 105.574 3.498 75.393 41.497 28.262 0.681 169971.712 115761.152 0.681 4096 4096 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=11 32p + 480d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=2 +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 1 candidate no_results 12.632 806.278 6450.224 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=512 no result +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 2 candidate candidate 12.632 13.269 1.050 0.637 806.278 391.022 0.485 12900.448 6256.352 0.485 16 16 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=512 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=512 +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 4 candidate candidate 12.632 13.450 1.065 0.818 806.278 520.205 0.645 25800.896 16646.560 0.645 32 32 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=512 1p + 3d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=56 +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 8 candidate candidate 12.632 13.382 1.059 0.750 806.278 575.078 0.713 51601.792 32204.368 0.624 64 56 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=512 1p + 6d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp2ep4, p_bs=1, d_bs=56 +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 16 candidate candidate 12.632 12.988 1.028 0.356 806.278 574.130 0.712 103203.584 64302.560 0.623 128 112 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=512 2p + 12d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp8ep1, p_bs=1, d_bs=56 +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 32 candidate candidate 12.632 12.988 1.028 0.356 806.278 574.130 0.712 206407.168 128605.120 0.623 256 224 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=512 4p + 24d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp8ep1, p_bs=1, d_bs=56 +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 64 candidate candidate 12.632 12.988 1.028 0.356 806.278 574.130 0.712 412814.336 289361.520 0.701 512 504 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=512 9p + 54d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp8ep1, p_bs=1, d_bs=56 +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 128 candidate candidate 12.632 12.988 1.028 0.356 806.278 574.130 0.712 825628.672 578723.040 0.701 1024 1008 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=512 18p + 108d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp8ep1, p_bs=1, d_bs=56 +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 256 candidate candidate 12.632 13.450 1.065 0.818 806.278 586.898 0.728 1651257.344 1159710.448 0.702 2048 1976 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=512 38p + 209d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=56 +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 512 candidate candidate 12.632 13.450 1.065 0.818 806.278 586.898 0.728 3302514.688 2380458.288 0.721 4096 4056 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=512 78p + 429d @ p:tp1pp1dp8etp1ep8 / d:tp1pp1dp8etp1ep8, p_bs=1, d_bs=56 +practical_32k_8k 32K / 8K 40960 32768 8192 1 candidate no_results 15.008 59.778 478.224 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=32 no result +practical_32k_8k 32K / 8K 40960 32768 8192 2 candidate candidate 15.008 14.161 0.944 -0.847 59.778 29.204 0.489 956.448 467.264 0.489 16 16 agg 15 1 1 tp8pp1dp1etp1ep8, bs=32 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=36 +practical_32k_8k 32K / 8K 40960 32768 8192 4 candidate candidate 15.008 14.161 0.944 -0.847 59.778 43.807 0.733 1912.896 1401.824 0.733 32 32 agg 15 1 1 tp8pp1dp1etp1ep8, bs=32 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=36 +practical_32k_8k 32K / 8K 40960 32768 8192 8 candidate candidate 15.008 14.161 0.944 -0.847 59.778 51.108 0.855 3825.792 3270.912 0.855 64 64 agg 15 1 1 tp8pp1dp1etp1ep8, bs=32 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=36 +practical_32k_8k 32K / 8K 40960 32768 8192 16 candidate candidate 15.008 14.161 0.944 -0.847 59.778 54.758 0.916 7651.584 7009.024 0.916 128 128 agg 15 1 1 tp8pp1dp1etp1ep8, bs=32 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=36 +practical_32k_8k 32K / 8K 40960 32768 8192 32 candidate candidate 15.008 14.161 0.944 -0.847 59.778 54.758 0.916 15303.168 14018.048 0.916 256 256 agg 15 1 1 tp8pp1dp1etp1ep8, bs=32 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=36 +practical_32k_8k 32K / 8K 40960 32768 8192 64 candidate candidate 15.008 14.161 0.944 -0.847 59.778 54.758 0.916 30606.336 28036.096 0.916 512 512 agg 15 1 1 tp8pp1dp1etp1ep8, bs=32 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=36 +practical_32k_8k 32K / 8K 40960 32768 8192 128 candidate candidate 15.008 14.161 0.944 -0.847 59.778 54.758 0.916 61212.672 56072.192 0.916 1024 1024 agg 15 1 1 tp8pp1dp1etp1ep8, bs=32 8p + 120d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=36 +practical_32k_8k 32K / 8K 40960 32768 8192 256 candidate candidate 15.008 14.161 0.944 -0.847 59.778 54.758 0.916 122425.344 112144.384 0.916 2048 2048 agg 15 1 1 tp8pp1dp1etp1ep8, bs=32 16p + 240d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=36 +practical_32k_8k 32K / 8K 40960 32768 8192 512 candidate candidate 15.008 14.161 0.944 -0.847 59.778 54.758 0.916 244850.688 224288.768 0.916 4096 4096 agg 15 1 1 tp8pp1dp1etp1ep8, bs=32 32p + 480d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=36 +practical_64k_8k 64K / 8K 73728 65536 8192 1 candidate no_results 21.193 39.163 313.304 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=15 no result +practical_64k_8k 64K / 8K 73728 65536 8192 2 candidate candidate 21.193 17.602 0.831 -3.591 39.163 18.371 0.469 626.608 293.936 0.469 16 16 agg 15 1 1 tp8pp1dp1etp1ep8, bs=15 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=18 +practical_64k_8k 64K / 8K 73728 65536 8192 4 candidate candidate 21.193 17.602 0.831 -3.591 39.163 27.556 0.704 1253.216 881.792 0.704 32 32 agg 15 1 1 tp8pp1dp1etp1ep8, bs=15 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=18 +practical_64k_8k 64K / 8K 73728 65536 8192 8 candidate candidate 21.193 17.602 0.831 -3.591 39.163 32.148 0.821 2506.432 2057.472 0.821 64 64 agg 15 1 1 tp8pp1dp1etp1ep8, bs=15 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=18 +practical_64k_8k 64K / 8K 73728 65536 8192 16 candidate candidate 21.193 17.602 0.831 -3.591 39.163 33.745 0.862 5012.864 3509.480 0.700 128 104 agg 15 1 1 tp8pp1dp1etp1ep8, bs=15 1p + 12d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=18 +practical_64k_8k 64K / 8K 73728 65536 8192 32 candidate candidate 21.193 17.602 0.831 -3.591 39.163 33.745 0.862 10025.728 7018.960 0.700 256 208 agg 15 1 1 tp8pp1dp1etp1ep8, bs=15 2p + 24d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=18 +practical_64k_8k 64K / 8K 73728 65536 8192 64 candidate candidate 21.193 17.602 0.831 -3.591 39.163 33.745 0.862 20051.456 14037.920 0.700 512 416 agg 15 1 1 tp8pp1dp1etp1ep8, bs=15 4p + 48d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=18 +practical_64k_8k 64K / 8K 73728 65536 8192 128 candidate candidate 21.193 17.602 0.831 -3.591 39.163 33.745 0.862 40102.912 31585.320 0.788 1024 936 agg 15 1 1 tp8pp1dp1etp1ep8, bs=15 9p + 108d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=18 +practical_64k_8k 64K / 8K 73728 65536 8192 256 candidate candidate 21.193 17.602 0.831 -3.591 39.163 33.745 0.862 80205.824 66680.120 0.831 2048 1976 agg 15 1 1 tp8pp1dp1etp1ep8, bs=15 19p + 228d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=18 +practical_64k_8k 64K / 8K 73728 65536 8192 512 candidate candidate 21.193 17.602 0.831 -3.591 39.163 33.745 0.862 160411.648 136869.720 0.853 4096 4056 agg 15 1 1 tp8pp1dp1etp1ep8, bs=15 39p + 468d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=18 +stress_128k 128K profile 131072 114688 16384 1 candidate no_results 42.750 31.053 248.424 8 0 missing 15 1 1 tp8pp1dp1etp1ep8, bs=6 no result +stress_128k 128K profile 131072 114688 16384 2 candidate candidate 42.750 31.848 0.745 -10.902 31.053 15.073 0.485 496.848 241.168 0.485 16 16 agg 15 1 1 tp8pp1dp1etp1ep8, bs=6 1p + 1d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +stress_128k 128K profile 131072 114688 16384 4 candidate candidate 42.750 31.848 0.745 -10.902 31.053 22.610 0.728 993.696 723.520 0.728 32 32 agg 15 1 1 tp8pp1dp1etp1ep8, bs=6 1p + 3d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +stress_128k 128K profile 131072 114688 16384 8 candidate candidate 42.750 31.848 0.745 -10.902 31.053 26.378 0.849 1987.392 1688.192 0.849 64 64 agg 15 1 1 tp8pp1dp1etp1ep8, bs=6 1p + 7d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +stress_128k 128K profile 131072 114688 16384 16 candidate candidate 42.750 57.215 1.338 14.465 31.053 24.730 0.796 3974.784 3165.440 0.796 128 128 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=6 1p + 15d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +stress_128k 128K profile 131072 114688 16384 32 candidate candidate 42.750 57.215 1.338 14.465 31.053 24.730 0.796 7949.568 6330.880 0.796 256 256 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=6 2p + 30d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +stress_128k 128K profile 131072 114688 16384 64 candidate candidate 42.750 57.215 1.338 14.465 31.053 24.730 0.796 15899.136 12661.760 0.796 512 512 disagg 15 1 1 tp8pp1dp1etp1ep8, bs=6 4p + 60d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=4 +stress_128k 128K profile 131072 114688 16384 128 candidate candidate 42.750 31.848 0.745 -10.902 31.053 27.993 0.901 31798.272 28216.944 0.887 1024 1008 agg 15 1 1 tp8pp1dp1etp1ep8, bs=6 9p + 117d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +stress_128k 128K profile 131072 114688 16384 256 candidate candidate 42.750 31.848 0.745 -10.902 31.053 27.993 0.901 63596.544 56433.888 0.887 2048 2016 agg 15 1 1 tp8pp1dp1etp1ep8, bs=6 18p + 234d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +stress_128k 128K profile 131072 114688 16384 512 candidate candidate 42.750 31.848 0.745 -10.902 31.053 27.993 0.901 127193.088 112867.776 0.887 4096 4032 agg 15 1 1 tp8pp1dp1etp1ep8, bs=6 36p + 468d @ p:tp8pp1dp1etp1ep8 / d:tp8pp1dp1etp1ep8, p_bs=1, d_bs=8 +stress_256k 256K profile 262144 229376 32768 1 no_results no_results 8 0 missing 15 1 1 agg no result +stress_256k 256K profile 262144 229376 32768 2 no_results no_results 16 0 missing 15 1 1 agg no result +stress_256k 256K profile 262144 229376 32768 4 no_results no_results 32 0 missing 15 1 1 agg no result +stress_256k 256K profile 262144 229376 32768 8 no_results no_results 64 0 missing 15 1 1 agg no result +stress_256k 256K profile 262144 229376 32768 16 no_results no_results 128 0 missing 15 1 1 agg no result +stress_256k 256K profile 262144 229376 32768 32 no_results no_results 256 0 missing 15 1 1 agg no result +stress_256k 256K profile 262144 229376 32768 64 no_results no_results 512 0 missing 15 1 1 agg no result +stress_256k 256K profile 262144 229376 32768 128 no_results no_results 1024 0 missing 15 1 1 agg no result +stress_256k 256K profile 262144 229376 32768 256 no_results no_results 2048 0 missing 15 1 1 agg no result +stress_256k 256K profile 262144 229376 32768 512 no_results no_results 4096 0 missing 15 1 1 agg no result diff --git a/docs/k2-aic-exhaustive-study/data/126_inference_profile_summary.tsv b/docs/k2-aic-exhaustive-study/data/126_inference_profile_summary.tsv new file mode 100644 index 000000000000..dfc028af2b40 --- /dev/null +++ b/docs/k2-aic-exhaustive-study/data/126_inference_profile_summary.tsv @@ -0,0 +1,21 @@ +workload_profile summary_label context_budget isl osl load_bands_considered agg_load_invariant_all_nodes disagg_load_invariant_all_nodes agg_win_node_count disagg_win_node_count missing_disagg_node_count agg_win_nodes disagg_win_nodes missing_disagg_nodes user_ratio_min user_ratio_max gpu_ratio_min gpu_ratio_max cluster_ratio_min cluster_ratio_max verdict +balanced_128k Balanced 128K 131072 65536 65536 15 1 1 0 9 1 2,4,8,16,32,64,128,256,512 1 1.659 1.659 0.415 0.778 0.415 0.778 disagg +balanced_16k Balanced 16K 16384 8192 8192 15 1 1 9 0 1 2,4,8,16,32,64,128,256,512 1 0.929 0.929 0.492 0.922 0.492 0.922 agg +balanced_1k Balanced 1K 1024 512 512 15 1 1 0 9 1 2,4,8,16,32,64,128,256,512 1 1.007 1.007 0.495 0.743 0.495 0.743 disagg +balanced_256k Balanced 256K 262144 131072 131072 15 1 1 9 0 1 2,4,8,16,32,64,128,256,512 1 0.540 0.540 0.662 1.242 0.662 1.242 agg +balanced_2k Balanced 2K 2048 1024 1024 15 1 1 9 0 1 2,4,8,16,32,64,128,256,512 1 0.996 0.996 0.489 0.856 0.489 0.856 agg +balanced_32k Balanced 32K 32768 16384 16384 15 1 1 9 0 1 2,4,8,16,32,64,128,256,512 1 0.923 0.923 0.467 0.875 0.467 0.875 agg +balanced_4k Balanced 4K 4096 2048 2048 15 1 1 8 1 1 2,4,16,32,64,128,256,512 8 1 0.978 1.017 0.471 0.766 0.471 0.766 mixed +balanced_64k Balanced 64K 65536 32768 32768 15 1 1 0 9 1 2,4,8,16,32,64,128,256,512 1 2.140 2.140 0.415 0.777 0.415 0.777 disagg +balanced_8k Balanced 8K 8192 4096 4096 15 1 1 9 0 1 2,4,8,16,32,64,128,256,512 1 0.939 0.960 0.459 0.847 0.459 0.839 agg +chat_4k_500 Chat 4K / 500 4500 4000 500 15 1 1 0 9 1 2,4,8,16,32,64,128,256,512 1 1.377 1.412 0.618 0.672 0.618 0.661 disagg +observed_high_7k_128k 7K / 128K 138240 7168 131072 15 1 1 0 9 1 2,4,8,16,32,64,128,256,512 1 2.661 2.661 0.384 0.720 0.384 0.720 disagg +observed_high_7k_1k 7K / 1K 8192 7168 1024 15 1 1 0 9 1 2,4,8,16,32,64,128,256,512 1 1.169 1.215 0.565 0.781 0.565 0.774 disagg +observed_low_0p5k_128k 0.5K / 128K 131584 512 131072 15 1 1 0 9 1 2,4,8,16,32,64,128,256,512 1 3.495 3.495 0.362 0.679 0.362 0.679 disagg +observed_low_0p5k_1k 0.5K / 1K 1536 512 1024 15 1 1 9 0 1 2,4,8,16,32,64,128,256,512 1 0.890 0.942 0.500 0.889 0.500 0.875 agg +observed_mean_1p5k_128k 1.5K / 128K 132608 1536 131072 15 1 1 0 9 1 2,4,8,16,32,64,128,256,512 1 3.498 3.498 0.363 0.681 0.363 0.681 disagg +observed_mean_1p5k_1k 1.5K / 1K 2560 1536 1024 15 1 1 0 9 1 2,4,8,16,32,64,128,256,512 1 1.028 1.065 0.485 0.728 0.485 0.721 disagg +practical_32k_8k 32K / 8K 40960 32768 8192 15 1 1 9 0 1 2,4,8,16,32,64,128,256,512 1 0.944 0.944 0.489 0.916 0.489 0.916 agg +practical_64k_8k 64K / 8K 73728 65536 8192 15 1 1 9 0 1 2,4,8,16,32,64,128,256,512 1 0.831 0.831 0.469 0.862 0.469 0.853 agg +stress_128k 128K profile 131072 114688 16384 15 1 1 6 3 1 2,4,8,128,256,512 16,32,64 1 0.745 1.338 0.485 0.901 0.485 0.887 mixed +stress_256k 256K profile 262144 229376 32768 15 1 1 0 0 10 1,2,4,8,16,32,64,128,256,512 no_disagg_candidate diff --git a/docs/k2-aic-exhaustive-study/figures/126_inference_tok_s_user_scatter.png b/docs/k2-aic-exhaustive-study/figures/126_inference_tok_s_user_scatter.png new file mode 100644 index 000000000000..3b5df58da537 Binary files /dev/null and b/docs/k2-aic-exhaustive-study/figures/126_inference_tok_s_user_scatter.png differ diff --git a/docs/k2-aic-exhaustive-study/figures/126_inference_user_margin_vs_gpu_ratio.png b/docs/k2-aic-exhaustive-study/figures/126_inference_user_margin_vs_gpu_ratio.png new file mode 100644 index 000000000000..2f0c7b8b5f1e Binary files /dev/null and b/docs/k2-aic-exhaustive-study/figures/126_inference_user_margin_vs_gpu_ratio.png differ diff --git a/docs/k2-aic-exhaustive-study/sections/126-inference-decision.md b/docs/k2-aic-exhaustive-study/sections/126-inference-decision.md new file mode 100644 index 000000000000..ca38d6ddf375 --- /dev/null +++ b/docs/k2-aic-exhaustive-study/sections/126-inference-decision.md @@ -0,0 +1,59 @@ +# Epic 126: Inference Decision + +This section reduces the additive-band AIConfigurator catalog to the inference-only question for RL360: +when should the serving plan stay aggregated, and when should it split prefill from decode? + +## Scope + +- Source data: [../data/126_inference_pairwise.tsv](../data/126_inference_pairwise.tsv) +- Profile rollup: [../data/126_inference_profile_summary.tsv](../data/126_inference_profile_summary.tsv) +- Compact table: [../data/126_inference_decision_table.md](../data/126_inference_decision_table.md) + +The catalog spans `15` additive load bands from `load_000001` through `load_016384`. +For every profile, node count, and topology, the selected inference candidate is identical across those load bands. +That means the inference decision collapses to a unique `20 x 10` profile-by-node frontier. + +## Executive Read + +- Disaggregation is the clear user-throughput winner for long-context and observed `128K`-class shapes. + The strongest gains are `3.495x` to `3.498x` on the observed `128K` profiles, `2.661x` on `7K / 128K`, `2.140x` on balanced `64K`, and `1.659x` on balanced `128K`. +- Aggregation retains the edge for the practical RL `32K / 8K` and `64K / 8K` shapes, for most balanced `8K` to `32K` profiles, and for the balanced `256K` extreme. + The largest aggregate holdout is balanced `256K`, where disagg drops to `0.540x` of aggregate `tok/s/user`. +- Near-parity cases exist, but they are narrow. + Balanced `1K` favors disagg by only `0.7%`, balanced `2K` favors agg by `0.4%`, and balanced `4K` flips only at `8` nodes. +- Candidate availability is itself part of the decision. + Every profile is missing a `1`-node disagg candidate, and the `256K` stress profile has no disagg candidate at any node count. + +## Figures + +![Aggregate vs disaggregated tok/s/user frontier](../figures/126_inference_tok_s_user_scatter.png) + +Figure 1 compares agg and disagg `tok/s/user` directly. +Small node-count jitter is applied only in the figure so stacked `2` to `512` node selections remain visible. +Points above the diagonal favor disagg on per-user throughput; points below it favor agg. + +![User margin versus tok/s/gpu ratio](../figures/126_inference_user_margin_vs_gpu_ratio.png) + +Figure 2 shows the key tradeoff that drives the decision. +Many disagg winners live left of `1.0` on `tok/s/gpu` while still sitting above `0` on user margin. +In other words, disaggregation often gives up GPU efficiency in exchange for materially better `tok/s/user`. + +## Decision Rules + +1. Prefer disagg for `64K+` decode-heavy or observed `128K`-class profiles. + These are the profiles where user throughput improves by `1.66x` to `3.50x` even though `tok/s/gpu` usually falls into the `0.36x` to `0.78x` range of agg. +2. Prefer agg for the practical RL rollout shapes and for the balanced short-to-mid context band. + Practical `32K / 8K` lands at `0.944x` on disagg user throughput, practical `64K / 8K` at `0.831x`, and balanced `16K` to `32K` stays in the `0.923x` to `0.929x` range. +3. Treat `1K` to `4K` and the `128K` stress profile as boundary cases. + Balanced `1K`, balanced `2K`, and balanced `4K` are effectively parity decisions, while `128K` stress flips from agg at `2` to `8` nodes, to disagg at `16` to `64`, and back to agg at `128+`. + +## What This Means For RL360 + +For the inference-side branch decision alone, the cleanest split is: + +- Default to agg for practical RL training shapes and for short-to-mid balanced profiles. +- Switch to disagg when the workload looks like the observed `128K` cluster or the balanced `64K` and `128K` families. +- Flag `stress_128k`, balanced `1K` to `4K`, and any `1`-node deployment as measurement-first exceptions rather than fixed policy. + +This keeps the epic focused on the serving decision itself. +Measured runtime validation and reproducibility automation stay outside this PR.