Add ContextFuse-2048-BigramSmear submission

[Julz19]

Summary

This PR updates the track_10min_16mb submission:

• records/track_10min_16mb/2026-03-20_ContextFuse-2048-BigramSmear

This remains a follow-up to PR #143 (ContextFuse-2048) and stays focused on val_bpb as the primary challenge metric.

Corrected canonical run (seed=1337):

• val_loss = 1.94796677
• val_bpb = 1.15369565
• original training time for the saved model = 589978ms
• corrected fixed-eval time = 150215ms
• standalone total bytes = 15331125

Relative to PR #143, this improves the canonical val_bpb from 1.17792945 to 1.15369565.

Scoring Correction

The prior revision of this submission used a sliding-window evaluator that could rescore the same final stride tokens more than once in truncated tail windows when EVAL_STRIDE < TRAIN_SEQ_LEN.

This PR fixes that evaluator in the standalone train_gpt.py and updates the submission metadata to use the corrected canonical metric from an exact reevaluation of the saved seed=1337 raw checkpoint.

Because of that fix:

• train.log, train_seed42.log, and train_seed7.log are retained as original pre-fix training logs for transparency
• the old three-seed mean / median claim is withdrawn
• this PR no longer presents a post-fix statistical multi-seed claim

Method

Relative to PR #143, this submission keeps the same baseline-derived train@2048 path and adds the stronger compression-aware stack that transferred honestly:

• BigramHash token-pair features
• SmearGate input smoothing
• mixed int6 export
• SWA checkpoint averaging
• Muon weight decay
• corrected bigram control-tensor handling
• fixed sliding-window evaluation

Attribution

This submission builds on ideas previously explored in the repo, especially:

• PR Add ContextFuse-2048 submission #143 for the original ContextFuse-2048 base
• PR Record: OrthoInit + Int6 MLP3x + BigramHash + SmearGate (val_bpb: 1.1539) #135 for the BigramHash + SmearGate + mixed-int6 direction
• PR Record: Int6 MLP3x + SmearGate + BigramHash + MuonWD + SWA (mean val_bpb=1.1483) #162 for the later high-scoring stacked recipe

Validation

Included in the submission folder:

• standalone train_gpt.py that compiles and runs from inside the record folder
• original train.log
• corrected canonical reevaluation log: train_fixed_eval_seed1337.log
• original train_seed42.log and train_seed7.log for transparency
• updated artifact accounting in README.md and submission.json

The standalone script in the submission folder is 1500 lines and the canonical standalone artifact is 15331125 bytes.

Prepared with assistance from OpenAI Codex.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0f75a5fae7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Julz19]

Resolved bug identified in comment by codex which cause our script to double-count tail tokens and bias the val_loss/val_bpb.