Skip to content

refactor(sae): move SST-2 SAE example into notebook tutorials#110

Merged
alirezabayatmk merged 3 commits into
mainfrom
pr-89
Jul 8, 2026
Merged

refactor(sae): move SST-2 SAE example into notebook tutorials#110
alirezabayatmk merged 3 commits into
mainfrom
pr-89

Conversation

@phusroyal

Copy link
Copy Markdown
Collaborator

Summary 📝

Adds SAE feature visualization utilities and a notebook-based SST-2/GPT-2 SAE interpretation workflow. The PR makes it easier to inspect why one SAE feature behaves like positive or negative evidence by connecting three views: feature ranking, vocabulary-level logit effects, and token-level activations.

The SST-2 SAE example has also been moved out of examples/ and into the SAE notebook/tutorial flow, so the long-form interpretability walkthrough lives with the other notebook-based SAE material.

Details

  • Adds plot_sae_feature_logit_effects(feature_id, decoder, unembedding, ...)

    • Projects one SAE decoder direction through the model unembedding.
    • Shows strongest negative and positive token effects.
    • Adds a signed vocabulary-wide logit-effect histogram.
    • Handles decoder/unembedding orientation explicitly.
  • Adds plot_sae_token_activations(...)

    • Displays top token activations for one SAE feature.
    • Accepts either prepared activation rows or raw [N, seq] / [N, seq, features] arrays.
    • Supports BOS and padding masking.
    • Highlights readable token activations in compact rows.
  • Replaces the old SST-2 SAE script with a notebook workflow:

    • moved from examples/sae_sst2_feature_enrichment.py
    • now lives as tutorials/sae/03_sst2_feature_enrichment.ipynb
    • loads SST-2, encodes GPT-2 SAE activations, ranks sentiment-selective features, and displays positive/negative feature visualizations.
  • Updates SAE feature ranking:

    • readable-token filtering
    • smoothed label-token weights
    • sentence-level selectivity
    • contribution-ranked examples
    • compact output summaries
  • Exports the new SAE plotting APIs from murano.plotting.

    • adds nltk to the notebook optional dependency check
    • regenerates uv.lock

Bugfixes 🐛

  • SAE logit-effect plotting now validates decoder and unembedding shape/orientation instead of assuming one layout.
  • Notebook target selection now fails loudly if either positive or negative candidates are missing.
  • The notebook no longer stores huge printed rows or Plotly payloads by default, keeping the committed notebook lightweight.

Validation

  • ruff format --check
  • ruff check --fix --exit-non-zero-on-fix
  • uv lock --check
  • pyright src
  • git diff --cached --check

@alirezabayatmk alirezabayatmk merged commit 869dce0 into main Jul 8, 2026
5 checks passed
@alirezabayatmk alirezabayatmk deleted the pr-89 branch July 8, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants