Skip to content

Buffer Abstraction#260

Closed
zkfriendly wants to merge 24 commits into
worldfnd:mainfrom
zkfriendly:zkfr/buffer-abstraction
Closed

Buffer Abstraction#260
zkfriendly wants to merge 24 commits into
worldfnd:mainfrom
zkfriendly:zkfr/buffer-abstraction

Conversation

@zkfriendly

@zkfriendly zkfriendly commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR introduces the buffer abstraction layer. All buffer abstraction traits are defined in the buffer crate, with a CPU implementation that essentially wraps the existing in-memory code paths.

At this moment, the buffer crate still depends on existing algebra and protocol crates. Future work can consolidate the relevant logic directly into the buffer crate as the abstraction settles.

Notes

There is also an experimental Metal implementation, but it is not included in this PR.

zk_whir is not fully ported to the buffer abstraction yet. It uses .as_slice() in places to keep the code working, so behavior is essentially unchanged compared to directly using CPU buffers. Since WHIR v3 is coming and is expected to replace zk_whir, that path is intentionally not fully ported for now.

@zkfriendly

Copy link
Copy Markdown
Collaborator Author

cc @recmo

@shreyas-londhe shreyas-londhe left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @zkfriendly. Really like where this lands. Trait split is the right cut, the CPU backend stays a thin wrapper with no second implementation to drift, and the prover got simpler dropping the Any/Cow recycling for mixed_linear_combination and linear_forms_rlc. Left a few inline comments, but the shape is solid.

Comment thread src/protocols/basecase.rs
Comment thread src/protocols/merkle_tree.rs
Comment thread src/algebra/ntt/mod.rs
Comment on lines +131 to +136
fn interleaved_encode(
&self,
messages: Messages<'_, F>,
masks: &ActiveBuffer<F>,
codeword_length: usize,
) -> ActiveBuffer<F>;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: ReedSolomon is a backend-abstraction trait but hardcodes the global ActiveBuffer alias in its signature, which defeats the purpose for the one trait that most needs it.

Give it an associated type Buffer: Buffer<F> and thread it through the registry (type Dyn<F> = dyn ReedSolomon<F, Buffer = ActiveBuffer<F>>); NttEngine then declares type Buffer = CpuBuffer<F>. The payoff is enforcement, not tidiness: today a second backend can't even be expressed here, since the trait forces every impl to speak CpuBuffer. With the associated type, an impl whose buffer doesn't match the build's ActiveBuffer fails to register as a type error instead of sitting as dead code that forces host round-trips if called.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this point. But I think we can keep it simple for now, and refactor it once we introduce another backend type.

Comment thread src/buffer/cpu/buffer.rs Outdated
@zkfriendly zkfriendly mentioned this pull request Jul 2, 2026
@zkfriendly

Copy link
Copy Markdown
Collaborator Author

Superseded by #263 — same branch now pushed directly to worldfnd/whir (GitHub can't change a PR's head repo, so it had to be reopened). Review comments here have been addressed on the branch: the blocking basecase check is fixed, build_nodes caller obligation documented, and the fold_pair overrides removed as part of a larger simplification that drops the slice/view layer entirely. Replies on the individual threads to follow on #263.

@zkfriendly zkfriendly closed this Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants