Skip to content

Add OrthogonalRandomFeaturesKernel#2735

Open
bulochka68 wants to merge 1 commit intocornellius-gp:mainfrom
bulochka68:add-orf-kernel
Open

Add OrthogonalRandomFeaturesKernel#2735
bulochka68 wants to merge 1 commit intocornellius-gp:mainfrom
bulochka68:add-orf-kernel

Conversation

@bulochka68
Copy link
Copy Markdown

Summary

Adds OrthogonalRandomFeaturesKernel - a drop-in replacement for RFFKernel
that uses block-orthogonal random projections instead of i.i.d. Gaussian samples.

ORF strictly dominates standard RFF in mean squared error of kernel approximation
(Choromanski et al., 2017) while maintaining the same computational complexity O(nD)
and the same interface.

Usage

# Before
kernel = gpytorch.kernels.RFFKernel(num_samples=512)

# After — identical interface, better approximation
kernel = gpytorch.kernels.OrthogonalRandomFeaturesKernel(num_samples=512)

Implementation details

Each block of d frequency vectors shares a Haar-distributed orthogonal basis (via QR
decomposition) but has independent chi(d)-distributed norms, so the marginal distribution
of each frequency matches N(0, I_d) — preserving the unbiasedness of the RBF approximation.

Changes

  • gpytorch/kernels/orf_kernel.py — new kernel
  • gpytorch/kernels/__init__.py — export
  • test/kernels/test_orf_kernel.py — 18 tests (BaseKernelTestCase + ORF-specific)
  • docs/source/kernels.rst — documentation entry

References

  • Yu et al., Orthogonal Random Features, NeurIPS 2016
  • Choromanski et al., The Geometry of Random Features, AISTATS 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant