Skip to content

Add option to compress vertex normal tangent to axis angle #24535

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
beicause wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add option to compress vertex normal tangent to axis angle #24535

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
951b232
Add option to compress vertex normal tangent to axis angle
beicause Jun 5, 2026
2389341
Simplify `axis_angle_to_normal_tangent`
beicause Jun 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
356 changes: 280 additions & 76 deletions crates/bevy_mesh/src/mesh.rs
View file
Open in desktop

Large diffs are not rendered by default.

122 changes: 117 additions & 5 deletions crates/bevy_mesh/src/vertex.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use bevy_derive::EnumVariantMeta;
use bevy_ecs::resource::Resource;
use bevy_math::{
bounding::{Aabb2d, Aabb3d, BoundingVolume},
vec2, Vec2, Vec3, Vec3A, Vec3Swizzles,
ops, vec2, vec3, Mat3, Quat, Vec2, Vec3, Vec3A, Vec3Swizzles, Vec4, Vec4Swizzles,
};
#[cfg(feature = "serialize")]
use bevy_platform::collections::HashMap;
Expand Down Expand Up @@ -1155,8 +1155,10 @@ pub fn octahedral_encode_signed(v: Vec3) -> Vec2 {

/// Encode tangent vectors as octahedral coordinates with range [-1, 1]. The sign is encoded in y component. Use [`octahedral_decode_tangent`] to decode.
pub fn octahedral_encode_tangent(v: Vec3, sign: f32) -> Vec2 {
// Bias to ensure that encoding as unorm16 preserves the sign. See https://github.com/godotengine/godot/pull/73265
let bias = 1.0 / 32767.0;
// Bias to ensure that encoding as snorm16 preserves the sign.
let bits = 16.;
let bias = 1. / (ops::powf(2.0, bits - 1.) - 1.);

let mut n_xy = octahedral_encode_signed(v);
// Map y to always be positive.
n_xy.y = n_xy.y * 0.5 + 0.5;
Expand Down Expand Up @@ -1184,12 +1186,48 @@ pub fn octahedral_decode_tangent(v: Vec2) -> (Vec3, f32) {
(octahedral_decode_signed(f), sign)
}

/// Convert the normal and tangent to the equivalent axis-angle representation.
/// The range of angle is [-2pi, 2pi], where the sign represents the handedness of the tangent.
pub fn normal_tangent_to_axis_angle(normal: Vec3, tangent_signed: Vec4) -> (Vec3, f32) {
// Bias to ensure that encoding as snorm16 preserves the sign.
let bits = 16.;
let bias = 1. / (ops::powf(2.0, bits - 1.) - 1.);

let tangent = tangent_signed.xyz();
let bitangent = normal.cross(tangent);
let (axis, angle) =
Quat::from_mat3(&Mat3::from_cols(tangent, bitangent, normal)).to_axis_angle();

@greeble-dev greeble-dev Jun 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this will handle cases where the normal and tangent are scaled or not perpendicular? Maybe scaled doesn't need to be handled if they're normalized elsewhere - I didn't check. For non-perpendicular the glam docs vaguely say that Quat::from_mat3 is "ill-defined".

@beicause beicause Jun 8, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are normalized in create_compressed_axis_angle_attribute_values, though the perpendicularity is unchecked.

let angle = angle
.rem_euclid(2.0 * core::f32::consts::PI)
.max(bias * 2.0 * core::f32::consts::PI);

(axis, angle * tangent_signed.w)
}

/// Convert the axis-angle representation back to normal and tangent.
/// The range of angle is [-2pi, 2pi], where the sign represents the handedness of the tangent.
pub fn axis_angle_to_normal_tangent(axis: Vec3, angle: f32) -> (Vec3, Vec4) {

@greeble-dev greeble-dev Jun 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm reading this right, it's doing normal = rotation * vec3(0.0, 0.0, 1.0) and tangent = rotation * vec3(1.0, 0.0, 0.0), where the rotation is axis/angle? If so, a downside of that approach is that the accuracy is pretty unevenly distributed - it will be less accurate the closer the angle is to 180.

I'm not an expert on this stuff, but I believe the more common approach is to create an orthogonal basis from the axis and then rotate it by the angle. Depending on how the basis is constructed the accuracy will be more evenly distributed. This blog post has a survey of different approaches, although it focuses on 4 byte encodings rather than this PR's 6 bytes: https://zeux.io/2026/04/30/quantizing-tangent-frames/

Should this PR use the orthogonal basis? I'm not sure - it's more complicated and costs more ALU, so maybe the extra accuracy isn't worth it.

A spicier option is to encode a quaternion with an octahedral projection - this is simple to decode and has reasonably distributed accuracy. However, I haven't seen this approach described publicly, and I haven't tested if it's better than the approaches mentioned in the above blog post, so it would be a bit of a gamble. Will also need one bit stolen from a component to store the tangent.w.

// Encodes to a Vec3 with range [-1, 1].
fn encode(q: Quat) -> Vec3 {
    let s = 1.0 / (q.xyz().abs().max_element() + q.w.abs());
    q.xyz() * s * q.w.signum()
}

fn decode(v: Vec3) -> Quat {
    let w = 1.0 - v.abs().max_element();
    Quat::from_xyzw(v.x, v.y, v.z, w).normalize()
}

TLDR: There's more accurate options but they have trade-offs. I don't think that should block this PR - other options could be explored later.

@beicause beicause Jun 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://zeux.io/2026/04/30/quantizing-tangent-frames Good article, which mentions many approaches If using 32bits for normal tangent.

If we want to use 48bits for normal tangent, I think this PR (oct16x2+a15, orientation is stolen from angle) might be the best choice. Diamond encoding avoids sin()/cos() but I don't think it's much faster/more precise than angle.

To utilize the unused position.w, another choice is quantizing position to uint21x3 then packing 16x3+5x3 bits into Uint16x4 to improve position precision, but unpacking it has overhead. I'm not sure if it's worth doing.

@beicause beicause Jun 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PS: The tangent angle encoding in the article constructs an orthogonal basis from the normal and then rotates by an angle, which differs from the axis-angle representation of TBN matrix in this PR.

Constructing basis from the normal allows the normal encoding/decoding to be independent of the tangent, but I suspect the precision will be lower.

@greeble-dev greeble-dev Jun 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I worded that poorly. I mentioned creating the basis from the axis, but that's nonsense - there's no axis, just the normal.

My intuition is that the basis method would be more accurate. Axis/angle is similar to latitude/longitude encodings - a lot of bits are wasted near the poles where the normal is close to (0, 0, 1) or (0, 0, -1). I've tested this for a different case - using axis/angle to encode quaternions - and axis/angle clearly lost to cubic/octahedral projections (which are kinda similar to basis construction in dividing up space). But I haven't tested it for TBN, so maybe my intuition is wrong.

let sign = if angle >= 0.0 { 1.0 } else { -1.0 };
let angle_abs = angle * sign;
let c = ops::cos(angle_abs);
let s = ops::sin(angle_abs);
let v = axis * s;
let omc = axis * (1.0 - c);

let tangent = omc.xxx() * axis + vec3(c, v.z, -v.y);
// let bitangent = omc.yyy() * axis + vec3(-v.z, c, v.x);
let normal = omc.zzz() * axis + vec3(v.y, -v.x, c);

(normal, tangent.extend(sign))
}

#[cfg(test)]
mod tests {
use bevy_math::{vec2, vec3, Vec4Swizzles};
use bevy_math::{vec2, vec3, Mat3, Quat, Vec3, Vec4, Vec4Swizzles};

use crate::{
octahedral_decode_signed, octahedral_decode_tangent,
axis_angle_to_normal_tangent, normal_tangent_to_axis_angle, octahedral_decode_signed,
octahedral_decode_tangent,
vertex::{octahedral_encode_signed, octahedral_encode_tangent},
};

Expand All @@ -1198,18 +1236,21 @@ mod tests {
let vectors = [
vec3(1.0, 2.0, 3.0).normalize().extend(1.0),
vec3(1.0, 0.0, 0.0).extend(-1.0),
vec3(0.0, 1.0, 0.0).extend(-1.0),
vec3(0.0, 0.0, -1.0).extend(1.0),
vec3(0.0, 0.0, -1.0).extend(-1.0),
];
let expected_encoded_normals = [
vec2(0.16666667, 0.33333334),
vec2(1.0, 0.0),
vec2(0.0, 1.0),
vec2(-1.0, -1.0),
vec2(-1.0, -1.0),
];
let expected_encoded_tangents = [
vec2(0.16666667, 0.6666667),
vec2(1.0, -0.5),
vec2(0.0, -1.0),
vec2(-1.0, 3.051851e-5),
vec2(-1.0, -3.051851e-5),
];
Expand All @@ -1226,4 +1267,75 @@ mod tests {
assert!(decoded_tangent.distance(v.xyz()) < 1e-4);
}
}

pub fn axis_angle_to_normal_tangent_glam(axis: Vec3, angle: f32) -> (Vec3, Vec4) {
let sign = if angle >= 0.0 { 1.0 } else { -1.0 };
let tbn = Mat3::from_quat(Quat::from_axis_angle(axis, angle * sign));
(tbn.col(2), tbn.col(0).extend(sign))
}

#[test]
fn normal_tangent_axis_angle_encode_decode() {
let normal_tangent = [
(vec3(1.0, 0.0, 0.0), vec3(0.0, 0.0, 1.0).extend(1.0)),
(vec3(1.0, 0.0, 0.0), vec3(0.0, 0.0, 1.0).extend(-1.0)),
(vec3(0.0, 0.0, 1.0), vec3(1.0, 0.0, 0.0).extend(1.0)),
(vec3(0.0, 0.0, 1.0), vec3(1.0, 0.0, 0.0).extend(-1.0)),
(vec3(0.0, 1.0, 0.0), vec3(0.0, 0.0, 1.0).extend(1.0)),
(vec3(1.0, 0.0, 0.0), vec3(0.0, -1.0, 0.0).extend(1.0)),
(
vec3(1.0, 1.0, 1.0).normalize(),
vec3(1.0, -1.0, 0.0).normalize().extend(1.0),
),
(
vec3(1.0, 2.0, 0.0).normalize(),
vec3(0.0, 0.0, 1.0).extend(-1.0),
),
(
vec3(0.0, 1.0, 1.0).normalize(),
vec3(1.0, 0.0, 0.0).extend(1.0),
),
(
vec3(-1.0, 1.0, 1.0).normalize(),
vec3(1.0, 1.0, 0.0).normalize().extend(-1.0),
),
(
vec3(3.0, 1.0, 2.0).normalize(),
vec3(0.0, 1.0, -0.5).normalize().extend(1.0),
),
];

#[expect(
clippy::approx_constant,
reason = "The values are taken from the test results"
)]
let expected_axis_angle = [
(vec3(0.7071068, 0.0, 0.7071068), 3.1415927),
(vec3(0.7071068, 0.0, 0.7071068), -3.1415927),
(vec3(1.0, 0.0, 0.0), 3.051851e-5),
(vec3(1.0, 0.0, 0.0), -3.051851e-5),
(vec3(0.57735026, 0.57735026, 0.57735026), 4.1887903),
(vec3(0.57735026, -0.57735026, 0.57735026), 4.1887903),
(vec3(-0.7429061, 0.3077218, -0.5944728), 1.2171159),
(vec3(0.64793617, 0.40044653, 0.64793617), -3.9033751),
(vec3(-1.0, 0.0, 0.0), 0.7853981),
(vec3(-0.7429061, -0.3077218, 0.5944728), -1.2171159),
(vec3(0.22525999, 0.6253928, 0.7470889), 1.6242763),
];

for (i, &(normal, tangent)) in normal_tangent.iter().enumerate() {
let (axis, angle) = normal_tangent_to_axis_angle(normal, tangent);
assert_eq!(angle.signum(), tangent.w.signum());
assert!(axis.distance(expected_axis_angle[i].0) < 1e-8);

let (decoded_normal, decoded_tangent) = axis_angle_to_normal_tangent(axis, angle);
let (decoded_normal_glam, decoded_tangent_glam) =
axis_angle_to_normal_tangent_glam(axis, angle);

assert!(decoded_normal.distance(normal) < 1e-3);
assert!(decoded_tangent.distance(tangent) < 1e-3);
assert!(decoded_normal_glam.distance(normal) < 1e-3);
assert!(decoded_tangent_glam.distance(tangent) < 1e-3);
}
}
}
8 changes: 8 additions & 0 deletions crates/bevy_pbr/src/prepass/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -535,6 +535,14 @@ impl PrepassPipeline {
}
vertex_attributes.push(Mesh::ATTRIBUTE_TANGENT.at_shader_location(4));
}
if layout
.0
.get_attribute_compression()
.contains(MeshAttributeCompressionFlags::PACKED_AXIS_ANGLE_TBN)
{
shader_defs.push("VERTEX_TANGENTS".into());
shader_defs.push("VERTEX_PACKED_AXIS_ANGLE_TBN".into());
}
}
if mesh_key
.intersects(MeshPipelineKey::MOTION_VECTOR_PREPASS | MeshPipelineKey::DEFERRED_PREPASS)
Expand Down
66 changes: 51 additions & 15 deletions crates/bevy_pbr/src/prepass/prepass_io.wgsl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,13 @@ struct UncompressedVertex {
#ifdef VERTEX_NORMALS
@location(3) normal: vec3<f32>,
#endif

#ifdef VERTEX_TANGENTS
@location(4) tangent: vec4<f32>,
#else ifdef VERTEX_PACKED_AXIS_ANGLE_TBN
@location(4) tangent: vec4<f32>,
#endif

#endif // NORMAL_PREPASS_OR_DEFERRED_PREPASS

#ifdef SKINNED
Expand All @@ -39,26 +43,20 @@ struct UncompressedVertex {

struct Vertex {
@builtin(instance_index) instance_index: u32,

#ifdef VERTEX_PACKED_AXIS_ANGLE_TBN
@location(0) compressed_position_angle: vec4<f32>,
#ifdef NORMAL_PREPASS_OR_DEFERRED_PREPASS
@location(3) compressed_axis: vec2<f32>,
#endif

#else // VERTEX_PACKED_AXIS_ANGLE_TBN

#ifdef VERTEX_POSITIONS_COMPRESSED
@location(0) compressed_position: vec4<f32>,
#else
@location(0) position: vec3<f32>,
#endif
#ifdef VERTEX_UVS_A
#ifdef VERTEX_UVS_A_COMPRESSED
@location(1) compressed_uv: vec2<f32>,
#else
@location(1) uv: vec2<f32>,
#endif
#endif
#ifdef VERTEX_UVS_B
#ifdef VERTEX_UVS_B_COMPRESSED
@location(2) compressed_uv_b: vec2<f32>,
#else
@location(2) uv_b: vec2<f32>,
#endif
#endif

#ifdef NORMAL_PREPASS_OR_DEFERRED_PREPASS
#ifdef VERTEX_NORMALS
#ifdef VERTEX_NORMALS_COMPRESSED
Expand All @@ -76,6 +74,23 @@ struct Vertex {
#endif
#endif // NORMAL_PREPASS_OR_DEFERRED_PREPASS

#endif // VERTEX_PACKED_AXIS_ANGLE_TBN

#ifdef VERTEX_UVS_A
#ifdef VERTEX_UVS_A_COMPRESSED
@location(1) compressed_uv: vec2<f32>,
#else
@location(1) uv: vec2<f32>,
#endif
#endif
#ifdef VERTEX_UVS_B
#ifdef VERTEX_UVS_B_COMPRESSED
@location(2) compressed_uv_b: vec2<f32>,
#else
@location(2) uv_b: vec2<f32>,
#endif
#endif

#ifdef SKINNED
@location(5) joint_indices: vec4<u32>,
@location(6) joint_weights: vec4<f32>,
Expand All @@ -96,6 +111,24 @@ fn decompress_vertex(vertex_in: Vertex, instance_index: u32) -> UncompressedVert
let mesh_metadata = bevy_pbr::mesh_functions::get_metadata(instance_index);
var uncompressed_vertex: UncompressedVertex;
uncompressed_vertex.instance_index = instance_index;

#ifdef VERTEX_PACKED_AXIS_ANGLE_TBN
uncompressed_vertex.position = bevy_render::utils::decompress_vertex_position(vertex_in.compressed_position_angle, mesh_metadata.aabb_center, mesh_metadata.aabb_half_extents);
#ifdef NORMAL_PREPASS_OR_DEFERRED_PREPASS
var normal: vec3f;
var tangent: vec4f;
bevy_render::utils::decompress_vertex_axis_angle_to_normal_tangent(
vertex_in.compressed_axis,
vertex_in.compressed_position_angle.w,
&normal,
&tangent,
);
uncompressed_vertex.normal = normal;
uncompressed_vertex.tangent = tangent;
#endif // NORMAL_PREPASS_OR_DEFERRED_PREPASS

#else // VERTEX_PACKED_AXIS_ANGLE_TBN

#ifdef VERTEX_POSITIONS_COMPRESSED
uncompressed_vertex.position = bevy_render::utils::decompress_vertex_position(vertex_in.compressed_position, mesh_metadata.aabb_center, mesh_metadata.aabb_half_extents);
#else
Expand All @@ -117,6 +150,9 @@ fn decompress_vertex(vertex_in: Vertex, instance_index: u32) -> UncompressedVert
#endif
#endif
#endif // NORMAL_PREPASS_OR_DEFERRED_PREPASS

#endif // VERTEX_PACKED_AXIS_ANGLE_TBN

#ifdef VERTEX_UVS_A
#ifdef VERTEX_UVS_A_COMPRESSED
let uv_min_and_extents_a = mesh_metadata.uv_channels_min_and_extents[0];
Expand Down
Loading
Loading