Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions _codeql_detected_source_root
151 changes: 151 additions & 0 deletions runtime/animation/MOTION_MATCHING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Motion Matching System

## Overview

This motion matching system provides advanced character animation through data-driven motion selection. It allows characters to naturally select and transition between animation clips based on desired movement characteristics.

## Components

### MotionDatabase

The `MotionDatabase` class stores a collection of animation clips and their extracted features for fast querying.

**Key Features:**
- Stores motion frames with extracted features (pose, velocity, trajectory)
- Supports multiple animation clips
- Configurable sampling rate for feature extraction

**Usage:**
```cpp
// Create database
auto database = std::make_shared<MotionDatabase>();

// Add animation clips
database->AddClip(walkClip, 30); // Sample at 30 fps
database->AddClip(runClip, 30);
database->AddClip(jumpClip, 30);
```

### MotionMatcher

The `MotionMatcher` class performs the motion matching algorithm, finding the best matching frame in the database for a given query.

**Key Features:**
- Multi-factor cost calculation (pose, velocity, trajectory)
- Configurable weights for different matching factors
- Fast search through database

**Usage:**
```cpp
MotionMatcher matcher;
matcher.SetDatabase(database);

// Create query
MotionQuery query;
query.desiredRootVelocity = Vector3(1.0f, 0.0f, 0.0f);
query.desiredTrajectory = {Vector3(0.2f, 0.0f, 0.0f), Vector3(0.4f, 0.0f, 0.0f)};

// Find best match
MotionMatchResult result = matcher.FindBestMatch(query);
```

### MotionMatchingNode

The `MotionMatchingNode` is an animation graph node that integrates motion matching into the animation system.

**Key Features:**
- Automatic motion selection based on desired velocity and trajectory
- Smooth transitions between matched motions
- Configurable search interval for performance optimization
- Root motion support
- Blending between current and target animations

**Configuration:**
```cpp
MotionMatchingNode::PersistentData data;
data.database = database; // Motion database
data.rootMotion = true; // Enable root motion
data.blendTime = 0.2f; // 200ms transition time
data.searchInterval = 3; // Search every 3 frames

MotionMatchingNode node(data);
```

**Runtime Control:**
```cpp
// Set desired movement
node.SetDesiredVelocity(Vector3(2.0f, 0.0f, 1.0f));
node.SetDesiredTrajectory({
Vector3(0.2f, 0.0f, 0.1f),
Vector3(0.4f, 0.0f, 0.2f),
Vector3(0.6f, 0.0f, 0.3f)
});

// Adjust parameters
node.SetBlendTime(0.3f);
node.SetSearchInterval(5);
```

## Motion Matching Algorithm

The motion matching algorithm works as follows:

1. **Feature Extraction**: When clips are added to the database, features are extracted at regular intervals:
- Joint positions (relative to root)
- Joint velocities
- Root velocity
- Future trajectory positions

2. **Query Construction**: At runtime, a query is constructed with:
- Desired root velocity
- Desired future trajectory
- Current pose (optional)

3. **Matching**: The matcher compares the query against all frames in the database using a weighted cost function:
```
Cost = (PoseCost × PoseWeight) + (VelocityCost × VelocityWeight) + (TrajectoryCost × TrajectoryWeight)
```

4. **Transition**: When a better match is found, the system smoothly blends from the current animation to the new matched animation over the specified blend time.

## Integration with Animation System

The MotionMatchingNode follows the standard AnimNode interface:

```cpp
// Initialize
AnimContext context;
node.InitAny(context);

// Update (per frame)
AnimLayerContext layerContext;
node.TickAny(layerContext, deltaTime);

// Evaluate pose
AnimationEval evalContext(skeleton);
node.EvalAny(evalContext);
```

## Performance Considerations

- **Search Interval**: Set higher values (e.g., 5-10 frames) for better performance
- **Database Size**: Larger databases provide more natural motion but increase search time
- **Blend Time**: Longer blend times are smoother but less responsive
- **Feature Weights**: Adjust weights to prioritize different aspects of matching

## Future Enhancements

Potential areas for improvement:

1. **Feature Extraction**: Full implementation of pose and velocity extraction from animation clips
2. **Spatial Acceleration**: Use spatial data structures (KD-tree, etc.) for faster searches
3. **Pose Mirroring**: Support for mirrored animations to reduce database size
4. **Constraint Matching**: Add support for environmental constraints (walls, obstacles)
5. **Multi-threaded Search**: Parallelize database search for better performance
6. **Animation Tagging**: Support for semantic tags to filter search space

## References

Motion matching is based on research from:
- "Motion Matching and The Road to Next-Gen Animation" (GDC 2016)
- "Learned Motion Matching" (ACM SIGGRAPH 2020)
90 changes: 90 additions & 0 deletions runtime/animation/include/animation/graph/MotionMatchingNode.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
//
// Created by Zach Lee on 2025/6/3.
//

#pragma once

#include <animation/graph/AnimationNode.h>
#include <animation/motion/MotionMatcher.h>
#include <animation/motion/MotionDatabase.h>
#include <animation/core/AnimationPlayer.h>
#include <animation/core/AnimationUtils.h>

namespace sky {

// Motion matching node provides advanced animation selection using motion matching
class MotionMatchingNode : public AnimNode {
public:
// Persistent configuration data
struct PersistentData {
MotionDatabasePtr database; // Motion database to search
bool rootMotion = false; // Enable root motion
float blendTime = 0.2f; // Transition blend time
uint32_t searchInterval = 3; // Frames between searches (optimization)
};

// Runtime data
struct Data : PersistentData {
size_t currentFrameIndex = 0; // Current frame in database
size_t targetFrameIndex = 0; // Target frame for transition
float currentTime = 0.f; // Current playback time
bool transitioning = false; // Whether currently transitioning
float transitionTime = 0.f; // Time spent in transition
uint32_t framesSinceSearch = 0; // Frames since last search
};

explicit MotionMatchingNode(const PersistentData& inData);
~MotionMatchingNode() override = default;

// Set motion query parameters
void SetDesiredVelocity(const Vector3& velocity);
void SetDesiredTrajectory(const std::vector<Vector3>& trajectory);

// Control
void SetEnableRootMotion(bool enable);
void SetBlendTime(float time);
void SetSearchInterval(uint32_t interval);

// AnimNode interface
void PreTick(const AnimationTick& tick) override;
void InitAny(const AnimContext& context) override;
void TickAny(const AnimLayerContext& context, float deltaTime) override;
void EvalAny(AnimationEval& context) override;

private:
// Perform motion matching search
void SearchBestMatch();

// Build motion query from current state
MotionQuery BuildQuery() const;

// Update animation playback
void UpdatePlayback(float deltaTime);

// Sample pose from current frame
void SampleCurrentPose(AnimPose& pose);

// Sample pose from target frame (for blending)
void SampleTargetPose(AnimPose& pose);

// Runtime data
Data data;

// Motion matcher
MotionMatcher matcher;

// Query parameters (updated by user)
Vector3 desiredVelocity;
std::vector<Vector3> desiredTrajectory;

// Animation player for current clip
AnimationSequencePlayer currentPlayer;

// Animation player for target clip (during transition)
AnimationSequencePlayer targetPlayer;

// Blend helper for transitions
AnimFadeInOut fadeInOut;
};

} // namespace sky
59 changes: 59 additions & 0 deletions runtime/animation/include/animation/motion/MotionDatabase.h
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,65 @@

#pragma once

#include <animation/core/AnimationClip.h>
#include <animation/core/AnimationPose.h>
#include <core/math/Vector3.h>
#include <vector>
#include <memory>

namespace sky {

// Motion feature represents the characteristics of a pose at a specific frame
struct MotionFeature {
// Joint positions in local space (relative to root)
std::vector<Vector3> jointPositions;
// Joint velocities
std::vector<Vector3> jointVelocities;
// Root velocity
Vector3 rootVelocity;
// Future trajectory positions (e.g., 0.2s, 0.4s, 0.6s ahead)
std::vector<Vector3> futureTrajectory;
};

// Motion frame represents a single frame in the database
struct MotionFrame {
AnimClipPtr clip; // Animation clip this frame belongs to
uint32_t frameIndex; // Frame index in the clip
float timeInSeconds; // Time in seconds
MotionFeature feature; // Features for this frame

MotionFrame() : frameIndex(0), timeInSeconds(0.f) {}
};

// Motion database stores all motion data and features for matching
class MotionDatabase {
public:
MotionDatabase() = default;
~MotionDatabase() = default;

// Add an animation clip to the database
void AddClip(const AnimClipPtr& clip, uint32_t sampleRate = 30);

// Clear all data
void Clear();

// Get all motion frames
const std::vector<MotionFrame>& GetFrames() const { return frames; }

// Get number of frames
size_t GetNumFrames() const { return frames.size(); }

// Get a specific frame
const MotionFrame* GetFrame(size_t index) const;

private:
// Extract features from animation clip
void ExtractFeatures(const AnimClipPtr& clip, uint32_t sampleRate);

// All motion frames in the database
std::vector<MotionFrame> frames;
};

using MotionDatabasePtr = std::shared_ptr<MotionDatabase>;

} // namespace sky
15 changes: 0 additions & 15 deletions runtime/animation/include/animation/motion/MotionMacher.h

This file was deleted.

66 changes: 66 additions & 0 deletions runtime/animation/include/animation/motion/MotionMatcher.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
//
// Created by Zach Lee on 2025/6/3.
//

#pragma once

#include <animation/motion/MotionDatabase.h>
#include <core/math/Vector3.h>

namespace sky {

// Motion query describes what we're looking for in the database
struct MotionQuery {
// Current joint positions (for pose matching)
std::vector<Vector3> currentJointPositions;
// Current joint velocities
std::vector<Vector3> currentJointVelocities;
// Desired root velocity
Vector3 desiredRootVelocity;
// Desired future trajectory
std::vector<Vector3> desiredTrajectory;

// Weight factors for different features
float poseWeight = 1.0f;
float velocityWeight = 1.0f;
float trajectoryWeight = 2.0f;
};

// Motion matching result
struct MotionMatchResult {
size_t frameIndex; // Index in the motion database
float cost; // Match cost (lower is better)
bool valid; // Whether the result is valid

MotionMatchResult() : frameIndex(0), cost(0.f), valid(false) {}
};

// Motion matcher finds the best matching frame in the database
class MotionMatcher {
public:
MotionMatcher() = default;
~MotionMatcher() = default;

// Set the motion database to search
void SetDatabase(const MotionDatabasePtr& db);

// Find the best matching frame for the given query
MotionMatchResult FindBestMatch(const MotionQuery& query) const;

// Calculate cost between query and a database frame
float CalculateCost(const MotionQuery& query, const MotionFeature& feature) const;

private:
// Calculate pose matching cost
float CalculatePoseCost(const MotionQuery& query, const MotionFeature& feature) const;

// Calculate velocity matching cost
float CalculateVelocityCost(const MotionQuery& query, const MotionFeature& feature) const;

// Calculate trajectory matching cost
float CalculateTrajectoryCost(const MotionQuery& query, const MotionFeature& feature) const;

MotionDatabasePtr database;
};

} // namespace sky
Loading