MotionMatchingCharacterController stomping Unity IK in Late Update

[jhughes2112]

I spent a good part of the day trying to figure out why IK was rotated wrong. We do head tracking sometimes in another layer. I finally discovered that we were taking the SkeletonTransform list and applying them in OnSkeletonTransformUpdated, which happens in LateUpdate. That overwrites transforms because IK happens before this.

Our setup has a character controller that is physical and roams around the world as a capsule, and a set of joints and a skinned mesh. That does not animate, per se, but it is assigned the outcomes from a blend of what our Animator and MotionMatching give us. There is a separate object that has the MotionMatching scripts on it, has a separate copy of the skeleton but no meshes and no colliders. We feed it trajectory information and it moves around the world (invisibly) and animates. We scrape the skeleton to get the animation data, essentially.

I was considering making a Playables object that would represent the Motion Matching animation data, that could then easily be integrated and interpolated into/out of when we needed more control or wanted to use traditional animation. That would be absolutely fantastic.

Anyway, questions.

1. It's unclear if we are set up properly. And if we are, LateUpdate is not a good place to grab joint information because it stomps IK and other blended animation data.

2. We do not get the benefit of foot locking because our actual character moves according to physics not animation data. Would be great if there was a clean way to snap the motion matching object to where our character is, so it could do useful things like foot IK for us in the actual coordinate space we are in.

Thanks!

[JLPM22]

Hi! Thank you for your comment. Integrating Motion Matching (or any custom animation system) with Unity has always been tricky because of how closed the Mecanim Animator system is :( I hope that Unity will allow us to extend it in the future so we don't have to constantly find workarounds!

About your questions:

1- LateUpdate() is not ideal, but right now it is the only way to share the same skeleton with the animator. Because the animator will always override all the joints in the skeleton. If you take a look at the Unity Animator Integration Sample (in the advanced examples folder, in the package), I have coded a simple masking and blending system that allows you to easily select which bones are controlled by the animator and which ones by Motion Matching. This mask can be changed at run-time too. I hope this can be helpful for your use case.

2- The MotionMatchingController component has a function SetPosAdjustment() that allows adding an offset to the animation space. For example, it is what I use in the SpringCharacterController class to pull the character towards the target position of the user input. See an example on ClampMotionMatching() here: https://github.com/JLPM22/MotionMatching/blob/main/com.jlpm.motionmatching/Runtime/CharacterController/SpringCharacterController.cs#L207C22-L207C41

[jhughes2112]

I did eventually figure it out. What we ended up doing is having the MotionMatchingRenderer component be in a child object of the physical capsule that moves around the world. That way forward vectors always line up. The MotionMatching component is adjusted as you mentioned so it is in the same position as the physical object it represents.

We are currently using your masking system to mask out various bone sets based on the animations we are overlaying on top of motion matching, since the combinatorics make motion capture infeasible for everything the upper body might be doing separately from locomotion. I think that's true for most games, actually.

The ideal system would be using a Playables component to provide motion matching animation data and drive the base layer in an Animator (there are hooks for this), and then we could blend in animation layers above that as needed, rather than wholly mask out bones.

I haven't done this (yet), but ChatGPT says it should work something like this:

Animator animator;
PlayableGraph graph;
AnimationMixerPlayable motionMatchMixer;
AnimatorControllerPlayable controllerPlayable;

void Start()
{
    animator = GetComponent<Animator>();

    graph = PlayableGraph.Create("MotionMatchingGraph");
    graph.SetTimeUpdateMode(DirectorUpdateMode.GameTime);

    // Load your AnimatorController here
    RuntimeAnimatorController controller = animator.runtimeAnimatorController;

    // Wrap the AnimatorController so it can be used as a Playable
    controllerPlayable = AnimatorControllerPlayable.Create(graph, controller);

    // Create a mixer to drive layer 0
    motionMatchMixer = AnimationMixerPlayable.Create(graph, 2, true);

    // Connect the motion matching mixer to the controllerPlayable's input
    controllerPlayable.ConnectInput(0, motionMatchMixer, 0, 1.0f);

    // Set output to animator
    var output = AnimationPlayableOutput.Create(graph, "Output", animator);
    output.SetSourcePlayable(controllerPlayable);

    graph.Play();
}

void ApplyMotionMatchClip(AnimationClip clip, float startTime)
{
    var clipPlayable = AnimationClipPlayable.Create(graph, clip);
    clipPlayable.SetTime(startTime);
    clipPlayable.SetSpeed(1f);

    // Optionally blend with a previous pose
    motionMatchMixer.DisconnectInput(0);
    motionMatchMixer.ConnectInput(0, clipPlayable, 0);
    motionMatchMixer.SetInputWeight(0, 1f);
}

Thanks for building this system, by the way. We have found it relatively easy to use (if at times a little strangely designed). The other MM system we tried (MxM) was very difficult to get good results.

[JLPM22]

Thank you for the feedback! I will take a look at the PlayableGraph and see if it can be integrated. I added it to the TODO list!

Thanks for building this system, by the way. We have found it relatively easy to use (if at times a little strangely designed). The other MM system we tried (MxM) was very difficult to get good results.

Thank you!! This means a lot to me. I mainly develop this Motion Matching for some projects as part of my PhD and also during my free time. I am very happy it is useful :)