Inquiry About Supporting Additional Approaches and WholeBody Models

[sarim-next]

Hello,

First, thank you for the incredible work you've put into this library! Your dedication to maintaining and improving it has clearly created a valuable resource for many of us in the community.

I wanted to ask if you have any plans to support additional approaches for pose estimation and body tracking, beyond the existing options like MediaPipe Holistic and OpenPose. Specifically, I’m curious about the possibility of integrating solutions such as OpenPifPaf WholeBody, which offers whole-body tracking capabilities.

Additionally, I’m interested in whether there are any plans to enable support for models trained on datasets like COCO-WholeBody, such as the lightweight models in the Sapiens project. These could potentially enhance functionality for specific use cases where detailed whole-body keypoints are essential.

I realize that expanding support can be a significant undertaking, so I completely understand if this is not currently a priority or in scope. Regardless, I appreciate any insights you can share about future directions or potential considerations for these approaches.

Thank you again for all your hard work on this project—it’s greatly appreciated!

Best regards.

[AmitMY]

Hi! This repository is mainly designed to work with the binary file format, and is mostly agnostic to the specific pose estimation model used.

For example, let's say you run OpenPose which gives you a directory of JSON files. What this library provides is a loader (https://github.com/sign-language-processing/pose/blob/master/src/python/pose_format/utils/openpose.py) that defines what pose components OpenPose has (hands, body, face), what is the names of the points and how they are connected, and then, how to load it from the JSON files. Once defined and loaded, you can now save it as a .pose file which would work with the rest of the infrastructure and utilities.