MetaBall

Training Guide | Assembly Guide | Data Viewer | Model | Dataset

MetaBall is a soft end-of-robot module capable of vision-based deformable perception. It is designed to provide a wide range of motion and sensing capabilities, making it suitable for various applications in robotics, including manipulation, exploration, and human-robot interaction. MetaBall is a member of the Proprioceptive Soft Robotic Modules (ProSoRo) family.

📦 Installation

Clone this repository:

git clone https://github.com/han-xudong/metaball.git

We use uv to manage Python dependencies. See uv documentation for installation instructions. Once uv is installed, run the following commands to set up the environment:

uv sync
uv pip install -e ".[all]"

which will install all dependencies for training and deploying MetaBall. If you only want to train the model, install only the training dependencies:

uv pip install -e ".[train]"

Or if you only want to deploy it, install only the deploying dependencies:

uv pip install -e ".[deploy]"

⚡ Training

Before training the model, you need to prepare the dataset according to the training guide. Install the package with [train] extras first.

Then, run the following command to train the model:

uv run python scripts/train.py [options]

There are several configurable options for training:

Options	Description	Type	Default
--batch-size	Batch size for training.	int	128
--lr	Learning rate for the optimizer.	float	1e-5
--max-epochs	Maximum number of training epochs.	int	2000
--save-dir	Directory to save training logs and checkpoints.	str	lightning_logs
--zero-loss-weight	Weight for the zero loss component.	float	0.5
--data.dataset-path	Path to the dataset directory.	str	./data/metaball/sim
--data.num-workers	Number of workers for data loading.	int	4
--data.pin-memory	Whether to pin memory during data loading.	bool	False
--data.persistent-workers	Whether to use persistent workers for data loading.	bool	True
--data.train-val-split	Train-validation split ratios.	tuple	0.875 0.125
--model.name	Model name.	str	BallNet
--model.x-dim	Input dimension.	tuple	6
--model.y-dim	Output dimension.	tuple	6 2862
--model.hidden-dim	Hidden layer dimensions for each part of the model.	tuple	512 512 1024 1024

You can also follow the training guide to test the model by calculating the R2 score, and RMSE, and visualizing the prediction results, etc.

🤖 Hardware

The MetaBall hardware mainly consists of a camera, a controller board, a power board, an LED light board, a soft struture and several 3D-printed parts. The camera is used for capturing images, while the controller board publishes the images through TCP protocol. The power board supports 6-36V input and powers the controller board. The LED light board provides illumination for the camera. The soft struture is made of polyurethane (PU), which is the main part to interact with the environment. 3D-printed parts are used to assemble the camera, controller board, and power board together.

CAD files of the MetaBall are available on Fusion. Please refer to the assembly guide for more details on how to assemble the MetaBall.

🚀 Deployment

After installing the package and connecting the MetaBall to the computer, modify the configuration and deploy it by running:

uv run metaball [options]

Various configuration options are available:

Options	Description	Type	Default
--host	Host address for the publisher.	str	127.0.0.1
--port	Port number for the publisher.	int	6666
--onnx-path	Path to the ONNX model file.	str	./models/BallNet.onnx
--camera.host	Camera host address.	str	10.114.201.1
--camera.port	Camera port number.	int	5555

All data, including images, poses, forces, and node displacements, will be published through TCP (ZeroMQ + Protocol Buffers) in the following format:

package MetaBall;

message MetaBall
{
    double timestamp = 1;
    bytes img = 2;
    repeated double pose = 3;
    repeated double force = 4;
    repeated double node = 5;
}

They can be visualized through the MetaBall Viewer.

📄 License

This project is licensed under the MIT License.

🔗 Citation

If you find this project useful in your research, please consider citing:

@article{han2025anchoring,
  title={Anchoring Morphological Representations Unlocks Latent Proprioception in Soft Robots},
  author={Han, Xudong and Guo, Ning and Xu, Ronghan and Wan, Fang and Song, Chaoyang},
  journal={Advanced Intelligent Systems},
  volume={7},
  pages={e202500444},
  year={2025}
}