代码提交

tactile/undergraduat/code/.gitignore

@@ -0,0 +1,172 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+# VSCode
+.vscode
+
+# Specific
+/hand_script/output*
+benchmark.csv
+MUJOCO_LOG.TXT
+# symlink for real-time dev of weart sdk
+weartsdk

tactile/undergraduat/code/README.md

@@ -0,0 +1,114 @@
+![](assets/cover.png)
+
+# Introduction
+The goal of this project is to link a haptic device (the WEART TouchDIVER), a VR headset (the Oculus Rift S) and a physics simulation software (CoppeliaSim with the MuJoCo engine, or directly MuJoCo) to simulate real-time touch of a deformable object.
+
+This project is conducted at the DIAG Robotics Laboratory of Sapienza University of Rome, under the supervision of Marilena Vendittelli. It is the subject of one of my internship at INSA Rennes, a French engineering school.
+
+The simulator integrates code to retarget the motion of the fingers (usinig the closure and abduction values fromm WEART) to the virtual hand in MuJoCo. This has been done by a group of Sapienza students.
+
+# MuJoCoXR
+During the project, a way to display MuJoCo in a VR headset has been developed.
+
+You will find the attempts in [mjxr_tests](mjxr_tests). The final, working one is [mujoco_openxr.py](mjxr_tests/mujoco_openxr.py) and is also available on [GitHub Gists](https://gist.github.com/SkytAsul/b1a48a31c4f86b65d72bc8edcb122d3f).
+
+To better understand how everything work, see [the paper I wrote](mujocoxr-paper/MuJoCoXR.pdf).
+
+# Deformable mesh tutorial
+I made a tutorial on how to convert a 3D model to a deformable material in MuJoCo. It is available [here](deformable-mesh-tuto/deformable-mesh-tuto.pdf).
+
+# Simulator
+## Dependencies
+Before installing dependencies, remember to create a Python virtual environment!
+
+- [WEART Python SDK](https://github.com/WEARTHaptics/WEART-SDK-Python)
+    - for the TouchDIVER
+    - `pip install weartsdk-sky`
+- [CoppeliaSim](https://coppeliarobotics.com/) ZMQ API
+    - for the simulation
+    - `pip install coppeliasim_zmqremoteapi_client`
+    - see [the manual](https://manual.coppeliarobotics.com/en/zmqRemoteApiOverview.htm)
+- [MuJoCo](https://mujoco.readthedocs.io/en/stable/overview.html)
+    - for the simulation
+    - `pip install mujoco`
+    - see [the manual](https://mujoco.readthedocs.io/en/stable/python.html)
+- [pyopenxr](https://github.com/cmbruns/pyopenxr/)
+    - for the VR headset and the controllers
+    - `pip install pyopenxr`
+- [pynput](https://pypi.org/project/pynput/)
+    - to listen to keyboard press
+    - `pip install pynput`
+- [colorama](https://pypi.org/project/colorama/)
+    - for pretty-printing
+    - `pip install colorama`
+- [matplotlib](https://pypi.org/project/matplotlib/)
+    - for real-time performance plots
+    - `pip install matplotlib`
+- [numpy](https://pypi.org/project/numpy/)
+    - for fingers motion retargeting
+    - `pip install numpy`
+
+You can install all of these dependencies at once by executing this command while being in the repository directory:
+```sh
+$ pip install -r requirements.txt
+```
+
+## How to use
+### CoppeliaSim simulation
+1. Open the [scene](<assets/CoppeliaSim scene.ttt>) in CoppeliaSim.
+1. Open the WEART Middleware and connect your TouchDIVER.
+1. Change the options in [`simulator.py`](simulator.py) so they match your setup.
+1. Launch the [`simulator.py`](simulator.py) python file.
+
+### MuJoCo simulation
+1. Open the WEART Middleware and connect your TouchDIVERs.
+1. Wear your TouchDIVERs in the same way as in the following picture:  
+    ![](assets/weart-wear.jpg)
+    > [!IMPORTANT]
+    > Make sure to wear the Oculus Touch Controllers in the same position as in the picture  (facing the exterior of the hands).
+1. Connect your VR device and launch the runtime program (Meta Quest Link for instance).
+1. Change the options in [`simulator.py`](simulator.py) so they match your setup.
+1. Launch the [`simulator.py`](simulator.py) python file.
+
+## Platforms
+The project is currently written in pure Python code and depends on cross-platform libraries. It is therefore also cross-platform.  
+However, to use a TouchDIVER, the *WEART Middleware* must be opened and this software is Windows-only. There is fortunately a workaround in the following section.
+
+### Use on non-Windows platforms
+If you have a secondary computer with Windows installed, it is possible to run the simulation on a primary (Linux for example) computer:
+1. Launch CoppeliaSim on the primary computer
+1. Launch WEART Middleware on the Windows computer
+1. Link the primary and the secondary computers and allow the primary one to interact with the loopback-bound 13031 TCP port
+    - Either by putting them in the same network and using a tool like *socat* to redirect the TCP port
+    - Either by using [ZeroTierOne](https://github.com/zerotier/ZeroTierOne) as an easy solution (untested, and you will still need to setup the TCP redirection)
+    - Either by setting up tunnels if there is no way to do the above
+        - This depends on your network configuration.
+        - If both computers are "hidden" from each other, for instance between two NATs, you can use a third publicly accessible machine (for instance a cloud VPS) as a relay. Setup a reverse SSH tunnel between the Windows and the relay which exposes the port 13031 to a public one, and make the Linux client connect to the relay's port.
+        - `ssh -N -4 -R <relay-port>:localhost:13031 <user>@<relay address>` (the `-4` switch is necessary on Windows, see [this issue](https://github.com/PowerShell/Win32-OpenSSH/issues/1265))
+
+Use-case: CoppeliaSim runs *much* faster on Linux.
+
+## MuJoCo simulation scene
+You can plug any simulation scene you want. The only requirements are:
+- the hands mocap bodies must follow the names `{side}_hand_mocap`, where `{side}` is _left_ or _right_.
+- the hands "real" bodies must follow the names `{side}_hand` and their rotations must be expressed with the `euler` parameter.
+- the fingertip sensors must be of type `contact` and follow the names `{side}_fingertip_{finger}`, where `{finger}` is _thumb_, _index_ or _middle_.
+
+# Cite this project
+A conference paper has been submitted here: https://inria.hal.science/hal-04733238v1. You can cite it like this:
+```bibtex
+@inproceedings{desantis:hal-04733238,
+  TITLE = {{Haptic interaction with virtual deformable objects}},
+  AUTHOR = {de Santis, Emanuele and Le Jeune, Youenn and Marchal, Maud and Pacchierotti, Claudio and Vendittelli, Marilena},
+  URL = {https://inria.hal.science/hal-04733238},
+  BOOKTITLE = {{I-RIM 3D 2024 -Conference on  Italian Institute of Robotics and Intelligent Machines}},
+  ADDRESS = {Rome, Italy},
+  PAGES = {1-3},
+  YEAR = {2024},
+  MONTH = Oct,
+  KEYWORDS = {Simulation ; Soft material ; Virtual Reality ; Haptic rendering ; Motion retargeting},
+  PDF = {https://inria.hal.science/hal-04733238v1/file/Extended_Abstract_I_RIM_3D_2024.pdf},
+  HAL_ID = {hal-04733238},
+  HAL_VERSION = {v1},
+}
+```