Open
Conversation
* fix: renaming of uploader to importer
Co-authored-by: Steven Jacobs <steven.jacobs@neuracore.com>
* feat: add local training metadata for local runs
Co-authored-by: favour-neuraco <favour@neurco.com>
Removed note about the 'main' branch being a development branch.
Contributor
|
✅ PR source branch is valid
|
sdas-neuraco
added
the
version:patch
sdas-neuraco
force-pushed
the
feat/add-model-docs
branch
from
6552ee5 to
0d56471
Compare
sdas-neuraco
requested review from
kwangneuraco,
mark-neuracore and
stepjam
and removed request for
kwangneuraco
Contributor
There was a problem hiding this comment.
Both Franka and Kuka has 7 Dofs, so using Franka here is not appropriate, using UR-5 is good because it has 6 DoF
| It usually begins with a single robot. You train a model on a Franka Emika Panda, define the input tensors, fix the output dimensionality, and everything works as expected. The model converges, inference is stable, and the system feels clean and well-structured. At this stage, the architecture appears robot-agnostic — but in reality, it is tightly coupled to one embodiment. | ||
|
|
||
| Then a second dataset is introduced, perhaps from a KUKA LBR iiwa. On the surface, the robots are similar: both are 7-DoF manipulators with wrist joints and RGB inputs. But under the hood, the differences begin to surface. Joint names differ. One robot has an extra joint. Camera placements are not identical. The ordering of data points in the logs does not match. Even wrist joint data may be represented differently. | ||
|
|
Contributor
There was a problem hiding this comment.
same as the pic, a bit confused, why one robot has an extra robot
| ## What Is a Cross Embodiment Description? | ||
|
|
||
| ```python | ||
| EmbodimentDescription = dict[DataType, dict[int, str]] |
|
|
||
| An `EmbodimentDescription` defines what data points exist for one robot and where each datapoint lives in index space. | ||
|
|
||
| Each `DataType` (for example `JOINTS`, `RGB_IMAGES`, `GRIPPER_JOINT_OPEN_AMOUNT`) maps to: |
Contributor
There was a problem hiding this comment.
JOINT_POSITION? I feel it is better we can strictly match names in the doc to the code
docs/model_construction.md
Outdated
| 0: "panda_joint_1", | ||
| 1: "panda_joint_2", | ||
| 2: "panda_joint_3", | ||
| 3: "panda_joint_4", |
Contributor
There was a problem hiding this comment.
I find it be better to put kuka first and franka second, otherwise the reader may feel confused why there is a missing without too much context
| Different robots can have different numbers of joints, but training requires a **consistent tensor shape**. | ||
| In this example, the wrist joint value stays at its globally assigned index, even when some intermediate indices are empty for a given robot. | ||
|
|
||
| If one robot has fewer joints, Neuracore fills missing dimensions with `0` so all records align to the same width. |
Contributor
There was a problem hiding this comment.
To be more precise, it is 0.0, float
* refactor: encapsulate logic in main function * chore: example script renamed and docs updated --------- Co-authored-by: Gabriele Tiboni <gabriele.tiboni@neuraco.com>
… dataset upload and all relevant tests (#400)
… and stopped recordings (#421)
Co-authored-by: Gabriele Tiboni <gabriele.tiboni@neuraco.com>
Co-authored-by: StevenJacobs61 <stevenjacobs61@gamil.com>
|
|
||
| Then a second dataset is introduced, perhaps from a KUKA LBR iiwa. On the surface, the robots are similar: both are 7-DoF manipulators with wrist joints and RGB inputs. But under the hood, the differences begin to surface. Joint names differ. One robot has an extra joint. Camera placements are not identical. The ordering of data points in the logs does not match. Even wrist joint data may be represented differently. | ||
|
|
||
| At first, the solution is incremental. You reorder tensors. You insert padding. You add a mapping layer. You write conditionals in the inference code. The pipeline still runs. But now structure has started leaking. The model input definition is no longer cleanly separated from embodiment logic. |
Co-authored-by: Steven Jacobs <steven.jacobs@neuracore.com>
Co-authored-by: Cougar Tasker <cougar@neuraco.com>
* docs: cleaner bigym installation instructions * chore: import try clause and cleaner comments --------- Co-authored-by: Gabriele Tiboni <gabriele.tiboni@neuraco.com>
stepjam
force-pushed
the
develop
branch
3 times, most recently
from
3982231 to
e37ad91
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Features