Skip to content

feat: massively expand functionality on assets and runs#527

Draft
drake-nominal wants to merge 6 commits intomainfrom
deidukas/expand-assets
Draft

feat: massively expand functionality on assets and runs#527
drake-nominal wants to merge 6 commits intomainfrom
deidukas/expand-assets

Conversation

@drake-nominal
Copy link
Copy Markdown
Collaborator

@drake-nominal drake-nominal commented Nov 19, 2025

Overview

Make assets and runs way more powerful. Instead of needing to handle runs and asset objects, or even think about dataset objects, now users can work with mostly just assets. A typical ingestion flow would now look like:

asset = client.get_or_create_asset_by_properties(...)
asset.get_or_create_dataset("data_scope", ...)

dataset_file = asset.add_tabular_data("data_scope", "path/to.csv", ...)
dataset_file.poll_until_ingestion_completed()

run = asset.create_run("run name", start=dataset_file.start, end=dataset_file.end)
run.create_workbook(client.get_workbook_template("rid..."))

A future change I want to make is to allow users to pass a Run into add_tabular_data so that ingest will update the run bounds upon ingestion completion. I also want to allow users to poll_until_ingestion_completed() on both an Asset and Run-- also left as future work.

Another future change I would like to make is exposing a as_completed() variant for dataset files, and to also make it easier to retrieve dataset files for runs, assets (e.g. get_dataset_files("data_scope_name", start_time=..., end_time=...)).

Deprecations

  • NominalClient.search_runs_by_asset => Asset.search_runs

Addditions to NominalClient

  • Can now create a run with exactly one of {no asset, 1 asset, or many assets}. New keyword argument assets should be preferred over asset, with the former potentially deserving deprecation.

Additions to Asset

  • add_data utilities that take in datascope name & parameters that would normally be passed to their equivalents in the Dataset class:
    • add_tabular_data
    • add_journal_json
    • add_mcap
    • add_ardupilot_dataflash
    • add_containerized
    • add_from_io
  • promote: promote an implicitly created asset from a run into a full on asset
  • create_run
  • create_event
  • create_workbook
  • list_runs

Additions to Run

  • add_data utilities that take in datascope name & parameters that would normally be passed to their equivalents in the Dataset class:
    • add_tabular_data
    • add_journal_json
    • add_mcap
    • add_ardupilot_dataflash
    • add_containerized
    • add_from_io
  • get_or_create_dataset
  • get_or_create_video
  • create_event: targets all assets present on a run
  • create_workbook
  • get_dataset
  • get_video
  • get_connection
  • unarchive

Comment thread nominal/core/client.py Outdated
Comment thread nominal/core/client.py Outdated
Comment thread nominal/core/client.py Outdated
Comment thread nominal/core/event.py Outdated
Comment thread nominal/core/run.py Outdated
@drake-nominal drake-nominal force-pushed the deidukas/feat-add-refresh-mixin branch from f3ca2b8 to a9b2044 Compare November 19, 2025 20:16
@drake-nominal drake-nominal changed the base branch from deidukas/feat-add-refresh-mixin to deidukas/reorder-on-top-of-refresh November 19, 2025 20:40
@drake-nominal drake-nominal force-pushed the deidukas/reorder-on-top-of-refresh branch from db092b1 to 620d0e3 Compare November 19, 2025 22:22
@drake-nominal drake-nominal changed the base branch from deidukas/reorder-on-top-of-refresh to deidukas/chore-reorder-asset-run November 19, 2025 22:24
@drake-nominal drake-nominal changed the base branch from deidukas/chore-reorder-asset-run to deidukas/reorder-on-top-of-refresh November 19, 2025 22:25
Base automatically changed from deidukas/reorder-on-top-of-refresh to main December 1, 2025 20:24
@drake-nominal drake-nominal force-pushed the deidukas/expand-assets branch 4 times, most recently from 0df4358 to 75562b5 Compare December 4, 2025 19:41
@drake-nominal drake-nominal force-pushed the deidukas/expand-assets branch from 028a1fa to 13b1e38 Compare December 4, 2025 20:08
@drake-nominal drake-nominal force-pushed the deidukas/expand-assets branch from 13b1e38 to af60479 Compare December 4, 2025 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants