Skip to content

Add evaluation script for trajectory prediction and completion tasks (Yasoz UniTraj) #4

Description

@24daydayup

Summary

Adapt an evaluation script for Yasoz/UniTraj based on the evaluate_downstream.py from 24daydayup/UniTraj, in order to properly evaluate the performance on trajectory prediction and completion tasks.

Background

  • 24daydayup/UniTraj uses evaluate_downstream.py to evaluate MAE/RMSE for two tasks: trajectory prediction (预测) and completion (补全),using batch-wise haversine distance calculation and compatible with their data/preprocessing format.
  • Yasoz/UniTraj has its own transformer-based implementation (in utils/unitraj.py) and data pipeline (in utils/dataset.py).
  • Currently, Yasoz repo lacks a corresponding downstream evaluation script that can (1) mask points in the input, (2) denormalize/restore lat/lon, and (3) calculate metrics compatible with convention (meters, prediction/completion mode etc).

What needs to be done

  1. Create evaluate_downstream.py (or equivalent) in Yasoz/UniTraj, referencing the logic and metrics from 24daydayup/UniTraj:
    • Support both prediction (mask last N) and completion (mask arbitrary indices) evaluation modes.
    • Compatible with Yasoz/UniTraj's UniTraj model, data format, and normalization stats.
    • Batchwise metric (MAE, RMSE, meters, using haversine), support for torch DataLoader.
    • Output/save results and logs using Yasoz's logger or standard output.
  2. Sample usage:
    • Load dataset via Yasoz-style TrajectoryDataset
    • Restore model and normalizer config
    • Evaluate model for both prediction/completion

Reference

Acceptance criteria

  • Script can evaluate both prediction and completion tasks with proper masking scheme
  • Metric output matches 24daydayup/UniTraj when using same model/dataset
  • Results are logged/saved
  • Short run/usage instruction included as docstring or comments

If desired, label as documentation (since it's mostly script adaptation + usage guidance)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions