Summary
Adapt an evaluation script for Yasoz/UniTraj based on the evaluate_downstream.py from 24daydayup/UniTraj, in order to properly evaluate the performance on trajectory prediction and completion tasks.
Background
- 24daydayup/UniTraj uses
evaluate_downstream.py to evaluate MAE/RMSE for two tasks: trajectory prediction (预测) and completion (补全),using batch-wise haversine distance calculation and compatible with their data/preprocessing format.
- Yasoz/UniTraj has its own transformer-based implementation (in
utils/unitraj.py) and data pipeline (in utils/dataset.py).
- Currently, Yasoz repo lacks a corresponding downstream evaluation script that can (1) mask points in the input, (2) denormalize/restore lat/lon, and (3) calculate metrics compatible with convention (meters, prediction/completion mode etc).
What needs to be done
- Create
evaluate_downstream.py (or equivalent) in Yasoz/UniTraj, referencing the logic and metrics from 24daydayup/UniTraj:
- Support both prediction (mask last N) and completion (mask arbitrary indices) evaluation modes.
- Compatible with Yasoz/UniTraj's UniTraj model, data format, and normalization stats.
- Batchwise metric (MAE, RMSE, meters, using haversine), support for torch DataLoader.
- Output/save results and logs using Yasoz's logger or standard output.
- Sample usage:
- Load dataset via Yasoz-style TrajectoryDataset
- Restore model and normalizer config
- Evaluate model for both prediction/completion
Reference
Acceptance criteria
- Script can evaluate both prediction and completion tasks with proper masking scheme
- Metric output matches 24daydayup/UniTraj when using same model/dataset
- Results are logged/saved
- Short run/usage instruction included as docstring or comments
If desired, label as documentation (since it's mostly script adaptation + usage guidance)
Summary
Adapt an evaluation script for Yasoz/UniTraj based on the evaluate_downstream.py from 24daydayup/UniTraj, in order to properly evaluate the performance on trajectory prediction and completion tasks.
Background
evaluate_downstream.pyto evaluate MAE/RMSE for two tasks: trajectory prediction (预测) and completion (补全),using batch-wise haversine distance calculation and compatible with their data/preprocessing format.utils/unitraj.py) and data pipeline (inutils/dataset.py).What needs to be done
evaluate_downstream.py(or equivalent) in Yasoz/UniTraj, referencing the logic and metrics from 24daydayup/UniTraj:Reference
Acceptance criteria
If desired, label as
documentation(since it's mostly script adaptation + usage guidance)