Is there an existing issue / discussion for this? | 是否已有关于该错误的issue或讨论？
Is there an existing answer for this in tutorial? | 该问题是否在教程中有解答？
Current Behavior | 当前行为
I am having trouble reproducing the PatchTST results on the PEMS08 dataset. Here are the details:
Version Discrepancy for Long-term Forecasting: Under the same hyperparameter settings, the performance of version 1.0 is significantly worse than that of version 0.5.8.
Result Comparison on v0.5.8: On version 0.5.8, I obtained the following metrics: "MAE": 19.94, "RMSE": 34.02, "WAPE": 0.1121. However, the paper "Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis" reports "MAE": 19.86, "RMSE": 33.44, "WAPE": 0.0834.
Observation: While my reproduced MAE and RMSE are relatively close to the paper's values, my WAPE is significantly higher. I have also noticed similar high WAPE issues when testing other models.
Could anyone provide suggestions on how to lower the WAPE to be more in line with the reported results?
Expected Behavior | 期望行为
Could anyone provide suggestions on how to lower the WAPE to be more in line with the reported results?
Environment | 运行环境
- OS:
- DEVICE:
- NVIDIA Driver:
- CUDA:
- NVIDIA GPU Memory:
- PyTorch:
BasicTS logs | BasicTS日志
Result : [test/time: 13.21 (s), test/loss: 19.9398, test/MAE: 19.9398, test/MAPE: 0.1631, test/RMSE: 34.0181, test/WAPE: 0.1121]
Steps To Reproduce | 复现方法
No response
Anything else? | 备注
No response
Is there an existing issue / discussion for this? | 是否已有关于该错误的issue或讨论？
Is there an existing answer for this in tutorial? | 该问题是否在教程中有解答？
Current Behavior | 当前行为
I am having trouble reproducing the PatchTST results on the PEMS08 dataset. Here are the details:
Version Discrepancy for Long-term Forecasting: Under the same hyperparameter settings, the performance of version 1.0 is significantly worse than that of version 0.5.8.
Result Comparison on v0.5.8: On version 0.5.8, I obtained the following metrics: "MAE": 19.94, "RMSE": 34.02, "WAPE": 0.1121. However, the paper "Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis" reports "MAE": 19.86, "RMSE": 33.44, "WAPE": 0.0834.
Observation: While my reproduced MAE and RMSE are relatively close to the paper's values, my WAPE is significantly higher. I have also noticed similar high WAPE issues when testing other models.
Could anyone provide suggestions on how to lower the WAPE to be more in line with the reported results?
Expected Behavior | 期望行为
Could anyone provide suggestions on how to lower the WAPE to be more in line with the reported results?
Environment | 运行环境
BasicTS logs | BasicTS日志
Result : [test/time: 13.21 (s), test/loss: 19.9398, test/MAE: 19.9398, test/MAPE: 0.1631, test/RMSE: 34.0181, test/WAPE: 0.1121]
Steps To Reproduce | 复现方法
No response
Anything else? | 备注
No response