Uniad quantaization

[tpfvmfh]

Hello,Thansk for your great work!!
I am trying to follow uniad explict quantization
I have a couple of questions

1. After install Tensorrt model optimzier, i tried check out release/0.29.0 but there is no 0.29.0 version
2. I think there is an error on making calibration data
3. i want to run qunatized uniad model on Orin board, but i am not quite sure how to make qunatized_onnx_model
4. Thanks

[Jian-danai]

@tpfvmfh modelopt release/0.29.0 will be a future release recently. Please stay tuned.
For 2, can you provide the error log, thanks.

[gcunhase]

@tpfvmfh release/0.29.0 is available now, please try again!

[tpfvmfh]

@gcunhase
@Jian-danai
Hello,i tried to make a quantized onnx file,but there is an error...Could not find an implementation for RotateTRT(1) node with name 'RotateTRT_1477
Thanks
This is my log

2025-05-27 13:01:50.889792714 (W:onnxruntime: Default, tensorrt_execution provider.h:91 log] (2025-05-27 04:01:50 WARNING] onnx0pImporters.cpp:6521: Attribute center not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build. -05-27 13:01:52.550988836 (W:onnxruntime:Default, tensorrt execution provider.h:91 log] [2025-05-27 04:01:52 WARNING) onnxOpImporters.cpp:6521: Attribute center not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build. 2025-05 2025-05-27

13:03:49:257653995 Eonnxruntime: Default, tensorrt execution provider.h:89 log) 2025-05-27 04:03:49 ERROR) IBuilder::buildSerializedNetwork: Error Code 4: Shape Error (broadcast [E:onnxruntime:, sequential executor.cc:572 (broadcast dimensions aust be conformable) ExecuteKernel] Non-zero status code returned while running TRTKernel_graph_torch_jit_5352295870958585344 @ node. Name: TensorrtExecutionProvider TRTKernel graph_torch_jit_5352295870958585344 0 8 Status Message: Tens FrontErrorEP FAIL Non-zero status code returned while running TRTKernel graph_torch_jit 5352295870958585344 0 node. Name: 'TensorrtExecutionProvider_TRTKernel_graph_torch_jit 5352295870958585344_0 0' Status Message: TensorRT EP failed to create engine from networ back to 'CUDAExecutionProvider', 'CPUExecutionProvider') and retrying.

Falling 2025-05-27 (most recent call last)

13:04:03.824868647 (W:onnxruntime:, transformer_memcpy.cc:83 ApplyImpl] 1016 Memcpy nodes are added to the graph torch jit for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level-1 to sem

Traceback File " /home/shakr/myenv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 270, in run return self. sess.run(output_names, input feed, run options)

onnxruntime.capi.onnxruntime_pybindll_state.EPFail: [ONNXRuntimeError): 11: EP FAIL: Non-zero status code returned while running TRTKernel_graph_torch_jit_5352295870958585344_0 node. Name: 'TensorrtExecutionProvider_TRTKernel_graph_torch_jit_5352295870958585344_0 0' Status Message: Tensor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/usr/lib/python3.10/rumpy.py", line 196, in _run_module_as_main return run_code code, main globals, None,

File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run globals)

File "/home/shakr/DL4AGX/AV-Solutions/uniad-trt/TensorRT-Model-Optimizer/modelopt/onnx/quantization/main.py", line 265, in

main() File "/home/shakr/DL4AGX/AV-Solutions/uniad-trt/TensorRT-Model-Optimizer/modelopt/onnx/quantization/main.py", line 235, in main

quantize File "/home/shakr/DL4AGX/AV-Solutions/uniad-trt/TensorRT-Model-Optimizer/modelopt/onnx/quantization/quantize.py", line 363, in quantize nodes to exclude find nodes from nha to exclude

File "/home/shakr/DL4AGX/AV-Solutions/uniad-trt/TensorRT-Model-Optimizer/modelopt/onnx/quantization/graph_utils.py", line 778, in find nodes_from_mha_to_exclude output map get_extended_model_outputs(

File " outputs session.run(extended_model_output_names, inputs)

/home/shakr/DL4AGX/AV-Solutions/uniad-trt/TensorRT-Model-Optimizer/modelopt/onnx/quantization/graph_utils.py", line 636, in get_extended_model_outputs

File "/home/shakr/myenv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 275, in run

self.set providers (self. fallback_providers)

File "/home/shakr/myenv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 225, in set providers self._reset_session (providers, provider_options)

File "/home/shakr/myenv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 569, in _reset_session

self._create_inference_session(providers, provider_options)

File "/home/shakr/myenv/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 541, in _create_inference_session ess.initialize_session(providers, provider_options, disabled tions, disabled_optimizers)

I

inxruntime.capi.onnxruntime_pybind11_state.NotImplemented: [ONXRuntimeError): 9: NOT_IMPLEMENTED: Could not find an implementation for RotateTRT(1) node with name 'RotateTRT_1477

///build option
SHAPES-prev_track_intances0:901x512.prev_track_intances1:901x3,prev_track_intances3:901,prev_track_intances4:901,prev_track

LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib:/home/shakr/TensorRT-10.9 x86 cull8/lib:$LD LIBRARY PATH python3 modelopt.ond rack_intances5:901,prev_track_intances6:901,prev_track_intances8:901,prev_track_intances9:901x10,prev_track_intances11:901x4x256.prev_track_intances 12:901x4, prev_track_inta

--onnx_path=/home/shakr/ssd_1t/quant/DL4AGX/AV-Solutions/uniad-trt/UniAD/onnx/uniad_tiny_imgx0.25 cp.repaired.onnx python3. modelopt.onnx.quantization \

--trt_plugins=/home/shakr/ssd1t/new_quant/DL4AGX/AV-Solutions/uniad-trt/UniAD/plugins/lib/libtensorrt_ops.so

-calibration eps trt cuda: cpu -

-calibration_shapes=$(SHAPES) -

-simplify \ --op_types_to_exclude MatMul

--calibration_data_path=./calib_data_shape_901.npz

--output_path=/home/shakr/tmp.onnx --dq_only

[Jian-danai]

@tpfvmfh Did you do the quantization in the deployment docker container?
BTW, did you re-train a UniAD-tiny and export an onnx with real weights? The provided rand weights onnx will not work with our workflow, it's just for your reference.