How to analyze etdump results for QNN backend?

[liu-mengyang]

I have obtained etdump results from a QNN‑LLM execution and used the Inspector API to generate the profiling table. However, I’m not sure how to interpret this table. Specifically, I have the following questions:

1. After sorting the results, I see Accelerator (execute) time (cycles), Method::execute, and DELEGATE_CALL at the top. Their latencies are quite similar. Are these essentially representing the same thing?
2. The remaining entries all start with aten_, which I assume correspond to different operators. How can I determine which of these are running on the CPU due to fallback, and which are actually running on the NPU?
3. Is there a way to know whether each operator uses HMX or HVX inside the NPU?
4. Finally, can I extract more detailed profiling information such as memory consumption, TCM usage, or overall NPU utilization?

Any guidance or documentation pointers would be greatly appreciated. Thank you!

[yujiaoliang]

For QNN profiling, I would mainly look at two aspects:

• QNN runtime profiling itself already provides useful information about NPU execution and TCM usage. By enabling the built-in runtime profiling options, you can inspect per-op execution time and memory behavior, which is usually the primary source for understanding performance and resource utilization.

• As an additional reference, this document might be helpful:
  https://github.com/pytorch/executorch/blob/main/backends/qualcomm/debugger/README.md

It describes some related debugging and profiling workflows around the Qualcomm backend, though you may already be familiar with it.

[cccclai]

After sorting the results, I see Accelerator (execute) time (cycles), Method::execute, and DELEGATE_CALL at the top. Their latencies are quite similar. Are these essentially representing the same thing?

Delegate_CALL is inside Method::execute call. If the time is similar, that means the delegate call (the execution time in HTP) is dominant.

The remaining entries all start with aten_, which I assume correspond to different operators. How can I determine which of these are running on the CPU due to fallback, and which are actually running on the NPU?

DELEGATE_CALL is everything inside HTP, and the individual operator call (like aten_bmm) means they fall back to cpu.

Is there a way to know whether each operator uses HMX or HVX inside the NPU?

We can use optrace or the debugger https://github.com/pytorch/executorch/tree/main/backends/qualcomm/debugger#qairt-visualizer as shared by @yujiaoliang

Finally, can I extract more detailed profiling information such as memory consumption, TCM usage, or overall NPU utilization?

@haowhsu-quic @shewu-quic @winskuo-quic @DannyYuyang-quic do we have guidance on this?

[shewu-quic]

Hi @yujiaoliang

Finally, can I extract more detailed profiling information such as memory consumption, TCM usage, or overall NPU utilization?

Yes, you can dump QHAS and optrace to observe the NPU utilization and TCM usage.:
https://github.com/pytorch/executorch/tree/main/backends/qualcomm/debugger#2-generate-optrace-and-qhas