[RFC]: User Telemetry in TensorRT LLM #11659
Description
Motivation.
The trtllm team is considering adding anonymous usage data collection, to help us better understand what parts of the product are used/how they're used to inform future prioritization.
Proposed Change.
It would be fairly similar to what vllm describes here: (https://docs.vllm.ai/en/stable/usage/usage_stats/).
It would be opt-outable by CLI, config, or env variable. The data would be aggregated locally on some time scale (~10 minutes) before being sent to the telemetry endpoint. The data types that have been discussed are:
- uptime of deployment
- number of GPUs/nodes
- GPU type
- model architecture if it's a known one
- ingress point into trtllm (ex LLM API vs CLI)
- features used (quantization, parallelism, speculative decoding, P-D Dissag, LoRA)
Some important aspects:
- No identifying information, persistent identifiers, prompts, or model outputs will be collected
- Easy opt-out
- CI enforcement to avoid unintentional changes to data collection
I'm wondering what initial impressions of this are:
- Is it something that you think you would leave on, opt-out but not mind that it was added, or would it be problematic even if you opt out?
- Are there any of the data types that you would prefer not be collected or any others you would want the trtllm team to have insight into?
- Any other thoughts/questions/concerns
Thanks for your thoughts on this!
Feedback Period.
No response
CC List.
No response
Any Other Things.
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.