Skip to content

Interface for counting tokens #7290

@stephentoub

Description

@stephentoub

Most providers now have an API for counting tokens, i.e. submit the full request as you otherwise would, but get back a token count rather than an actual response. We should consider adding an interface for this purpose, e.g.

Task<int> CountTokensAsync(IEnumerable<ChatMessage>, ChatOptions, CancellationToken);

It could be implemented by IChatClients that support it and served up from their GetService implementations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-aiMicrosoft.Extensions.AI librariesuntriaged

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions