Product

BAML

Problem Statement / Use Case

https://discord.com/channels/1119368998161752075/1253172277449855029/1396768470540095508

Cached tokens was discussed here in Discord but it requires tapping in to underlying HTTP response

This breaks the abstraction

We should track cached input tokens on a normalised basis across providers

Does not seem to have an issue to track this

Proposed Solution

Count cached input tokens

Alternative Solutions

No response

Additional Context

No response