Skip to content

feat(semcov): Add Image Token Count and Cost Semantic Conventions #1787

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat(semcov): Add Image Token Count and Cost Semantic Conventions #1787

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1d05ce4
add image semantic conventions
sallyannarize Jun 16, 2025
43ff2d8
updates
sallyannarize Jun 18, 2025
256d982
fix sem conv
sallyannarize Jun 19, 2025
08f6583
Update js/packages/openinference-semantic-conventions/src/trace/Seman…
sallyannarize Jun 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions js/.changeset/long-terms-knock.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@arizeai/openinference-semantic-conventions": major
---

Add image semantic conventions
28 changes: 25 additions & 3 deletions js/packages/openinference-semantic-conventions/src/trace/SemanticConventions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,10 @@ export const LLM_TOKEN_COUNT_COMPLETION_DETAILS_REASONING =
export const LLM_TOKEN_COUNT_COMPLETION_DETAILS_AUDIO =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.completion_details.audio` as const;

/** Token count for image tokens generated by the model (in tokens) */
export const LLM_TOKEN_COUNT_COMPLETION_DETAILS_IMAGE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.completion_details.image` as const;

/** Token count for the prompt to the llm (in tokens) */
export const LLM_TOKEN_COUNT_PROMPT =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.prompt` as const;
Expand All @@ -229,6 +233,10 @@ export const LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_READ =
export const LLM_TOKEN_COUNT_PROMPT_DETAILS_AUDIO =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.prompt_details.audio` as const;

/** Token count for image input presented in the prompt (in tokens) */
export const LLM_TOKEN_COUNT_PROMPT_DETAILS_IMAGE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.prompt_details.image` as const;

/** Token count for the entire transaction with the llm (in tokens) */
export const LLM_TOKEN_COUNT_TOTAL =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.token_count}.total` as const;
Expand Down Expand Up @@ -258,14 +266,16 @@ export const LLM_TOKEN_COUNT_COMPLETION_DETAILS =
* "completion_details": {
* "output": 0.0009, # Cost in USD
* "reasoning": 0.0024, # Cost in USD (e.g., 80 tokens * $0.03/1K tokens)
* "audio": 0.0012 # Cost in USD (e.g., 40 tokens * $0.03/1K tokens)
* "audio": 0.0012, # Cost in USD (e.g., 40 tokens * $0.03/1K tokens)
* "image": 0.0018 # Cost in USD (e.g., 60 tokens * $0.03/1K tokens)
* },
* "prompt_details": {
* "input": 0.0003, # Cost in USD
* "cache_write": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
* "cache_read": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
* "cache_input": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
* "audio": 0.0003 # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
* "audio": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
* "image": 0.0009 # Cost in USD (e.g., 30 tokens * $0.03/1K tokens)
* }
* }
* Note: This is a key prefix - individual attributes are stored as separate span attributes with this prefix,
Expand Down Expand Up @@ -306,6 +316,10 @@ export const LLM_COST_COMPLETION_DETAILS_REASONING =
export const LLM_COST_COMPLETION_DETAILS_AUDIO =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.completion_details.audio` as const;

/** Cost of image tokens in the completion in USD */
export const LLM_COST_COMPLETION_DETAILS_IMAGE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.completion_details.image` as const;

/** Cost of prompt tokens written to cache in USD */
export const LLM_COST_PROMPT_DETAILS_CACHE_WRITE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.prompt_details.cache_write` as const;
Expand All @@ -322,6 +336,10 @@ export const LLM_COST_PROMPT_DETAILS_CACHE_INPUT =
export const LLM_COST_PROMPT_DETAILS_AUDIO =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.prompt_details.audio` as const;

/** Cost of image tokens in the prompt in USD */
export const LLM_COST_PROMPT_DETAILS_IMAGE =
`${SemanticAttributePrefixes.llm}.${LLMAttributePostfixes.cost}.prompt_details.image` as const;

/**
* The role that the LLM assumes the message is from
* during the LLM invocation
Expand Down Expand Up @@ -638,11 +656,13 @@ export const SemanticConventions = {
LLM_TOKEN_COUNT_COMPLETION_DETAILS,
LLM_TOKEN_COUNT_COMPLETION_DETAILS_REASONING,
LLM_TOKEN_COUNT_COMPLETION_DETAILS_AUDIO,
LLM_TOKEN_COUNT_COMPLETION_DETAILS_IMAGE,
LLM_TOKEN_COUNT_PROMPT,
LLM_TOKEN_COUNT_PROMPT_DETAILS,
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_WRITE,
LLM_TOKEN_COUNT_PROMPT_DETAILS_CACHE_READ,
LLM_TOKEN_COUNT_PROMPT_DETAILS_AUDIO,
LLM_TOKEN_COUNT_PROMPT_DETAILS_IMAGE,
LLM_TOKEN_COUNT_TOTAL,
LLM_SYSTEM,
LLM_PROVIDER,
Expand All @@ -655,10 +675,12 @@ export const SemanticConventions = {
LLM_COST_OUTPUT,
LLM_COST_COMPLETION_DETAILS_REASONING,
LLM_COST_COMPLETION_DETAILS_AUDIO,
LLM_COST_COMPLETION_DETAILS_IMAGE,
LLM_COST_PROMPT_DETAILS_CACHE_WRITE,
LLM_COST_PROMPT_DETAILS_CACHE_READ,
LLM_COST_PROMPT_DETAILS_CACHE_INPUT,
LLM_COST_PROMPT_DETAILS_AUDIO,
LLM_COST_PROMPT_DETAILS_IMAGE,
MESSAGE_ROLE,
MESSAGE_NAME,
MESSAGE_TOOL_CALLS,
Expand Down Expand Up @@ -747,4 +769,4 @@ export enum LLMProvider {
GOOGLE = "google",
AWS = "aws",
AZURE = "azure",
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file is missing a trailing newline character at the end. Adding a newline at the end of files is standard practice for POSIX compliance and helps prevent issues with certain tools and version control systems. Consider adding a newline character to maintain consistency with standard file formatting conventions.

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

47 changes: 43 additions & 4 deletions python/openinference-semantic-conventions/src/openinference/semconv/trace/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ class SpanAttributes:
"""
Number of tokens used for reasoning steps in the completion (in tokens).
"""
LLM_TOKEN_COUNT_COMPLETION_DETAILS_IMAGE = "llm.token_count.completion_details.image"
"""
The number of image tokens in the completion (in tokens).
"""
LLM_TOKEN_COUNT_PROMPT = "llm.token_count.prompt"
"""
Number of tokens in the prompt.
Expand All @@ -108,6 +112,10 @@ class SpanAttributes:
"""
Number of tokens in the prompt that were written to cache (in tokens).
"""
LLM_TOKEN_COUNT_PROMPT_DETAILS_IMAGE = "llm.token_count.prompt_details.image"
"""
The number of image tokens in the prompt (in tokens).
"""
LLM_TOKEN_COUNT_TOTAL = "llm.token_count.total"
"""
Total number of tokens, including both prompt and completion (in tokens).
Expand Down Expand Up @@ -138,10 +146,37 @@ class SpanAttributes:
"""
Cost of reasoning steps in the completion in USD.
"""
LLM_COST_PROMPT = "llm.cost.prompt"
"""
Total cost of all input tokens sent to the LLM in USD. This includes all tokens that were
processed as part of the prompt, including system messages, user messages, and any other input.
LLM_COST_COMPLETION_DETAILS_IMAGE = "llm.cost.completion_details.image"
"""
Cost of image tokens in the completion in USD.
"""
LLM_COST = "llm.cost"
"""
Key prefix for cost information. When these keys are transformed into a JSON-like structure,
it would look like:
{
"prompt": 0.0021, # Cost in USD
"completion": 0.0045, # Cost in USD
"total": 0.0066, # Cost in USD
"input": 0.0003, # Cost in USD
"output": 0.0009, # Cost in USD
"completion_details": {
"reasoning": 0.0024, # Cost in USD (e.g., 80 tokens * $0.03/1K tokens)
"audio": 0.0012, # Cost in USD (e.g., 40 tokens * $0.03/1K tokens)
"image": 0.0018 # Cost in USD (e.g., 60 tokens * $0.03/1K tokens)
},
"prompt_details": {
"cache_write": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
"cache_read": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
"cache_input": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
"audio": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
"image": 0.0009 # Cost in USD (e.g., 30 tokens * $0.03/1K tokens)
}
}
Note: This is a key prefix - individual attributes are stored as separate span attributes
with this prefix, e.g. llm.cost.prompt, llm.cost.completion_details.reasoning, etc.
The JSON structure shown above represents how these separate attributes can be conceptually
organized. All monetary values are in USD with floating point precision.
"""
LLM_COST_PROMPT_DETAILS = "llm.cost.prompt_details"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comments here about LLM_COST_PROMPT_DETAILS

"""
Expand Down Expand Up @@ -171,6 +206,10 @@ class SpanAttributes:
input to the model, which may be different from the prompt cost if there are additional
processing steps.
"""
LLM_COST_PROMPT_DETAILS_IMAGE = "llm.cost.prompt_details.image"
"""
Cost of image tokens in the prompt in USD.
"""
LLM_COST_TOTAL = "llm.cost.total"
"""
Total cost of the LLM call in USD (prompt + completion).
Expand Down
10 changes: 8 additions & 2 deletions spec/semantic_conventions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ The following attributes are reserved and MUST be supported by all OpenInference
| `llm.token_count.prompt_details.cache_read` | Integer | `5` | The number of tokens read from previously cached prompts |
| `llm.token_count.prompt_details.cache_write` | Integer | `0` | The number of tokens written to cache |
| `llm.token_count.prompt_details.audio` | Integer | `10` | The number of audio input tokens presented in the prompt |
| `llm.token_count.prompt_details.image` | Integer | `258` | The number of image input tokens presented in the prompt |
| `llm.token_count.completion_details.image` | Integer | `15` | The number of image output tokens generated by the model |
| `llm.token_count.total` | Integer | `20` | Total number of tokens, including prompt and completion |
| `llm.cost.prompt` | Float | `0.0021` | Total cost of all input tokens sent to the LLM in USD |
| `llm.cost.completion` | Float | `0.0045` | Total cost of all output tokens generated by the LLM in USD |
Expand All @@ -53,6 +55,8 @@ The following attributes are reserved and MUST be supported by all OpenInference
| `llm.cost.prompt_details.cache_read` | Float | `0.0003` | Cost of prompt tokens read from cache in USD |
| `llm.cost.prompt_details.cache_input` | Float | `0.0006` | Cost of input tokens in the prompt that were cached in USD |
| `llm.cost.prompt_details.audio` | Float | `0.0003` | Cost of audio tokens in the prompt in USD |
| `llm.cost.prompt_details.image` | Float | `0.0009` | Cost of image tokens in the prompt in USD |
| `llm.cost.completion_details.image` | Float | `0.0018` | Cost of image tokens in the completion in USD |
| `llm.tools` | List of objects<sup>†</sup> | `[{"tool.name": "calculate", "tool.json_schema": "{}"}, ...]` | List of tools that are advertised to the LLM to be able to call |
| `message.content` | String | `"What's the weather today?"` | The content of a message in a chat |
| `message.contents` | List of objects<sup>†</sup> | `[{"message_content.type": "text", "message_content.text": "Hello"}, ...]` | The message contents to the llm, it is an array of `message_content` objects. |
Expand Down Expand Up @@ -144,14 +148,16 @@ The `llm.cost` prefix is used to group cost-related attributes. When these keys
"completion_details": {
"output": 0.0009, # Cost in USD
"reasoning": 0.0024, # Cost in USD (e.g., 80 tokens * $0.03/1K tokens)
"audio": 0.0012 # Cost in USD (e.g., 40 tokens * $0.03/1K tokens)
"audio": 0.0012, # Cost in USD (e.g., 40 tokens * $0.03/1K tokens)
"image": 0.0018 # Cost in USD (e.g., 60 tokens * $0.03/1K tokens)
},
"prompt_details": {
"input": 0.0003, # Cost in USD
"cache_write": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
"cache_read": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
"cache_input": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
"audio": 0.0003 # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
"audio": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
"image": 0.0009 # Cost in USD (e.g., 30 tokens * $0.03/1K tokens)
}
}
```
Expand Down
Loading