In one of my previous articles, we discussed using Claude models in AWS Bedrock. One of the advantages of Bedrock was
the ability to log supported model
invocations made through the bedrock-runtime endpoint. The logged events are a goldmine of essential information for
monitoring how you or your users use Claude Code.
The goal of this post is to summarize the useful information you can extract from events and serve as a small reference for monitoring Claude Code usage. I understand if you only want to grasp what is possible, so I’ll start with the TL;DR.
Using Claude invocation logging, you can monitor:
identity.arn), the model
used, and a timestamp.metadata.user_id field carries device_id and a per-session
session_id, so events can be grouped into sessions.message_stop/message_delta output events report input,
output, cache-read, and cache-write token counts, as well as invocation latency and time to first byte, for every
invocation.tools field shows which native and MCP tools each invocation could call,
which matters both for security review and for diagnosing context-window pollution by overly verbose MCP servers.output_config.effort (low → max) helps explain unexpectedly high token usage.messages field contains the full conversation state: system prompts, CLAUDE.md
contents, user prompts, tool calls, and their results. This is what makes the logs powerful for analysis — and
extremely sensitive.Let’s dig into the structure of the Anthropic API calls made by Claude Code and understand how to extract this useful information from them.
You have multiple options for enabling logging, depending on your setup.
If you use the standard Claude Code setup with an Anthropic subscription, the easiest way is to run an LLM gateway locally (e.g., LiteLLM) and enable logging there.
If you use the Anthropic API via AWS Bedrock, you can enable model invocation logging. When S3 delivery is enabled, the log files are conveniently delivered to S3 as gzip-compressed JSONL files that are ready to be analyzed.
This feature is not unique to AWS Bedrock — similar capabilities are available through Google Vertex AI, Azure API Management AI gateway logs, LLM gateway logging (e.g., LiteLLM), or LLM observability tools such as Langfuse via instrumentation. You can use one of the solutions in this list, depending on your setup, to collect these logs.
AI harness logs are very sensitive. If your organization logs the use of Claude Code or other development harnesses, the logs can contain a great deal of sensitive information (leaked tokens and passwords, sensitive prompts, etc.) — everything that appears during AI coding sessions on your developers’ machines.
Note: Skip this section if you don’t use AWS.
I intentionally left out many fields that are not important for this blog post. Each request contains an ID, the model used, the identity of the caller (represented by an IAM or STS principal ARN), the input, and the output.
{
"timestamp": "2024-01-15T12:00:00Z",
"requestId": "abcd1234-5678-efgh-ijkl-mnopqrstuvwx",
"modelId": "arn:aws:bedrock:us-east-1:073865013699:inference-profile/global.anthropic.claude-haiku-4-5-20251001-v1:0",
"identity": {
"arn": "arn:aws:sts::123456789012:assumed-role/MyRole/user@mycompany.com"
},
"input": {
"inputBodyJson": { },
"inputBodyS3Path": null,
},
"output": {
"outputBodyJson": { },
"outputBodyS3Path": null,
},
...
}
Useful fields include:
timestamprequestId — for deduplication.modelId — the model ID or inference-profile ID used for the invocation. Depending on how the model was invoked,
this may be an ARN.identity.arn — identifies the role and session that called the model. Note that the session name frequently includes
the user identifier if you use IAM Identity Center for your users’ AWS logins.The input payload may be stored in the log entry itself (inputBodyJson), or, if it’s too big, it will be stored as a
JSON file in S3, and the entry will contain the path to it (inputBodyS3Path). The same applies to the output.
To conduct analysis and build data sources on top of these events, you need to ensure that inputBodyJson is always
present by augmenting it with data from inputBodyS3Path. One approach is to use a streaming process or a Lambda
function that resolves inputBodyS3Path when present and writes full-fat events back to S3.
A typical event contains input and output parts, denoting the LLM input and output. In the following sections, we’ll
go through the useful bits in the event payload.
input.system fieldFrom the system field, you can extract the following information.
Block 1 contains the Anthropic headers. It includes the version of Claude Code used to invoke the model and whether the model was invoked by a subagent.
Block 2 is always the same, based on my observations.
Block 3 contains the actual system prompt of Claude Code, describing memory configuration, Git setup, available tools (including MCP), the current branch, environment details, and recent commits.
Example system value:
[
{
"type": "text",
"text": "x-anthropic-billing-header: cc_version=2.1.178.2f1; cc_entrypoint=cli; cc_is_subagent=true;",
"cache_control": null
},
{
"type": "text",
"text": "You are Claude Code, Anthropic's official CLI for Claude.",
"cache_control": {
"type": "ephemeral"
}
},
{
"type": "text",
"text": "\nYou are an interactive agent that helps users with software engineering tasks.\n\nIMPORTANT: Assist ...",
"cache_control": {
"type": "ephemeral"
}
}
]
input.metadata fieldThe contents of this field look like this:
"metadata": {
"user_id": "{\"device_id\":\"5e71...\",\"session_id\":\"296...\"}"
},
Currently, metadata contains a user_id field with device_id and session_id.
device_id identifies the device on which the harness runs; it doesn’t change between different coding sessions.session_id is maintained within the coding session. For example, if you start a Claude Code session, all events
generated by it will contain the same session_id.input.output_config fieldExample:
"output_config": {"effort": "high"}
This field contains information about reasoning effort (low/medium/high/xhigh/max). It may be useful for debugging high token usage by some AI agents.
input.tools fieldThe tools field contains the loaded native and MCP tools that the agent can call. It describes which tool schemas were
loaded into this particular model invocation, not which MCP servers or tools were connected (see the messages field
and its system role).
Using this field, you can:
Here’s what the event looks like (I included only one native tool and one custom MCP tool, and trimmed the descriptions):
{
"tools": [
# Anthropic native tools.
# Only "Bash" is included in this example.
{
"name": "Bash",
"description": "Executes a Bash command and returns...",
"input_schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"command": {
"description": "The command to execute.",
"type": "string",
},
"timeout": {
"description": (
"Optional timeout in milliseconds, up to 600000."
),
"type": "number",
},
"description": {
"description": "A clear, concise description of...",
"type": "string",
},
"run_in_background": {
"description": (
"Set to True to run the command in the background."
),
"type": "boolean",
},
"dangerouslyDisableSandbox": {
"description": (
"Set to True to override sandbox mode and run "
"commands without sandboxing."
),
"type": "boolean",
},
},
"required": ["command"],
"additionalProperties": False,
},
"eager_input_streaming": True,
},
# Optionally loaded MCP tools.
# This example uses a Databricks MCP tool.
{
"name": "mcp__databricks__execute_sql",
"description": (
"Executes SQL queries in Databricks Unity Catalog. "
"This tool executes SQL..."
),
"input_schema": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": (
"The SQL query to execute. Use fully qualified "
"Unity Catalog names (catalog.schema.table) "
"for best results."
),
},
},
"required": ["query"],
},
"eager_input_streaming": True,
"defer_loading": True,
},
],
}
In Claude Code 2.1.178, with my configuration, I observed the following initial root-agent tool set:
Agent
AskUserQuestion
Bash
Edit
Read
Skill
ToolSearch
Workflow
Write
The subagents I inspected received this reduced tool set:
Bash
Read
Skill
ToolSearch
Additional tools are loaded on demand.
At the beginning of each coding session, Claude Code receives the list of built-in tools and MCP tools in the system
prompt (messages field, events with the system role). The schema of a tool describes the tool, its inputs, and its
outputs. At the start, only the schemas of the built-in tools are loaded into context. Full schemas of the MCP tools
are loaded dynamically using the built-in ToolSearch tool; otherwise, they’d pollute the model context.
The behavior is configurable through ENABLE_TOOL_SEARCH and the individual alwaysLoad settings in the mcpServers
configuration of Claude Code.
Example request to load JetBrains MCP tools:
{
"type": "tool_use",
"id": "toolu_bdrk_01Sx8LNCXi9boS4p2iqMAzUk",
"name": "ToolSearch",
"input": {
"query": "select:mcp__jetbrains__get_symbol_info,mcp__jetbrains__search_symbol",
"max_results": 2
}
}
Response:
{
"type": "tool_result",
"tool_use_id": "toolu_bdrk_01Sx8LNCXi9boS4p2iqMAzUk",
"content": [
{
"type": "tool_reference",
"tool_name": "mcp__jetbrains__get_symbol_info"
},
{
"type": "tool_reference",
"tool_name": "mcp__jetbrains__search_symbol"
}
]
}
After the ToolSearch tool is invoked and returns tools, they are added to the tools field, and Claude Code can then
call them.
input.messages fieldThe messages field contains the message history of the agentic session. It’s worth highlighting that it is the
complete conversation state — it contains all messages within the session, including tool call requests and results,
and it always grows while the session is ongoing (until the next compaction). Each subsequent API call contains an
increasingly large messages array, resulting in higher token consumption.
Here’s what it looks like:
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "<session context reminder>"
},
{
"type": "text",
"text": "<runtime context>"
}
]
},
{
"role": "system",
"content": "<runtime-state>"
},
// ... actual conversation begins.
{
"role": "assistant",
"content": [
{
"type": "text",
"text": "I'll load the duckdb skill first as instructed, then search the logs."
},
{
"type": "tool_use",
"id": "toolu_bdrk_01VUdJmzUP5Z3o4Qa85jmpdM",
"name": "Skill",
"input": {
"skill": "duckdb"
}
}
]
},
// ... more messages there.
]
It contains:
CLAUDE.md files (including global and project-level files).It contains:
@ file mentions in the original prompt. For example, if you reference a directory, a listing
with file information is included; referencing an individual file includes that file’s contents.ToolSearch mechanism.output part of the Anthropic API eventThe output body is structured as a series of events:
[
{
"type": "message_start",
... # type-dependent fields here
},
{
"type": "content_block_start",
...
}
]
The event types that are useful for analytics are message_stop and message_delta.
In AWS Bedrock, message_stop contains the amazon-bedrock-invocationMetrics field with the token consumption of the
current invocation, including input, output, cache-read, and cache-write token counts.
{
"type": "message_stop",
"amazon-bedrock-invocationMetrics": {
"inputTokenCount": 6610,
"outputTokenCount": 503,
"invocationLatency": 10175,
"firstByteLatency": 5185,
"cacheReadInputTokenCount": 0,
"cacheWriteInputTokenCount": 18378
}
}
If you use Claude outside AWS, you can extract token consumption from the message_delta events.
session_id) can be a probe. For analyses such as “average first message
size” or “MCP tools available,” it’s better to filter these events out. Their messages will look like this:[
{
"role": "user",
"content": "."
}
]
Observability of Claude Code is very limited by default, but with minimal setup, you can record LLM invocations and extract many important monitoring signals from them. This rich source of data helps you:
Invocation logs are powerful precisely because they capture so much — and that is also why they must be handled with great care.
Comments
0 published comments
No published comments yet.
Leave a comment
Comments are published after review.