Add Token Count API Support

[cddlee]

Summary

Implements support for the Anthropic Token Count API (POST /v1/messages/count_tokens), allowing users to estimate token usage before sending messages. This helps with cost estimation and managing rate limits.

Changes

New Classes

MessageCountTokensRequest (src/commonMain/kotlin/message/Messages.kt)

• Request class for the Token Count API
• Includes only fields accepted by the API: model, messages, system, tools, tool_choice
• Excludes fields not accepted: max_tokens, metadata, stop_sequences, stream, temperature, top_k, top_p

MessageTokensCount (src/commonMain/kotlin/message/Messages.kt)

• Simple response class containing inputTokens: Int
• Does not extend Response since the API returns a plain JSON object without a type field

New API Method

Anthropic.Messages.countTokens() (src/commonMain/kotlin/Anthropic.kt)

suspend fun countTokens(
    block: MessageRequest.Builder.() -> Unit
): MessageTokensCount

• Uses familiar MessageRequest.Builder DSL for consistency with messages.create()
• Applies default model and tools from client configuration
• Converts to MessageCountTokensRequest before API call
• Posts to /v1/messages/count_tokens endpoint

Tests

Added comprehensive test coverage in MessageCountTokensTest.kt:

• ✅ Basic message token counting
• ✅ Token counting with system prompts
• ✅ Token counting with tools
• ✅ Multi-turn conversation token counting
• ✅ Consistency verification (same input → same count)
• ✅ Variable length message comparison

Usage Example

// Count tokens for a simple message
val count = anthropic.messages.countTokens {
    +"Hello, Claude!"
}
println("Input tokens: ${count.inputTokens}")

// Count tokens with system prompt and tools
val countWithTools = anthropic.messages.countTokens {
    system("You are a helpful assistant")
    tools = listOf(Tool<GetWeather>("get_weather"))
    +"What's the weather in San Francisco?"
}
println("Input tokens: ${countWithTools.inputTokens}")

Implementation Notes

1. Separate Request Class: Created MessageCountTokensRequest instead of reusing MessageRequest because the Token Count API explicitly rejects extra parameters (returns "Extra inputs are not permitted" error for fields like max_tokens)
2. No Cost Tracking: Unlike messages.create(), the countTokens() method does not update the CostCollector since the Token Count API is free to use
3. API Limitations:

• Token counts are estimates and may include system-added tokens
• Does NOT use prompt caching (even if cache_control blocks are provided)
• Subject to rate limits despite being free

4. Future Enhancement: The thinking parameter for extended thinking is not yet supported, as it requires adding support to MessageRequest first

Breaking Changes

None - this is a purely additive change.

References

• https://platform.claude.com/docs/en/build-with-claude/token-counting
• https://platform.claude.com/docs/en/api/messages/count_tokens