BedrockRuntime / Client / count_tokens
count_tokens¶
- BedrockRuntime.Client.count_tokens(**kwargs)¶
Returns the token count for a given inference request. This operation helps you estimate token usage before sending requests to foundation models by returning the token count that would be used if the same input were sent to the model in an inference request.
Token counting is model-specific because different models use different tokenization strategies. The token count returned by this operation will match the token count that would be charged if the same input were sent to the model in an
InvokeModelorConverserequest.You can use this operation to:
Estimate costs before sending inference requests.
Optimize prompts to fit within token limits.
Plan for token usage in your applications.
This operation accepts the same input formats as
InvokeModelandConverse, allowing you to count tokens for both raw text inputs and structured conversation formats.The following operations are related to
CountTokens:InvokeModel - Sends inference requests to foundation models
Converse - Sends conversation-based inference requests to foundation models
See also: AWS API Documentation
Request Syntax
response = client.count_tokens( modelId='string', input={ 'invokeModel': { 'body': b'bytes' }, 'converse': { 'messages': [ { 'role': 'user'|'assistant', 'content': [ { 'text': 'string', 'image': { 'format': 'png'|'jpeg'|'gif'|'webp', 'source': { 'bytes': b'bytes', 's3Location': { 'uri': 'string', 'bucketOwner': 'string' } } }, 'document': { 'format': 'pdf'|'csv'|'doc'|'docx'|'xls'|'xlsx'|'html'|'txt'|'md', 'name': 'string', 'source': { 'bytes': b'bytes', 's3Location': { 'uri': 'string', 'bucketOwner': 'string' }, 'text': 'string', 'content': [ { 'text': 'string' }, ] }, 'context': 'string', 'citations': { 'enabled': True|False } }, 'video': { 'format': 'mkv'|'mov'|'mp4'|'webm'|'flv'|'mpeg'|'mpg'|'wmv'|'three_gp', 'source': { 'bytes': b'bytes', 's3Location': { 'uri': 'string', 'bucketOwner': 'string' } } }, 'toolUse': { 'toolUseId': 'string', 'name': 'string', 'input': {...}|[...]|123|123.4|'string'|True|None }, 'toolResult': { 'toolUseId': 'string', 'content': [ { 'json': {...}|[...]|123|123.4|'string'|True|None, 'text': 'string', 'image': { 'format': 'png'|'jpeg'|'gif'|'webp', 'source': { 'bytes': b'bytes', 's3Location': { 'uri': 'string', 'bucketOwner': 'string' } } }, 'document': { 'format': 'pdf'|'csv'|'doc'|'docx'|'xls'|'xlsx'|'html'|'txt'|'md', 'name': 'string', 'source': { 'bytes': b'bytes', 's3Location': { 'uri': 'string', 'bucketOwner': 'string' }, 'text': 'string', 'content': [ { 'text': 'string' }, ] }, 'context': 'string', 'citations': { 'enabled': True|False } }, 'video': { 'format': 'mkv'|'mov'|'mp4'|'webm'|'flv'|'mpeg'|'mpg'|'wmv'|'three_gp', 'source': { 'bytes': b'bytes', 's3Location': { 'uri': 'string', 'bucketOwner': 'string' } } } }, ], 'status': 'success'|'error' }, 'guardContent': { 'text': { 'text': 'string', 'qualifiers': [ 'grounding_source'|'query'|'guard_content', ] }, 'image': { 'format': 'png'|'jpeg', 'source': { 'bytes': b'bytes' } } }, 'cachePoint': { 'type': 'default' }, 'reasoningContent': { 'reasoningText': { 'text': 'string', 'signature': 'string' }, 'redactedContent': b'bytes' }, 'citationsContent': { 'content': [ { 'text': 'string' }, ], 'citations': [ { 'title': 'string', 'sourceContent': [ { 'text': 'string' }, ], 'location': { 'documentChar': { 'documentIndex': 123, 'start': 123, 'end': 123 }, 'documentPage': { 'documentIndex': 123, 'start': 123, 'end': 123 }, 'documentChunk': { 'documentIndex': 123, 'start': 123, 'end': 123 } } }, ] } }, ] }, ], 'system': [ { 'text': 'string', 'guardContent': { 'text': { 'text': 'string', 'qualifiers': [ 'grounding_source'|'query'|'guard_content', ] }, 'image': { 'format': 'png'|'jpeg', 'source': { 'bytes': b'bytes' } } }, 'cachePoint': { 'type': 'default' } }, ] } } )
- Parameters:
modelId (string) –
[REQUIRED]
The unique identifier or ARN of the foundation model to use for token counting. Each model processes tokens differently, so the token count is specific to the model you specify.
input (dict) –
[REQUIRED]
The input for which to count tokens. The structure of this parameter depends on whether you’re counting tokens for an
InvokeModelorConverserequest:For
InvokeModelrequests, provide the request body in theinvokeModelfieldFor
Converserequests, provide the messages and system content in theconversefield
The input format must be compatible with the model specified in the
modelIdparameter.Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
invokeModel,converse.invokeModel (dict) –
An
InvokeModelrequest for which to count tokens. Use this field when you want to count tokens for a raw text input that would be sent to theInvokeModeloperation.body (bytes) – [REQUIRED]
The request body to count tokens for, formatted according to the model’s expected input format. To learn about the input format for different models, see Model inference parameters and responses.
converse (dict) –
A
Converserequest for which to count tokens. Use this field when you want to count tokens for a conversation-based input that would be sent to theConverseoperation.messages (list) –
An array of messages to count tokens for.
(dict) –
A message input, or returned from, a call to Converse or ConverseStream.
role (string) – [REQUIRED]
The role that the message plays in the message.
content (list) – [REQUIRED]
The message content. Note the following restrictions:
You can include up to 20 images. Each image’s size, height, and width must be no more than 3.75 MB, 8000 px, and 8000 px, respectively.
You can include up to five documents. Each document’s size must be no more than 4.5 MB.
If you include a
ContentBlockwith adocumentfield in the array, you must also include aContentBlockwith atextfield.You can only include images and documents if the
roleisuser.
(dict) –
A block of content for a message that you pass to, or receive from, a model with the Converse or ConverseStream API operations.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
text,image,document,video,toolUse,toolResult,guardContent,cachePoint,reasoningContent,citationsContent.text (string) –
Text to include in the message.
image (dict) –
Image to include in the message.
Note
This field is only supported by Anthropic Claude 3 models.
format (string) – [REQUIRED]
The format of the image.
source (dict) – [REQUIRED]
The source for the image.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
bytes,s3Location.bytes (bytes) –
The raw image bytes for the image. If you use an AWS SDK, you don’t need to encode the image bytes in base64.
s3Location (dict) –
The location of an image object in an Amazon S3 bucket. To see which models support S3 uploads, see Supported models and features for Converse.
uri (string) – [REQUIRED]
An object URI starting with
s3://.bucketOwner (string) –
If the bucket belongs to another AWS account, specify that account’s ID.
document (dict) –
A document to include in the message.
format (string) –
The format of a document, or its extension.
name (string) – [REQUIRED]
A name for the document. The name can only contain the following characters:
Alphanumeric characters
Whitespace characters (no more than one in a row)
Hyphens
Parentheses
Square brackets
Note
This field is vulnerable to prompt injections, because the model might inadvertently interpret it as instructions. Therefore, we recommend that you specify a neutral name.
source (dict) – [REQUIRED]
Contains the content of the document.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
bytes,s3Location,text,content.bytes (bytes) –
The raw bytes for the document. If you use an Amazon Web Services SDK, you don’t need to encode the bytes in base64.
s3Location (dict) –
The location of a document object in an Amazon S3 bucket. To see which models support S3 uploads, see Supported models and features for Converse.
uri (string) – [REQUIRED]
An object URI starting with
s3://.bucketOwner (string) –
If the bucket belongs to another AWS account, specify that account’s ID.
text (string) –
The text content of the document source.
content (list) –
The structured content of the document source, which may include various content blocks such as text, images, or other document elements.
(dict) –
Contains the actual content of a document that can be processed by the model and potentially cited in the response.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
text.text (string) –
The text content of the document.
context (string) –
Contextual information about how the document should be processed or interpreted by the model when generating citations.
citations (dict) –
Configuration settings that control how citations should be generated for this specific document.
enabled (boolean) – [REQUIRED]
Specifies whether document citations should be included in the model’s response. When set to true, the model can generate citations that reference the source documents used to inform the response.
video (dict) –
Video to include in the message.
format (string) – [REQUIRED]
The block’s format.
source (dict) – [REQUIRED]
The block’s source.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
bytes,s3Location.bytes (bytes) –
Video content encoded in base64.
s3Location (dict) –
The location of a video object in an Amazon S3 bucket. To see which models support S3 uploads, see Supported models and features for Converse.
uri (string) – [REQUIRED]
An object URI starting with
s3://.bucketOwner (string) –
If the bucket belongs to another AWS account, specify that account’s ID.
toolUse (dict) –
Information about a tool use request from a model.
toolUseId (string) – [REQUIRED]
The ID for the tool request.
name (string) – [REQUIRED]
The name of the tool that the model wants to use.
input (document) – [REQUIRED]
The input to pass to the tool.
toolResult (dict) –
The result for a tool request that a model makes.
toolUseId (string) – [REQUIRED]
The ID of the tool request that this is the result for.
content (list) – [REQUIRED]
The content for tool result content block.
(dict) –
The tool result content block.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
json,text,image,document,video.json (document) –
A tool result that is JSON format data.
text (string) –
A tool result that is text.
image (dict) –
A tool result that is an image.
Note
This field is only supported by Anthropic Claude 3 models.
format (string) – [REQUIRED]
The format of the image.
source (dict) – [REQUIRED]
The source for the image.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
bytes,s3Location.bytes (bytes) –
The raw image bytes for the image. If you use an AWS SDK, you don’t need to encode the image bytes in base64.
s3Location (dict) –
The location of an image object in an Amazon S3 bucket. To see which models support S3 uploads, see Supported models and features for Converse.
uri (string) – [REQUIRED]
An object URI starting with
s3://.bucketOwner (string) –
If the bucket belongs to another AWS account, specify that account’s ID.
document (dict) –
A tool result that is a document.
format (string) –
The format of a document, or its extension.
name (string) – [REQUIRED]
A name for the document. The name can only contain the following characters:
Alphanumeric characters
Whitespace characters (no more than one in a row)
Hyphens
Parentheses
Square brackets
Note
This field is vulnerable to prompt injections, because the model might inadvertently interpret it as instructions. Therefore, we recommend that you specify a neutral name.
source (dict) – [REQUIRED]
Contains the content of the document.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
bytes,s3Location,text,content.bytes (bytes) –
The raw bytes for the document. If you use an Amazon Web Services SDK, you don’t need to encode the bytes in base64.
s3Location (dict) –
The location of a document object in an Amazon S3 bucket. To see which models support S3 uploads, see Supported models and features for Converse.
uri (string) – [REQUIRED]
An object URI starting with
s3://.bucketOwner (string) –
If the bucket belongs to another AWS account, specify that account’s ID.
text (string) –
The text content of the document source.
content (list) –
The structured content of the document source, which may include various content blocks such as text, images, or other document elements.
(dict) –
Contains the actual content of a document that can be processed by the model and potentially cited in the response.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
text.text (string) –
The text content of the document.
context (string) –
Contextual information about how the document should be processed or interpreted by the model when generating citations.
citations (dict) –
Configuration settings that control how citations should be generated for this specific document.
enabled (boolean) – [REQUIRED]
Specifies whether document citations should be included in the model’s response. When set to true, the model can generate citations that reference the source documents used to inform the response.
video (dict) –
A tool result that is video.
format (string) – [REQUIRED]
The block’s format.
source (dict) – [REQUIRED]
The block’s source.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
bytes,s3Location.bytes (bytes) –
Video content encoded in base64.
s3Location (dict) –
The location of a video object in an Amazon S3 bucket. To see which models support S3 uploads, see Supported models and features for Converse.
uri (string) – [REQUIRED]
An object URI starting with
s3://.bucketOwner (string) –
If the bucket belongs to another AWS account, specify that account’s ID.
status (string) –
The status for the tool result content block.
Note
This field is only supported Anthropic Claude 3 models.
guardContent (dict) –
Contains the content to assess with the guardrail. If you don’t specify
guardContentin a call to the Converse API, the guardrail (if passed in the Converse API) assesses the entire message.For more information, see Use a guardrail with the Converse API in the Amazon Bedrock User Guide.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
text,image.text (dict) –
The text to guard.
text (string) – [REQUIRED]
The text that you want to guard.
qualifiers (list) –
The qualifier details for the guardrails contextual grounding filter.
(string) –
image (dict) –
Image within converse content block to be evaluated by the guardrail.
format (string) – [REQUIRED]
The format details for the image type of the guardrail converse image block.
source (dict) – [REQUIRED]
The image source (image bytes) of the guardrail converse image block.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
bytes.bytes (bytes) –
The raw image bytes for the image.
cachePoint (dict) –
CachePoint to include in the message.
type (string) – [REQUIRED]
Specifies the type of cache point within the CachePointBlock.
reasoningContent (dict) –
Contains content regarding the reasoning that is carried out by the model. Reasoning refers to a Chain of Thought (CoT) that the model generates to enhance the accuracy of its final response.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
reasoningText,redactedContent.reasoningText (dict) –
The reasoning that the model used to return the output.
text (string) – [REQUIRED]
The reasoning that the model used to return the output.
signature (string) –
A token that verifies that the reasoning text was generated by the model. If you pass a reasoning block back to the API in a multi-turn conversation, include the text and its signature unmodified.
redactedContent (bytes) –
The content in the reasoning that was encrypted by the model provider for safety reasons. The encryption doesn’t affect the quality of responses.
citationsContent (dict) –
A content block that contains both generated text and associated citation information, providing traceability between the response and source documents.
content (list) –
The generated content that is supported by the associated citations.
(dict) –
Contains the generated text content that corresponds to or is supported by a citation from a source document.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
text.text (string) –
The text content that was generated by the model and is supported by the associated citation.
citations (list) –
An array of citations that reference the source documents used to generate the associated content.
(dict) –
Contains information about a citation that references a specific source document. Citations provide traceability between the model’s generated response and the source documents that informed that response.
title (string) –
The title or identifier of the source document being cited.
sourceContent (list) –
The specific content from the source document that was referenced or cited in the generated response.
(dict) –
Contains the actual text content from a source document that is being cited or referenced in the model’s response.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
text.text (string) –
The text content from the source document that is being cited.
location (dict) –
The precise location within the source document where the cited content can be found, including character positions, page numbers, or chunk identifiers.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
documentChar,documentPage,documentChunk.documentChar (dict) –
The character-level location within the document where the cited content is found.
documentIndex (integer) –
The index of the document within the array of documents provided in the request.
start (integer) –
The starting character position of the cited content within the document.
end (integer) –
The ending character position of the cited content within the document.
documentPage (dict) –
The page-level location within the document where the cited content is found.
documentIndex (integer) –
The index of the document within the array of documents provided in the request.
start (integer) –
The starting page number of the cited content within the document.
end (integer) –
The ending page number of the cited content within the document.
documentChunk (dict) –
The chunk-level location within the document where the cited content is found, typically used for documents that have been segmented into logical chunks.
documentIndex (integer) –
The index of the document within the array of documents provided in the request.
start (integer) –
The starting chunk identifier or index of the cited content within the document.
end (integer) –
The ending chunk identifier or index of the cited content within the document.
system (list) –
The system content blocks to count tokens for. System content provides instructions or context to the model about how it should behave or respond. The token count will include any system content provided.
(dict) –
A system content block.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
text,guardContent,cachePoint.text (string) –
A system prompt for the model.
guardContent (dict) –
A content block to assess with the guardrail. Use with the Converse or ConverseStream API operations.
For more information, see Use a guardrail with the Converse API in the Amazon Bedrock User Guide.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
text,image.text (dict) –
The text to guard.
text (string) – [REQUIRED]
The text that you want to guard.
qualifiers (list) –
The qualifier details for the guardrails contextual grounding filter.
(string) –
image (dict) –
Image within converse content block to be evaluated by the guardrail.
format (string) – [REQUIRED]
The format details for the image type of the guardrail converse image block.
source (dict) – [REQUIRED]
The image source (image bytes) of the guardrail converse image block.
Note
This is a Tagged Union structure. Only one of the following top level keys can be set:
bytes.bytes (bytes) –
The raw image bytes for the image.
cachePoint (dict) –
CachePoint to include in the system prompt.
type (string) – [REQUIRED]
Specifies the type of cache point within the CachePointBlock.
- Return type:
dict
- Returns:
Response Syntax
{ 'inputTokens': 123 }
Response Structure
(dict) –
inputTokens (integer) –
The number of tokens in the provided input according to the specified model’s tokenization rules. This count represents the number of input tokens that would be processed if the same input were sent to the model in an inference request. Use this value to estimate costs and ensure your inputs stay within model token limits.
Exceptions