Skip to content

[DOCS] document how to obtain cache stats when caching system prompt #118

Open
@panditkbgh

Description

@panditkbgh

Documentation Section

https://strandsagents.com/latest/user-guide/concepts/model-providers/amazon-bedrock/#system-prompt-caching

Current State

We should document on https://strandsagents.com/latest/user-guide/concepts/model-providers/amazon-bedrock/#system-prompt-caching how to get the cache stats. Currently the page does not have those details.

Suggested Improvement

CacheReadInputTokens and CacheWriteInputTokens in https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html have this info. It is available in the event event type in Strands async iterators and callback handlers, for example: https://github.com/strands-agents/sdk-python/blob/1421aadd73c7a497503e43864649c9016dadea69/tests-integ/test_bedrock_cache_point.py#L21-L27

Documentations improvements: getting info on them should be documented in https://strandsagents.com/latest/user-guide/concepts/model-providers/amazon-bedrock/#system-prompt-caching

Another possible improvement is to return this info as a part of AgentResult.metrics and OpenTelemetry traces. That way, one does not need to write async handlers to get at this info.

Rationale

better documentation and ease of use

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions