Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: implement screenshot caching via Responses API reference IDs #90

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tongshu1943
Copy link

Design Approach

This implementation leverages the OpenAI Responses API reference ID mechanism to optimize screenshot handling in the ComputerTool. The system uses reference IDs to point to previously sent images, significantly reducing bandwidth usage and memory footprint.

Implementation Details

  • Modified Converter._convert_tool to set the include parameter "computer_call_output.output.image_url" for ComputerTool instances
  • Added test coverage to verify the reference ID functionality works correctly
  • Updated existing tests to maintain compatibility with the new include parameter while preserving original design intent

Technical Considerations

  • The OpenAI Responses API automatically handles reference management on the server side
  • This approach shifts the storage burden from client memory to the API service
  • The implementation maintains backward compatibility with existing code
  • Clean code principles were followed with minimal changes to the codebase

@tongshu1943
Copy link
Author

@rm-openai Thank you for your valuable suggestions! 🙏 I have updated the implementation to use the Responses API for caching, as per your recommendation. This approach simplifies the logic and improves efficiency.

Please take a moment to review the changes and let me know if there are any further adjustments or improvements needed.

Looking forward to your feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant