Skip to content

perf: implement screenshot caching via Responses API reference IDs #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

tongshu1943
Copy link

Design Approach

This implementation leverages the OpenAI Responses API reference ID mechanism to optimize screenshot handling in the ComputerTool. The system uses reference IDs to point to previously sent images, significantly reducing bandwidth usage and memory footprint.

Implementation Details

  • Modified Converter._convert_tool to set the include parameter "computer_call_output.output.image_url" for ComputerTool instances
  • Added test coverage to verify the reference ID functionality works correctly
  • Updated existing tests to maintain compatibility with the new include parameter while preserving original design intent

Technical Considerations

  • The OpenAI Responses API automatically handles reference management on the server side
  • This approach shifts the storage burden from client memory to the API service
  • The implementation maintains backward compatibility with existing code
  • Clean code principles were followed with minimal changes to the codebase

@tongshu1943
Copy link
Author

@rm-openai Thank you for your valuable suggestions! 🙏 I have updated the implementation to use the Responses API for caching, as per your recommendation. This approach simplifies the logic and improves efficiency.

Please take a moment to review the changes and let me know if there are any further adjustments or improvements needed.

Looking forward to your feedback!

Copy link

This PR is stale because it has been open for 10 days with no activity.

@github-actions github-actions bot added the stale label Mar 26, 2025
Copy link

github-actions bot commented Apr 2, 2025

This PR was closed because it has been inactive for 7 days since being marked as stale.

@github-actions github-actions bot closed this Apr 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant