Skip to content

Conversation

@billytrend-cohere
Copy link
Collaborator

@billytrend-cohere billytrend-cohere commented Jul 9, 2024

The PR introduces changes to the Client and AsyncClient classes in src/cohere/client.py.

Summary

The changes add a _executor attribute to both classes and set its value during initialization. The ThreadPoolExecutor is now passed as an argument during initialization, allowing for more flexibility and customization.

Changes

  • The _executor attribute is added to the Client and AsyncClient classes.
  • During initialization, the thread_pool_executor argument is added to the __init__ method of both classes.
  • The value of self._executor is set to the provided thread_pool_executor argument.
  • The previous instantiation of _executor outside the methods is removed.

@fede-kamel
Copy link

Hi! I wanted to follow up on this PR as I've just submitted PR #699 which complements this implementation.

This PR addressed part of issue #534 by allowing ThreadPoolExecutor configuration at the client level, which is great. However, issue #534 also mentioned the need for configurable batch_size, and some users might need per-call flexibility rather than client-wide settings.

PR #699 adds:

Both PRs work together nicely - users can set a default ThreadPoolExecutor via your implementation and then override it for specific operations using the per-call parameters.

Just wanted to connect these implementations since they're solving related parts of the same issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants