Skip to content

feat: add batch search #130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
micpst opened this issue Oct 22, 2024 · 1 comment
Open

feat: add batch search #130

micpst opened this issue Oct 22, 2024 · 1 comment
Labels
feature New feature or request
Milestone

Comments

@micpst
Copy link
Collaborator

micpst commented Oct 22, 2024

Feature description

The idea is to allow the user to submit multiple questions in a single document search run. We should leverage batch requests to LLMs to mitigate timeouts/rate limits and ensure proper error handling.

Motivation

This feature is especially useful for evaluation when we process thousands of questions in different configurations, but also if we think about production workflows, batch processing is a must.

Additional context

No response

@micpst micpst added feature New feature or request document search labels Oct 22, 2024
@micpst micpst added this to ragbits Oct 22, 2024
@micpst micpst moved this to Backlog in ragbits Oct 22, 2024
@micpst micpst self-assigned this Oct 22, 2024
@micpst micpst moved this from Backlog to Ready in ragbits Oct 22, 2024
@mhordynski mhordynski added this to the Ragbits 0.4 milestone Oct 23, 2024
@mhordynski mhordynski removed this from the Ragbits 0.6 milestone Dec 2, 2024
@mhordynski mhordynski added this to the Long-term milestone Jan 20, 2025
@micpst micpst changed the title feat(document-search): batch search feat: add batch search May 5, 2025
@mhordynski
Copy link
Member

Connected with #346

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
Status: Ready
Development

No branches or pull requests

2 participants