Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testing harness #79

Open
8 of 15 tasks
ryderwishart opened this issue Sep 11, 2023 · 0 comments
Open
8 of 15 tasks

Add testing harness #79

ryderwishart opened this issue Sep 11, 2023 · 0 comments
Assignees

Comments

@ryderwishart
Copy link
Contributor

ryderwishart commented Sep 11, 2023

We have some tests already written for testing 1. Loading UIs 2. Three different dataupload APIs 3. Chat endpoint

Enhancements Needed

  1. Make use of mock feature to avoid calling 3rd party APIs like OpenAI and Supabase repeatedly
  2. Use parameterized tests to easily change and test different tech stacks we have.
  3. Use fixtures to cleanup test data upon teardown enabling repeated runs of same tests with same data again

Tasks Break Down

  • Mock Supabase access to check token and role allowing dummy users to run tests
  • Fixture to clean up postgres test DB after each test (Chroma already done)
  • Dataupload tests
    • Run the 3 dataupload API tests on chroma and postgres alike
    • Run text data upload with different fileprocessors, Langchain and vanilla
    • Run dataupload with different embeddings(Mock the ones that need paid services)
  • Get labels API
    • Test get labels works with changing data and different databases
  • Chat endpoint tests
    • Mock LLM operations like generate, summarization
    • Run tests with different tech stacks
    • Test document filtering as per API input and user role
  • Test Auth APIs
    • Error handling in different cases, un-successful registartion, incorrect login creds, session out etc
  • Run tests automatically on PRs and pushes on github actions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants