feat(perf): add locust perf test for credential flows#418
Conversation
Why: need perf suite for issue/verify flows What: add perf/locustfile.py and instructions Risk: script assumes backend at localhost:4000; fails if server absent Tests: locust -f perf/locustfile.py --headless -u 1 -r 1 -t 2s --host http://localhost:4000 (fails without server) Docs: perf/README.md
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| self.client.post("/graphql", json=verify_payload, name="verify") | ||
|
|
||
| present_payload = {"query": "query{credentials{id}}"} | ||
| self.client.post("/graphql", json=present_payload, name="present") |
There was a problem hiding this comment.
Stale credential used when issue step fails
The self.credential_id instance variable is only initialized once in on_start() and persists across task iterations. If an issue request fails but a previous iteration succeeded, credential_id retains the old value, causing verify and present to run with a stale credential. This masks issue failures and gives misleading performance results since the test doesn't actually test a fresh "issue → verify → present" flow per iteration. The credential_id needs to be reset to None at the start of each issue_verify_present task execution.
Additional Locations (1)
| stats = environment.stats.get("verify", "POST") | ||
| if stats and stats.get_response_time_percentile(0.95) > 250: | ||
| print("Verify p95 >250ms") | ||
| environment.process_exit_code = 1 |
There was a problem hiding this comment.
Test passes silently when no verify requests occur
The quitting event handler only sets a non-zero exit code when stats exists AND p95 exceeds 250ms. If all issue requests fail (e.g., server down, endpoint broken), credential_id is never set, no verify requests are made, stats is None, and the test exits with code 0 (success). A completely broken system would pass the performance test, masking critical failures.
| self.client.post("/graphql", json=verify_payload, name="verify") | ||
|
|
||
| present_payload = {"query": "query{credentials{id}}"} | ||
| self.client.post("/graphql", json=present_payload, name="present") |
There was a problem hiding this comment.
GraphQL errors counted as success for verify/present
The verify and present requests don't use catch_response=True and don't validate the response body. GraphQL endpoints return HTTP 200 even for errors (with errors in the response body). Any GraphQL error responses would be recorded as successful requests in Locust stats, causing misleading p95 latency measurements and potentially passing a test where verifications are actually failing.
1 participant
Summary
Testing
locust -f perf/locustfile.py --headless -u 1 -r 1 -t 2s --host http://localhost:4000(fails: connection refused)npm test(fails: npm error canceled)Codex Task
Note
Introduces a Locust-based performance suite for credential flows.
perf/locustfile.pyto load test GraphQLissue(create),verify, andpresentsteps against/graphqlverifyp95 latency exceeds250msviaevents.quittingcheckperf/README.mdWritten by Cursor Bugbot for commit ddc6bfc. This will update automatically on new commits. Configure here.