Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/tm gitlab pagination #232

Merged
merged 12 commits into from
Dec 16, 2024
Merged

Feat/tm gitlab pagination #232

merged 12 commits into from
Dec 16, 2024

Conversation

y-eight
Copy link
Member

@y-eight y-eight commented Dec 11, 2024

Motivation

Referes to #230

This PR adds the ability to the GitLab targetManager to fetch files recursively in case the GitLab API provides a paginated response.

Changes

  • requests for result pages are made recursively

Additional changes:

  • Add helper function to get the next page link of pagination
  • Increased the result per page to 30 items (default by GitLab API is 20)
  • The ref is now added by the query.Set() instead of query.Add() -> result: query parameter is switched and not appended

For additional information look at the commits.

Tests done

  • Unit tests succeeded
  • E2E tests succeeded

TODO

  • adding test case to test recursive fetchNextFileList func

@y-eight y-eight self-assigned this Dec 11, 2024
@y-eight y-eight added feature Introduces a new feature area/target-manager Issues/PRs related to the TargetManager labels Dec 11, 2024
@y-eight
Copy link
Member Author

y-eight commented Dec 12, 2024

Test for paginated requests added.

PR is complete from my side.

@y-eight y-eight requested a review from lvlcn-t December 12, 2024 11:20
Copy link
Collaborator

@puffitos puffitos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I ran the sparrow with paginationPerPage set to 20 and still got 30 targets.

@y-eight y-eight merged commit ef1a86f into main Dec 16, 2024
8 checks passed
@y-eight y-eight deleted the feat/tm-gitlab--pagination branch December 16, 2024 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/target-manager Issues/PRs related to the TargetManager feature Introduces a new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants