Skip to content

johnbilt - Bulk flow segments #117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Mar 10, 2025
Merged

Conversation

johnbilt
Copy link
Contributor

Details

Ability to add flow segments in bulk via the API. This affects the existing segments end point due to semantics and existing functionality. Follows on from ADR 0024

Jira Issue (if relevant)

Jira URL: N/A

Related PRs

Related to ADR 0024, option 5.

Submitter PR Checks

(tick as appropriate)

  • PR completes task/fixes bug
  • API version has been incremented if necessary
  • ADR status has been updated, and ADR implementation has been recorded
  • Documentation updated (README, etc.)
  • PR added to Jira Issue (if relevant)
  • Follow-up stories added to Jira

Reviewer PR Checks

(tick as appropriate)

  • PR completes task/fixes bug
  • Design makes sense, and fits with our current code base
  • Code is easy to follow
  • PR size is sensible
  • Commit history is sensible and tidy

Info on PRs

The checks above are guidelines. They don't all have to be ticked, but they should all have been considered.

@johnbilt johnbilt requested a review from a team as a code owner February 14, 2025 11:28
Copy link
Contributor

@philipnbbc philipnbbc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would choose option 4 as POSTing a single segment dict rather than an array could be seen to be an API convenience.

If it is an array then the response needs to handle partial failures, issues with the array and there needs to be a description of what a TAMS should do when there are (partial) failures.

Should a server attempt to register all segments or stop at the first failure? First failure might mean a serial registration process which is not optimal. What if there are duplicates in the array? The specifications says the behaviour is undefined if a segment already exists in the store. Should that remain undefined?

The response should probably return all the segments that failed to be registered. This is probably not an API break as the current API doesn't have the segment in the response. Should it be a minimal segment with an error code and description? Should the segments that were successfully registered also be returned?


### Option 3: Rename existing segments end point and add bulk

Rename the existing `/flows/{flowId}/segments` to be singular `/flows/{flowId}/segment` and then replace the existing `/flows/{flowId}/segments` with the bulk end point
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think another downside with this option is it would potentially create an outlier endpoint without equivalent HEAD, GET, and DELETE. It's also at odds with the rest of the APIs resource endpoints which are plural. Although they are largely of the form <plaural_resource_type>/<id>.


* Good - does not break the existing API
* Good - keeps the number of end points down
* Bad - adds complexity to the API implementation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect the complexity on the spec side isn't that great. The Spec can handle validation/verification of both variants from a schema point of view (see the flow type "one of" list for the body on the flows endpoint). But @philipnbbc has some valid points on the implementation. The alternative would be something like the following pseudo-code:

if type(body) is object:
  body = [body]
for segment in body:
  validate_segment(segment)
for segment in body:
  add_segment(segment)

It's a little more complexity, but nothing too crazy.

This was referenced Feb 19, 2025
@j616
Copy link
Contributor

j616 commented Feb 19, 2025

CI bugs should be sorted in #119 . You may need to re-base for CI to pick up the changes

@johnbilt
Copy link
Contributor Author

johnbilt commented Mar 4, 2025

If it is an array then the response needs to handle partial failures, issues with the array and there needs to be a description of what a TAMS should do when there are (partial) failures.

Should a server attempt to register all segments or stop at the first failure? First failure might mean a serial registration process which is not optimal. What if there are duplicates in the array? The specifications says the behaviour is undefined if a segment already exists in the store. Should that remain undefined?

The response should probably return all the segments that failed to be registered. This is probably not an API break as the current API doesn't have the segment in the response. Should it be a minimal segment with an error code and description? Should the segments that were successfully registered also be returned?

I'm going to suggest that in the case of the failure of a single segment then the publish should continue to work through the remaining segments. Unless you're going to make TAMS roll back all the processed segments then you're going to end up with a partial item, so make it as complete as possible.

In terms of response - I think it's a 201 for successfully processing all segments (so works for one or multiple) and then for the partial success then this is more tricky. Current the best answer is probably a 200 with a list of failed segments in the body.

@philipnbbc philipnbbc self-requested a review March 5, 2025 14:28
Copy link
Contributor

@philipnbbc philipnbbc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple comments inline and then LGTM.

@philipnbbc
Copy link
Contributor

Changes look good, thanks.

@philipnbbc philipnbbc merged commit f2fe9c7 into bbc:main Mar 10, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants