dhis2: Implement `http` namespace, a generic request function, and remove all callbacks #1046

hunterachieng · 2025-03-10T15:21:41Z

Summary

Move all the generic functions into a http namespace, remove callbacks and add a generic request function.

Fixes #978

Details

All the functions such as post, get, and patch are moved to their own http name-spaced file to separate them from the other specific functions, and a generic request function was added as well.

All callbacks are being removed since the same behaviour can be achieved with promise chaining

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to
know!):

You can read more details in our
Responsible AI Policy

Review Checklist

Before merging, the reviewer should check the following items:

Does the PR do what it claims to do?
If this is a new adaptor, added the adaptor on marketing website ?
If this PR includes breaking changes, do we need to update any jobs in
production? Is it safe to release?
Are there any unit tests?
Is there a changeset associated with this PR? Should there be? Note that
dev only changes don't need a changeset.
Have you ticked a box under AI Usage?

Signed-off-by: Hunter Achieng <[email protected]>

josephjclark

I am having a hard time seperating "move http methods out into a clean namespace" from "fix all the problems with DHIS2 http APIs"

Docs are worrying me, base64 is worrying me, and I wonder if handleResponse needs to move into here.

I need to break here today but I'll think about this stuff more tomorrow.

It doesn't technically matter which PR fixes these problems - but it does matter what's in the next release. Dhis2 7 really needs to have a clean, well documented API. And do get to there we'll need more work.

I'll come back with a fresh brain.

.changeset/silly-bulldogs-complain.md

josephjclark · 2025-03-10T17:38:00Z

packages/dhis2/src/Adaptor.js

-          return update(resourceType, path, data, options)(state);
-        }
-      });
+      promise = http


ooh this is really annoying

Because we're using the http.get() operation directly, we have to feed state back into the call here.

it's an antipattern really. We certainly don't want to encourage it, and this is a big, brand-defining adaptor.

We don't have to do this in this PR, but what's the cost in using like util.request instead of http.request? Of using a function rather than an operation?

Also: if asbase64 is true, I think this stuff might be broken? Because response.data is a string and all these tests will fail 😬

josephjclark · 2025-03-10T17:45:36Z

packages/dhis2/src/http.js

+
+/**
+ * Options object
+ * @typedef {Object} RequestOptions


Worried about these docs but they're not technically part of this PR. Let me think about it. Referenced here: #975 (comment)

josephjclark · 2025-03-10T17:54:15Z

packages/dhis2/src/http.js

+ * @example <caption>a dataElement</caption>
+ * http.patch('dataElements', 'FTRrcoaog83', { name: 'New Name' });
+ */
+// TODO: @Elias, can this be deleted in favor of update? How does DHIS2 handle PATCH vs PUT?


@mtuchi can you help resolve this? Should we just remove patch?

Hiya @josephjclark this is what i was able to gather regarding PATCH method in DHIS2

The DHIS2 API supports the PATCH method for various metadata endpoints.

For example, you can use PATCH to:

Update properties of metadata objects

Add or remove items from collections

Change domain types and value types

Remove specific organization units from groups

When using the PATCH method with any of these endpoints, you must use the content type application/json-patch+json in your request headers as per the JSON patch specification. See DHIS2 Documentation - Partial Updates DHIS2 Documentation - Bulk Sharing

Meanwhile, you can use PUT on:

Metadata endpoints - For updating various metadata objects like data elements, organization units, etc. DHIS2 Documentation - Metadata CRUD

Data Store endpoints - For updating key-value pairs DHIS2 Documentation - Data Store

Maintenance endpoints - For performing system maintenance operations DHIS2 Documentation - Maintenance

Visualization endpoints - For updating visualizations DHIS2 Documentation - Visualizations

That being said, I have never used the API endpoints in this discovery but i guess it doesn't hurt to maintain both patch and put or if we decide to remove them we should have a http.request function which will still allow me to specify patch or put when there is a need for that

Thank you for the thorough report @mtuchi!

So this must be a JSON patch. That's interesting.

If we wanted, we could sit down a workout a nice easy JSON patch API. But we should not do that for a function that no-one is using.

How about:

We keep the patch function

We default the content type to application/json-patch+json

We document that data must be in JSON Patch format and link to DHIS2 docs

That's a reasonable position to take right now. Maybe later we can make this a bit slicker.

I am happy with that, and since the link to DHIS2 docs is subjected to either metadata apis or sharing apis maybe we add an example with JSON Patch format. For example we should convert this example 👇🏽 to openfn job code

PATCH /api/dataElements/{id} [ {"op": "add", "path": "/name", "value": "New Name"}, {"op": "add", "path": "/valueType", "value": "INTEGER"} ]

Signed-off-by: Hunter Achieng <[email protected]>

josephjclark · 2025-03-17T13:45:59Z

@hunterachieng after you've addressed my comments, you'll want to rebase this on top of #1059 (or maybe merge is easier). It might be a hard rebase so it might be easier to start again? You'll need to take a look and decide.

…feature/978-dhis2-namespace Signed-off-by: Hunter Achieng <[email protected]>

Signed-off-by: Hunter Achieng <[email protected]>

hunterachieng · 2025-03-31T10:39:35Z

@josephjclark I have re-worked this. Do you mind taking a look?

…feature/978-dhis2-namespace Signed-off-by: Hunter Achieng <[email protected]>

josephjclark

Ah, now the problem with this is is that there's no basic get() function to retrieve resources. This is partly my fault for not thinking this through.

What I'm looking for is something like OpenMRS, where we have a basic HTTP namespace and a fancier main rest namespace which doesn't use HTTP semantics. This is actually the focus of OpenMRS 5 (which isn't ready for release yet but see the PR)

The HTTP namespace is pretty good. I like that it's lower level - no clever tracker stuff. That's great. I've left a few comments but it's broadly right.

Let's finish the http namespace, but then we need to think a bit more about the main namespace. What should get() look like? Maybe we should just restore the old get() for now?

EDIT: Yes, looking back, I'm fairly sure that we should just restore original get() and have a more lightweight one in http (different docs, fewer examples). Maybe we'll tweak it before release, but not in this PR

packages/dhis2/src/http.js

josephjclark · 2025-04-01T17:58:33Z

Sorry @hunterachieng I posted this review a bit early - I'll update it in the morning :)

packages/dhis2/src/http.js

josephjclark · 2025-04-01T18:34:21Z

packages/dhis2/src/Adaptor.js

@@ -253,7 +251,7 @@ export function create(resourceType, data, options = {}, callback = s => s) {
    const { location } = response.headers;
    if (location) console.log(`Record available @ ${location}`);

-    return handleResponse(response, state, callback);
+    return handleResponse(response, state);


I think - and I'll come back to you on this - that in the main namespace here, we don't include response in the result. We just return to state.data.

The http methods should surface the response, but the resource helpers should not

@josephjclark What should we do here? I have fixed the other comments

Signed-off-by: Hunter Achieng <[email protected]>

feat: implement http namespace

7fe8f07

Signed-off-by: Hunter Achieng <[email protected]>

hunterachieng requested a review from josephjclark March 10, 2025 15:21

hunterachieng changed the base branch from main to epic/release-dhis2 March 10, 2025 15:23

feat: add changeset

1a422c0

Signed-off-by: Hunter Achieng <[email protected]>

josephjclark reviewed Mar 10, 2025

View reviewed changes

hunterachieng changed the title ~~dhis2: Implement http namespace~~ dhis2: Implement http namespace, a generic request function, and remove all callbacks Mar 11, 2025

fix: change changeset from minor to major

be489ab

Signed-off-by: Hunter Achieng <[email protected]>

hunterachieng mentioned this pull request Mar 11, 2025

dhis2: fix upsert anti pattern behavior #1049

Closed

hunterachieng requested a review from josephjclark March 11, 2025 12:46

hunterachieng added 2 commits March 31, 2025 10:47

Merge branch 'epic/release-dhis2' of github.com:OpenFn/adaptors into …

29a1c83

…feature/978-dhis2-namespace Signed-off-by: Hunter Achieng <[email protected]>

feat: re-work http namespace file functions

7cd1d3e

Signed-off-by: Hunter Achieng <[email protected]>

Merge branch 'epic/release-dhis2' of github.com:OpenFn/adaptors into …

1a24fb5

…feature/978-dhis2-namespace Signed-off-by: Hunter Achieng <[email protected]>

josephjclark reviewed Apr 1, 2025

View reviewed changes

packages/dhis2/src/http.js Show resolved Hide resolved

packages/dhis2/src/http.js Outdated Show resolved Hide resolved

packages/dhis2/src/http.js Outdated Show resolved Hide resolved

josephjclark reviewed Apr 1, 2025

View reviewed changes

hunterachieng added 4 commits April 4, 2025 12:35

fix: make state docs private

f9b44d8

Signed-off-by: Hunter Achieng <[email protected]>

fix: switch from generic helper to http helper

f8dcae4

Signed-off-by: Hunter Achieng <[email protected]>

fix: switch from resourceType to path

6b79517

Signed-off-by: Hunter Achieng <[email protected]>

fix: reduce get examples

ef16a4d

Signed-off-by: Hunter Achieng <[email protected]>

hunterachieng requested a review from josephjclark April 4, 2025 10:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dhis2: Implement `http` namespace, a generic request function, and remove all callbacks #1046

dhis2: Implement `http` namespace, a generic request function, and remove all callbacks #1046

hunterachieng commented Mar 10, 2025 •

edited

Loading

josephjclark left a comment

josephjclark Mar 10, 2025

josephjclark Mar 10, 2025

josephjclark Mar 10, 2025

mtuchi Mar 12, 2025

josephjclark Mar 12, 2025

mtuchi Mar 12, 2025

josephjclark commented Mar 17, 2025

hunterachieng commented Mar 31, 2025

josephjclark left a comment •

edited

Loading

josephjclark commented Apr 1, 2025

josephjclark Apr 1, 2025

hunterachieng Apr 4, 2025

dhis2: Implement http namespace, a generic request function, and remove all callbacks #1046

Are you sure you want to change the base?

dhis2: Implement http namespace, a generic request function, and remove all callbacks #1046

Conversation

hunterachieng commented Mar 10, 2025 • edited Loading

Summary

Details

AI Usage

Review Checklist

josephjclark left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josephjclark commented Mar 17, 2025

hunterachieng commented Mar 31, 2025

josephjclark left a comment • edited Loading

Choose a reason for hiding this comment

josephjclark commented Apr 1, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhis2: Implement `http` namespace, a generic request function, and remove all callbacks #1046

dhis2: Implement `http` namespace, a generic request function, and remove all callbacks #1046

hunterachieng commented Mar 10, 2025 •

edited

Loading

josephjclark left a comment •

edited

Loading