NIFI-14968 - Add support for Bitbucket Data Center #10322

pvillard31 · 2025-09-18T18:11:53Z

Summary

NIFI-14968 - Add support for Bitbucket Data Center

Breaking Change - the API version property is removed and Registry Clients do not support the migrateProperties feature so there is no nice way to remove it. However I think this is OK and can be clearly identified in the release notes / migration notes. Happy to re-add the property if needed and just ignore it.

Important - for Data Center, when committing a file using multipart, the existing code was not working also sending data using curl was working as expected. It turned out that the multipart was not formatted properly and it was expecting the filename as part of the content disposition.

Also... as of September 9th, Bitbucket does not allow users to create App Passwords anymore and API Tokens are the long term replacement. I have made the changes to reflect this.

Most of the work is to decouple the Data Center 1.0 APIs and the Cloud 2.0 APIs. I also fixed an issue that existed in case the registry client is used against a repo that has just been created with no commit at all (quite rare but this is now supported).

The change has been tested against both a Cloud and a Data Center instance. For Data Center, I tested against the latest version Data Center 10.0.

All the usual scenarios have been tested: branching, change version, commit version, start version control, as well as some of the usual edge cases.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Apache NiFi Jira issue created

Pull Request Tracking

Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

Pull Request based on current revision of the main branch
Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

Build completed using ./mvnw clean install -P contrib-check
- JDK 21
- JDK 25

Licensing

New dependencies are compatible with the Apache License 2.0 according to the License Policy
New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

Documentation formatting appears as expected in rendered files

exceptionfactory

Thanks for the substantial work on implementing support for Bitbucket Data Center @pvillard31. Most of the changes look clear enough, I noted a few recommendations.

...-client-api/src/main/java/org/apache/nifi/web/client/api/MultipartFormDataStreamBuilder.java

exceptionfactory · 2025-10-29T18:55:06Z

...api/src/main/java/org/apache/nifi/web/client/api/StandardMultipartFormDataStreamBuilder.java

+            final String boundaryPrefix = (index == 0 ? BOUNDARY_SEPARATOR + boundary + CARRIAGE_RETURN_LINE_FEED
+                    : CARRIAGE_RETURN_LINE_FEED + BOUNDARY_SEPARATOR + boundary + CARRIAGE_RETURN_LINE_FEED);


This ternary and string concatenation is rather difficult to read. Having something like the getFooter method would be helpful.

exceptionfactory · 2025-10-29T18:59:03Z

...xtensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketAuthenticationType.java

 public enum BitbucketAuthenticationType implements DescribedValue {

-    BASIC_AUTH("Basic Auth", """
+    BASIC_AUTH("Basic Auth & API Token", """


Recommend avoiding the use of the ampersand character in labels. On the other hand, is it necessary to mention API Token in the name when it is in the description?

Suggested change

BASIC_AUTH("Basic Auth & API Token", """

BASIC_AUTH("Basic Auth and API Token", """

exceptionfactory · 2025-10-29T19:00:06Z

...xtensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketFlowRegistryClient.java

    static final PropertyDescriptor APP_PASSWORD = new PropertyDescriptor.Builder()
            .name("App Password")
-            .description("The App Password to use for authentication")
+            .displayName("App Password or API Token")


Instead of adding the displayName, it would be better to migrate the property name. Alternatively, for the sake of clarity, it might be better to have a separate property.

migrateProperties is not available for registry clients, looking into having a separate property

exceptionfactory · 2025-10-29T19:01:02Z

...xtensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketFlowRegistryClient.java

+            final String apiHost = bitbucketRepositoryClient.getApiHost();
+            final String projectKey = bitbucketRepositoryClient.getProjectKey();
+            if (apiHost != null && projectKey != null) {
+                return "git@" + apiHost + ":" + projectKey + "/" + bitbucketRepositoryClient.getRepoName() + ".git";


Recommend using string formatting instead of concatenation to make this structure easier to follow.

exceptionfactory · 2025-10-29T19:02:25Z

...-extensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketRepositoryClient.java

+                .header(CONTENT_TYPE_HEADER, multipartBuilder.getHttpContentType().getContentType())
+                .retrieve();
+
+        if (response.statusCode() != HttpURLConnection.HTTP_OK && response.statusCode() != HttpURLConnection.HTTP_CREATED) {


Is it possible to return either 200 or 201? It seems like it should be one or the other.

Yes, it can be one or the other depending on potential existing content. I updated the logic to make it easier to understand.

exceptionfactory · 2025-10-29T19:03:53Z

...-extensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketRepositoryClient.java

+                throw new FlowRegistryException(String.format("Could not parse Bitbucket API response at %s", uri), e);
+            }
+
+            final JsonNode values = root.path("values");


It looks like there are a number of repeated uses of certain expected field names, which could be useful to declare as static final variables.

pvillard31 · 2025-10-29T22:08:57Z

Thanks for the review @exceptionfactory - pushed a commit addressing your comments

exceptionfactory

Thanks for the updates @pvillard31. I noted what appear to be a few opportunities to define reusable HTTP request and JSON response parsing, otherwise this looks close to completion.

exceptionfactory · 2025-10-31T19:05:49Z

...-extensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketRepositoryClient.java

+        if (formFactor == BitbucketFormFactor.DATA_CENTER) {
+            return commit.path("id").asText("");
+        }
+        return commit.path("hash").asText("");


It looks like "" could be declared as EMPTY_STRING and reused.

exceptionfactory · 2025-10-31T19:06:29Z

...-extensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketRepositoryClient.java

+            throw new FlowRegistryException("Could not parse response from Bitbucket API", e);
        }
+
+        final JsonNode values = jsonResponse.get("values");


It looks like values is used in a number of methods.

exceptionfactory · 2025-10-31T19:07:11Z

...-extensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketRepositoryClient.java

+    }
+
+    private boolean isFileEntry(final JsonNode entry) {
+        final JsonNode typeNode = entry.get("type");


type and path appear to be used in a number of places.

exceptionfactory · 2025-10-31T19:08:39Z

...-extensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketRepositoryClient.java

+        if (formFactor == BitbucketFormFactor.DATA_CENTER) {
+            return "FILE".equalsIgnoreCase(type);
+        }
+        return "commit_file".equals(type);


Can FILE be upper or lower case?

exceptionfactory · 2025-10-31T19:09:52Z

...-extensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketRepositoryClient.java

+            final JsonNode root;
+            try {
+                root = objectMapper.readTree(response.body());
+            } catch (IOException e) {
+                throw new FlowRegistryException(String.format("Could not parse Bitbucket API response at %s", uri), e);
+            }


This seems to be a common pattern, it would be helpful to create a shared method and reuse instead of copy the try-catch block across methods.

exceptionfactory · 2025-10-31T19:10:51Z

...-extensions/src/main/java/org/apache/nifi/atlassian/bitbucket/BitbucketRepositoryClient.java

+            final HttpResponseEntity response = this.webClient.getWebClientService()
+                    .get()
+                    .uri(uri)
+                    .header(AUTHORIZATION_HEADER, authToken.getAuthzHeaderValue())
+                    .retrieve();
+
+            if (response.statusCode() != HttpURLConnection.HTTP_OK) {
+                final String errorMessage = String.format("Error while listing commits for repository [%s] on branch %s", repoName, branch);
+                throw new FlowRegistryException(errorMessage + ": " + getErrorMessage(response));
+            }


It seems like there is an opportunity to have a reusable method that checks for the expected status code.

exceptionfactory · 2025-10-31T19:12:02Z

...api/src/main/java/org/apache/nifi/web/client/api/StandardMultipartFormDataStreamBuilder.java

+        final List<InputStream> streams = new ArrayList<>();
+        for (int index = 0; index < parts.size(); index++) {
+            final Part part = parts.get(index);
+            streams.add(new ByteArrayInputStream(getBoundaryPrefix(index).getBytes(HEADERS_CHARACTER_SET)));


Recommend declaring the return of getBoundaryPrefix as a separate variable for readability instead of the nested call and conversion to a byte array.

pvillard31 · 2025-10-31T22:17:57Z

Thanks @exceptionfactory - pushed another commit

pvillard31 force-pushed the NIFI-14968 branch from be14768 to 5f44977 Compare September 18, 2025 18:15

NIFI-14968 - Add support for Bitbucket Data Center

8bcc4bf

pvillard31 force-pushed the NIFI-14968 branch from 5f44977 to 8bcc4bf Compare September 18, 2025 18:49

exceptionfactory requested changes Oct 29, 2025

View reviewed changes

pvillard31 added 2 commits October 29, 2025 22:16

Merge branch 'main' of https://github.com/apache/nifi into NIFI-14968

248d8ef

review

1d3641e

exceptionfactory requested changes Oct 31, 2025

View reviewed changes

review

fa6b5c3

		final String boundaryPrefix = (index == 0 ? BOUNDARY_SEPARATOR + boundary + CARRIAGE_RETURN_LINE_FEED
		: CARRIAGE_RETURN_LINE_FEED + BOUNDARY_SEPARATOR + boundary + CARRIAGE_RETURN_LINE_FEED);

	BASIC_AUTH("Basic Auth & API Token", """
	BASIC_AUTH("Basic Auth and API Token", """

NIFI-14968 - Add support for Bitbucket Data Center #10322

Are you sure you want to change the base?

NIFI-14968 - Add support for Bitbucket Data Center #10322

Conversation

pvillard31 commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tracking

Issue Tracking

Pull Request Tracking

Pull Request Formatting

Verification

Build

Licensing

Documentation

Uh oh!

exceptionfactory left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pvillard31 commented Oct 29, 2025

Uh oh!

exceptionfactory left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pvillard31 commented Oct 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pvillard31 commented Sep 18, 2025 •

edited

Loading