Analyzer fails to authenticate with Artifactory when downloading artifacts (http-401)

[software-testing-professional]

Hi there,

I'm having trouble to authenticate against Artifactory when running ort analyze.

Credendials for Artifactory (username / password) are provided via .netrc file and Maven settings.xml.
But download attempts always result in a http-401 unauthorized.

Downloading these artifacts via cURL works fine, when I provide username / password.
So it seems that the configuration on Artifactory side is fine.

Did I miss some configuration? What can I do to solve this issue?
I appreciate your help! :-)

08:31:50.127 [DefaultDispatcher-worker-3] DEBUG org.ossreviewtoolkit.analyzer.managers.utils.MavenSupport - Remote location for 'external.c:openssl:jar:sources:1.1.1n': external/c/openssl/1.1.1n/openssl-1.1.1n-sources.jar
08:31:50.127 [DefaultDispatcher-worker-3] DEBUG org.eclipse.aether.internal.impl.DefaultTransporterProvider - Using transporter HttpTransporter with priority 5.0 for https://artifactory.*****.com/artifactory/its-external
08:31:50.127 [DefaultDispatcher-worker-3] DEBUG org.eclipse.aether.internal.impl.DefaultRepositoryConnectorProvider - Using connector BasicRepositoryConnector with priority 0.0 for https://artifactory.*****.com/artifactory/its-external
08:31:50.128 [DefaultDispatcher-worker-3] DEBUG org.apache.http.client.protocol.RequestAddCookies - CookieSpec selected: default
08:31:50.128 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection request: [route: {s}->https://artifactory.*****.com:443][total/ available: 2; route allocated: 1 of 50; total allocated: 2 of 100]
08:31:50.128 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection leased: [id: 1][route: {s}->https://artifactory.*****.com:443][total/ available: 1; route allocated: 1 of 50; total allocated: 2 of 100]
08:31:50.128 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-1: set socket timeout to 0
08:31:50.128 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-1: set socket timeout to 1800000
08:31:50.128 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.execchain.MainClientExec - Executing request HEAD /artifactory/its-external/external/c/openssl/1.1.1n/openssl-1.1.1n-sources.jar HTTP/1.1
08:31:50.128 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.execchain.MainClientExec - Target auth state: UNCHALLENGED
08:31:50.128 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED
08:31:50.154 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
08:31:50.154 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.auth.HttpAuthenticator - Authentication required
08:31:50.154 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.auth.HttpAuthenticator - artifactory.*****.com:443 requested authentication
08:31:50.154 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Authentication schemes in the order of preference: [Negotiate, Kerberos, NTLM, CredSSP, Digest, Basic]
08:31:50.155 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for Negotiate authentication scheme not available
08:31:50.155 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for Kerberos authentication scheme not available
08:31:50.155 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for NTLM authentication scheme not available
08:31:50.155 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for CredSSP authentication scheme not available
08:31:50.155 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for Digest authentication scheme not available
08:31:50.155 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection [id: 1][route: {s}->https://artifactory.*****.com:[443](https://gitlab.*****.com/mocca/oss-review-toolkit/-/jobs/5359208#L443)] can be kept alive indefinitely
08:31:50.155 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-1: set socket timeout to 0
08:31:50.155 [DefaultDispatcher-worker-3] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 1][route: {s}->https://artifactory.*****.com:443]/[total available: 2; route allocated: 1 of 50; total allocated: 2 of 100]
08:31:50.155 [DefaultDispatcher-worker-3] WARN  org.apache.http.client.protocol.ResponseProcessCookies - Invalid cookie header: "Set-Cookie: AWSALBTG=wXE1osuo2pSAo+9NrSYXQ0EgQ2RySOklfj2QVhTXb2XX8HUxx5SaEIuyM0g+1x0pNOGyJRicNpJu9twEyDD33tSMSP9Y1ErNosFyl01UlYBi14GuJKNcVbUymYQIuzH67Osj5QbGasz4uEtYTYXYnLmrRmZgASaGHeoNH6JETQIxu72H***=; Expires=Wed, 06 Jul 2022 08:31:50 GMT; Path=/". Invalid 'expires' attribute: Wed, 06 Jul 2022 08:31:50 GMT
08:31:50.155 [DefaultDispatcher-worker-3] WARN  org.apache.http.client.protocol.ResponseProcessCookies - Invalid cookie header: "Set-Cookie: AWSALBTGCORS=wXE1osuo2pSAo+9NrSYXQ0EgQ2RySOklfj2QVhTXb2XX8HUxx5SaEIuyM0g+1x0pNOGyJRicNpJu9twEyDD33tSMSP9Y1ErNosFyl01UlYBi14GuJKNcVbUymYQIuzH67Osj5QbGasz4uEtYTYXYnLmrRmZgASaGHeoNH6JETQIxu72H***=; Expires=Wed, 06 Jul 2022 08:31:50 GMT; Path=/; SameSite=None; Secure". Invalid 'expires' attribute: Wed, 06 Jul 2022 08:31:50 GMT
08:31:50.156 [DefaultDispatcher-worker-3] WARN  org.apache.http.client.protocol.ResponseProcessCookies - Invalid cookie header: "Set-Cookie: AWSALB=Aj+xDryH81fx1LpTd/dzakMUkCUkt999yNNXJF8kysW7vCWHmFINz4B1EKXdDg+QDsp61KiKbnP3qvBQP21oJMEyTFPnDTQRNl/KXxIoojVo0DDjX7niHOIXhG4a; Expires=Wed, 06 Jul 2022 08:31:50 GMT; Path=/". Invalid 'expires' attribute: Wed, 06 Jul 2022 08:31:50 GMT
08:31:50.156 [DefaultDispatcher-worker-3] WARN  org.apache.http.client.protocol.ResponseProcessCookies - Invalid cookie header: "Set-Cookie: AWSALBCORS=Aj+xDryH81fx1LpTd/dzakMUkCUkt999yNNXJF8kysW7vCWHmFINz4B1EKXdDg+QDsp61KiKbnP3qvBQP21oJMEyTFPnDTQRNl/KXxIoojVo0DDjX7niHOIXh***; Expires=Wed, 06 Jul 2022 08:31:50 GMT; Path=/; SameSite=None; Secure". Invalid 'expires' attribute: Wed, 06 Jul 2022 08:31:50 GMT
08:31:50.156 [DefaultDispatcher-worker-3] DEBUG org.ossreviewtoolkit.analyzer.managers.utils.MavenSupport - Transfer failed: GET_EXISTENCE FAILED https://artifactory.*****.com/artifactory/its-external/external/c/openssl/1.1.1n/openssl-1.1.1n-sources.jar <> /root/.m2/repository/external/c/openssl/1.1.1n/openssl-1.1.1n-sources.jar
08:31:50.156 [DefaultDispatcher-worker-3] DEBUG org.ossreviewtoolkit.analyzer.managers.utils.MavenSupport - Could not find 'external.c:openssl:jar:sources:1.1.1n' in 'https://artifactory.*****.com/artifactory/its-external (https://artifactory.*****.com/artifactory/its-external, default, releases+snapshots)': ArtifactTransferException: Could not transfer artifact external.c:openssl:jar:sources:1.1.1n from/to https://artifactory.*****.com/artifactory/its-external (https://artifactory.*****.com/artifactory/its-external): status code: 401, reason phrase:  (401)
Caused by: HttpResponseException: status code: 401, reason phrase:  (401)
08:31:50.156 [DefaultDispatcher-worker-3] DEBUG org.ossreviewtoolkit.analyzer.managers.utils.MavenSupport - Unable to find 'external.c:openssl:jar:sources:1.1.1n' in any of [https://repo.maven.apache.org/maven2, https://artifactory.*****.com/artifactory/its-external].
08:31:50.156 [DefaultDispatcher-worker-3] DEBUG org.ossreviewtoolkit.analyzer.managers.utils.MavenSupport - Writing empty remote artifact for 'external.c:openssl:jar:sources:1.1.1n' to disk cache.

(i) Some information like URLs and header information have been obfuscated with ***

[sschuberth]

Downloading these artifacts via cURL works fine, when I provide username / password.

Are you using exactly the same URL as the ORT analyzer for this check? Because I recall there was a subtle difference in URLs that artifactory shows, which sometimes contain "api" as part of the path, and sometimes not. An IIRC in one of the cases the user's API key instead of e.g. an AD password needs to be used.

Maybe @MarcelBochtler remembers some more details.

[software-testing-professional]

Yes, the download link shown in Artifactory's native file browser matches the one used by ORT.
This link was used in my cURL test.

I tried both username / token and username / API key in the settings.xml.
Each time it resulted in a 401, wen ORT analyze was run.

[software-testing-professional]

I did some more investigation and found out, that the download requests performed by ORT do not reach Artifactory at all.
The access log of Artifactory does not show any download attempts at this time.
Could this be related to the invalid cookie messages? And the AWS application load balancer blocks all download requests because missing request attributes?

[sschuberth]

Could this be related to the invalid cookie messages?

Could be. Looks like this is more or less a know issue with the Apache HTTP client that the Maven resolver uses. The answer on SO has a solution on how to fix this for the Apache HTTP client directly, but we'd yet need to find out how to fix this for the Maven resolver / the client that the resolver uses.

[software-testing-professional]

Meanwhile we tried some configuration on the loadbalancer. But without effect.

The credentials used for these type of downloads (executed by MavenSupport class) are taken either from .netrc or ENV variables (ORT_HTTP_USERNAME and ORT_HTTP_PASSWORD), right?

Requests on Artifactory side look like

2022-07-01T10:17:55.773Z|2936dde2010f2f4e|79.219.239.209|non_authenticated_user|HEAD|/its-external/external/c/openvpn/2.5.1/openvpn-2.5.1.tar.xz|401|-1|0|1|Apache-Maven/3.8.5 (Java 11.0.15; Linux 5.13.0-44-generic)

Is there any way to add a custom request header to these requests? Might be worth a try.
I'm using Artifactory as httpStorageBackend for the scanner, with an Authorization header that holds a bearer token.
This works fine.

[software-testing-professional]

Might our issues be related to this?
https://github.com/oss-review-toolkit/ort/blob/main/analyzer/src/main/kotlin/managers/Gradle.kt#L225

Because we are only having authentication errors with Gradle builds.
And Artifactory says non_authenticated_user is requesting files.

[sschuberth]

Might our issues be related to this? https://github.com/oss-review-toolkit/ort/blob/main/analyzer/src/main/kotlin/managers/Gradle.kt#L225

@software-testing-professional Unfortunately, you didn't use a permalink, and by now that line points to if (!initScriptFile.delete()) {, which I believe is unrelated. Would you mind repeating which line you had in mind?

[software-testing-professional]

@sschuberth Sorry for that.
I found a code comment here:

ort/analyzer/src/main/kotlin/managers/Gradle.kt

Line 230 in aa942fa

// TODO: Also handle authentication and snapshot policy.

// TODO: Also handle authentication and snapshot policy.

This led me to the assumption that something related to authentication might still be missing.

[sschuberth]

Sorry for that.

No worries. Thanks for being quick in posting an update!

This led me to the assumption that something related to authentication might still be missing.

I'm currently unsure whether this old comment of @mnonnenmacher from 2017 is still valid.

@software-testing-professional looks like you're running ORT from a Docker image. Could you also try running ORT natively with the same configuration to rule out any Docker-related issues?

[software-testing-professional]

@sschuberth Sorry, I'm a bit late with my answer. ;-)
I also tried to use ORT natively. With same results.
Normally I use the ORT Docker image and mount all the configuration (settings.xml, .netrc, etc.) into the container.

The repository configuration is done via Gradle.
And the Artifactory repositories are configured as

maven {
          credentials {
              username lUsr
              password lPwd
          }
          url lUrl

Although everything was configured, Artifactory only got requests from an "unauthenticated" user.
Because the requested resource was part of the Gradle project, these requests definitely came from ORT.

But:
I did some more testing, and could solve the authentication issue by adding this to gradle.properties and mounting it into the container:

systemProp.http.proxyUser=<user>
systemProp.http.proxyPassword=<pw>

So currently the Gradle authentication still does not work. But solved via proxy authentication.

[sschuberth]

@software-testing-professional would you mind checking whether the current version of ORT that includes #6498 fixes the issue for you?

[software-testing-professional]

@sschuberth Yes, I'll be able to try that next week from Wednesday on. 👍

[sschuberth]

I can confirm that the scanner does not freeze anymore, if ClearlyDefined is defined as storageReader

Hmm, this sounds a bit as if you're confusing this issue with #4540 😉

But what about this:

I'm having trouble to authenticate against Artifactory when running ort analyze.

[software-testing-professional]

Ah right - that happens if you have too many open tabs. 😆
I'll move that answer over to the other issue.

Regarding authentication, this also works with the Docker-based ORT built from commit 19c89ff.

[sschuberth]

Regarding authentication, this also works with the Docker-based ORT built from commit 19c89ff.

Great, thanks for confirming, so I'll be closing this!