Gradle example broken/output not parseable

[SebastianSchildt]

Not an expert in Dash nor java/gradle.

I am doing this

./gradlew kuksa-sdk:dependencies --configuration releaseCompileClasspath | ggrep -Poh "(?<=\s)[\w\.-]+:[\w\.-]+:[^:\s]+" | grep -v "^org\.eclipse" | sort | uniq > DASHFILE

(ggrep because using gnu grep on mac, on Linux just use grep)

The resulting file has weird entries, see attached file:
DASHFILE.txt

specifically I guess the entries with {strictly in them look weird, they seem to follow a valid entry, i.e.

io.grpc:grpc-stub:1.56.1
io.grpc:grpc-stub:{strictly

I could "butcher them out", but as I said I am not really knowing what I am doing here, so maybe the grep fromt he example is just outdated? Somebody has a better version?

For reference the gradle output

dashraw.txt

It seems entries like this

+--- org.jetbrains.kotlin:kotlin-stdlib-common:{strictly 1.9.0} -> 1.9.0 (c)

make problems.

Any hints?

[waynebeaton]

It's long past time that I engage our friends at Gradle to get a little help with this. I'll start that discussion later today.

In the meantime, thanks for providing the example. Let's use that to refine the expression.

As indicated at the bottom of the Gradle output, any line with (c) at the end is a "dependency constraint, not a dependency. The dependency affected by the constraint occurs elsewhere in the tree." Similarly, the entries with "(*)" don't add any value. So we can skip these lines.

So... insert cat dashraw.txt | grep -Pv "\([c\8*]\) | ... in your expression to prune those out before grabbing anything else.

The bigger challenge is lines like this:

|         +--- com.google.errorprone:error_prone_annotations:2.11.0 -> 2.18.0

Which I believe means that the the resolved version is 2.18.0, but the regex matches 2.11.0.

This requires a little help from perl (I tried to use sed, but the lack of PCRE support always stumps me).

Something like this should do it.

./gradlew | grep -Poh "(?<=\-\-\- ).*" | grep -Pv "\([c\*]\)" | perl -pe 's/([\w\.\-]+):([\w\.\-]+):(?:[\w\.\-]+ -> )?([\w\.\-]+).*$/$1:$2:$3/gmi;t' | sort -u > DASHFILE

I'll update the README.

[SebastianSchildt]

Thank you for digging into this, I will check

Your regexp scare me a little, I fear looking too closely might trigger an RCE in my brain :)

[waynebeaton]

Your regexp scare me a little,

Good. I was trying for scary. 😃

[waynebeaton]

@SebastianSchildt

Do you have cycles to check the the suggestion for tooling Gradle builds in #295?

If yes, please add your comments there.

[SebastianSchildt]

Hi let me refer you to our resident gradle masters @wba2hi and @Chrylo

Can you guys check if the Readme changes make sense and/or alternatively share how you solved the dash file generation for the KUKSA Android components?

[wba2hi]

Can you guys check if the Readme changes make sense

Yes, I can check it shortly and give feedback in the corresponding ticket

and alternatively share how you solved the dash file generation for the KUKSA Android components

We fixed it by adapting the regular expression a bit, because we noticed it was not working well with all use cases / all different representations of libraries inside the dependency tree.

Check here for the changes we made and here for the resulting regular expression

We use this script to create a report for each subproject and then merge the reports together to get an "all-dependencies.txt" for the whole project, which we then use for the dash license check.

[waynebeaton]

I'm basically a n00b when it comes to Gradle, so forgive the very basic question...

How do I know what value to provide for $projectName ?

[wba2hi]

Sorry didn't have time to check yet, but I will and report back the result, e.g. if our approaches are comparable w.r.t. the "found libraries".
Since our approach is still pretty project specific (there might still be other library representations which might be introduced in future or simply can not be found in our project) I could imagine that the approach used in the PR might be more future proof when it comes to the evaluation of the dependency tree.

How do I know what value to provide for $projectName?

Normally the subprojects are defined in settings.gradle.

Our script basically executes the dash.sh for each subproject (kuksa-sdk, vss-processor, app,...) to make sure all dependencies of all (sub)projects are taken into account then merges all reports into a single "all-dependencies.txt" report.

[wba2hi]

@waynebeaton
Comments were added: https://github.com/eclipse/dash-licenses/pull/295/files

[nagyesta]

Hi Everyone,

sorry for resurrecting this old thread! Just in case someone is still facing issues with the Gradle use-case I have made a plugin that could be a good alternative: https://github.com/nagyesta/run-dash-license-tool-gradle-plugin

I think it could be a lot more relevant than the current Bash script because the plugin is working with the actually resolved artifacts so it sidesteps the issue of "how the artifact got there".