Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(spdx): Expose the validation of license text files' existence #9523

Closed
wants to merge 1 commit into from
Closed

feat(spdx): Expose the validation of license text files' existence #9523

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions utils/spdx/src/main/kotlin/Utils.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,15 +116,42 @@ fun getLicenseText(
licenseTextDirectories: List<File> = emptyList()
): String? = getLicenseTextReader(id, handleExceptions, addScanCodeLicenseTextsDir(licenseTextDirectories))?.invoke()

/**
* Get a lambda that allows to read the full text of the license with the provided SPDX [id]. Several locations are
* searched for the license text: [licenseTextDirectories], the ScanCode license texts directory, the ORT's list of SPDX
* licenses and the resources of this module. If [handleExceptions] is enabled, the [id] may also refer to a license
* with an SPDX exception.
*/
fun getLicenseTextReader(
id: String,
handleExceptions: Boolean = false,
licenseTextDirectories: List<File> = emptyList()
): (() -> String)? {
return getLicenseTextReader(id, handleExceptions, licenseTextDirectories) { dir, filename ->
dir.resolve(filename).takeIf { it.isFile }
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered refactoring the function to stop returning a lambda, in favour of the actual text / String?

Copy link
Member Author

@nnobelis nnobelis Nov 29, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, what would be the motivation to do it ? Just for code cleanup ?

BTW, do you know why it was decided to return a lambda in the first place ?

EDIT: I think @mnonnenmacher wrote the implementation. Could you please share your opinion on this ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, do you know why it was decided to return a lambda in the first place ?

Git blame is your friend, the rationale is in the commit message of 45a3087.

Copy link
Member

@fviernau fviernau Nov 29, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, what would be the motivation to do it ?

Motivation IMO would be to have a cleaner / simpler (public) API.
The approach to do this lazily could still be kept, but by an ORT internal function.

(To me this API looks a bit more complicated than necessary / not clean enough, which I believe it should be fixed before made public).

I don't understand why exposing something like the following does not suffice (which I believe we already have).

interface LicenseTextReader {
    fun getLicenseText(licenseId: String, handleExceptions: Boolean)
}

fun getLicenseTextReader(textDirectories: List<File>)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fviernau I think this is already the case. The public function hides the lambda return type https://github.com/boschglobal/oss-review-toolkit/blob/ed2f67ed59a0d7970ab7d4efb0767622cef05659/utils/spdx/src/main/kotlin/Utils.kt#L113.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR exposes return type (() -> String)? of getLicenseTextReader(), doesn't it ?

Copy link
Member Author

@nnobelis nnobelis Nov 29, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but this is because the LicenseTextProvider.getLicenseTextReader also exposes this return type:

ort/reporter/src/main/kotlin/LicenseTextProvider.kt

Line 36 in 6a7d63d

fun getLicenseTextReader(licenseId: String): (() -> String)?

getLicenseText -> String?
getLicenseTextReader -> (() -> String)?

Therefore this is consistent with the current state of the implementation.


/**
* Get a lambda that allows to read the full text of the license with the provided SPDX [id]. Several locations are
* searched for the license text: [licenseTextDirectories], the ScanCode license texts directory, the ORT's list of SPDX
* licenses and the resources of this module. If [handleExceptions] is enabled, the [id] may also refer to a license
* with an SPDX exception.
* [validateFile] is a lambda that is called with the directory and filename to look if the license file exists. It
* should return the file if it exists, null otherwise.
*/
fun getLicenseTextReader(
id: String,
handleExceptions: Boolean = false,
licenseTextDirectories: List<File> = emptyList(),
validateFile: (File, String) -> File?
): (() -> String)? {
return if (id.startsWith(LICENSE_REF_PREFIX)) {
getLicenseTextResource(id)?.let { { it.readText() } }
?: addScanCodeLicenseTextsDir(licenseTextDirectories).firstNotNullOfOrNull { dir ->
getLicenseTextFile(id, dir)?.let { file ->
getLicenseTextFile(id) { filename ->
validateFile(dir, filename)
}?.let { file ->
{
file.readText().removeYamlFrontMatter()
}
Expand All @@ -140,7 +167,7 @@ private fun getLicenseTextResource(id: String): URL? = object {}.javaClass.getRe

private val LICENSE_REF_FILENAME_REGEX by lazy { Regex("^$LICENSE_REF_PREFIX\\w+-") }

private fun getLicenseTextFile(id: String, dir: File): File? =
fun getLicenseTextFile(id: String, validateFile: (String) -> File?): File? =
id.replace(LICENSE_REF_FILENAME_REGEX, "").let { idWithoutLicenseRefNamespace ->
listOfNotNull(
id,
Expand All @@ -153,7 +180,7 @@ private fun getLicenseTextFile(id: String, dir: File): File? =
id == "LicenseRef-scancode-x11-xconsortium-veillard"
}
).firstNotNullOfOrNull { filename ->
dir.resolve(filename).takeIf { it.isFile }
validateFile(filename)
}
}

Expand Down
16 changes: 16 additions & 0 deletions utils/spdx/src/test/kotlin/UtilsTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,22 @@ class UtilsTest : WordSpec() {
}
}

"getLicenseTextReader provided a custom dir" should {
"call the custom validation lambda if provided" {
val id = "LicenseRef-ort-abc"
val text = "a\nb\nc"
val candidateFilenames = mutableListOf<String>()

setupTempFile(id, text)

getLicenseTextReader(id, true, listOf(tempDir)) { directory, filename ->
candidateFilenames += filename
directory.resolve(filename).takeIf { it.isFile }
}?.invoke() shouldBe text
candidateFilenames shouldBe listOf("LicenseRef-ort-abc")
}
}

"removeYamlFrontMatter" should {
"remove a YAML front matter" {
val text = """
Expand Down
Loading