[ABI Validation] Klib support

Added experimental Klib ABI dump validation support. Validation could be performed using the Gradle plugin which provides almost the same validation and dump workflow as for JVM ABI validation. Also, a public API was added to allow users who don't want or can't use the plugin to implement their own workflows. --------- Co-authored-by: Leonid Startsev <[email protected]> Co-authored-by: ilya-g <[email protected]> Pull request Kotlin/binary-compatibility-validator#183 Moved from Kotlin/binary-compatibility-validator@ad1bea6
JetBrains · Jan 3, 2025 · 6e7a2a4 · 6e7a2a4
1 parent 3994275
commit 6e7a2a4
Show file tree

Hide file tree

Showing 107 changed files with 6,696 additions and 146 deletions.
diff --git a/libraries/tools/abi-validation/README.md b/libraries/tools/abi-validation/README.md
@@ -13,6 +13,7 @@ The tool allows dumping binary API of a JVM part of a Kotlin library that is pub
   * [Tasks](#tasks)
   * [Optional parameters](#optional-parameters)
   * [Workflow](#workflow)
+  * [Experimental KLib ABI validation support](#experimental-klib-abi-validation-support)
 * [What constitutes the public API](#what-constitutes-the-public-api)
   * [Classes](#classes)
   * [Members](#members)
@@ -175,6 +176,59 @@ When starting to validate your library public API, we recommend the following wo
     the resulting diff in `.api` file should be verified: only signatures you expected to change should be changed.
   * Commit the resulting `.api` diff along with code changes. 
 
+### Experimental KLib ABI validation support
+
+The KLib validation support is experimental and is a subject to change (applies to both an API and the ABI dump format).
+A project has to use Kotlin 1.9.20 or newer to use this feature.
+
+To validate public ABI of a Kotlin library (KLib) corresponding option should be enabled explicitly:
+```kotlin
+apiValidation {
+    @OptIn(kotlinx.validation.ExperimentalBCVApi::class)
+    klib {
+        enabled = true
+    }
+}
+```
+
+When enabled, KLib support adds additional dependencies to existing `apiDump` and `apiCheck` tasks.
+Generate KLib ABI dumps are places alongside JVM dumps (in `api` subfolder, by default) 
+in files named `<project name>.klib.api`.
+The dump file combines all dumps generated for individual targets with declarations specific to some targets being
+annotated with corresponding target names.
+During the validation phase, that file is compared to the dump extracted from the latest version of the library, 
+and any differences between these two files are reported as errors.
+
+Currently, all options described in [Optional parameters](#optional-parameters) section are supported for klibs too.
+The only caveat here is that all class names should be specified in the JVM-format,
+like `package.name.ClassName$SubclassName`.
+
+Please refer to a [design document](docs/design/KLibSupport.md) for details on the format and rationale behind the 
+current implementation.
+
+#### KLib ABI dump generation and validation on Linux and Windows hosts
+
+Currently, compilation to Apple-specific targets (like `iosArm64` or `watchosX86`) supported only on Apple hosts.
+To ease the development on Windows and Linux hosts, binary compatibility validator does not validate ABI for targets
+not supported on the current host, even if `.klib.api` file contains declarations for these targets.
+
+This behavior could be altered to force an error when klibs for some targets could not be compiled:
+```kotlin
+apiValidation {
+    @OptIn(kotlinx.validation.ExperimentalBCVApi::class)
+    klib {
+        enabled = true
+        // treat a target being unsupported on a host as an error
+        strictValidation = true
+    }
+}
+```
+
+When it comes to dump generation (`apiDump` task) on non-Apple hosts, binary compatibility validator attempts
+to infer an ABI from dumps generated for supported targets and an old dump from project's `api` folder (if any).
+Inferred dump may not match an actual dump,
+and it is recommended to update a dump on hosts supporting all required targets, if possible. 
+
 # What constitutes the public API
 
 ### Classes

diff --git a/libraries/tools/abi-validation/api/binary-compatibility-validator.api b/libraries/tools/abi-validation/api/binary-compatibility-validator.api
@@ -5,11 +5,13 @@ public class kotlinx/validation/ApiValidationExtension {
 	public final fun getIgnoredClasses ()Ljava/util/Set;
 	public final fun getIgnoredPackages ()Ljava/util/Set;
 	public final fun getIgnoredProjects ()Ljava/util/Set;
+	public final fun getKlib ()Lkotlinx/validation/KlibValidationSettings;
 	public final fun getNonPublicMarkers ()Ljava/util/Set;
 	public final fun getPublicClasses ()Ljava/util/Set;
 	public final fun getPublicMarkers ()Ljava/util/Set;
 	public final fun getPublicPackages ()Ljava/util/Set;
 	public final fun getValidationDisabled ()Z
+	public final fun klib (Lkotlin/jvm/functions/Function1;)V
 	public final fun setAdditionalSourceSets (Ljava/util/Set;)V
 	public final fun setApiDumpDirectory (Ljava/lang/String;)V
 	public final fun setIgnoredClasses (Ljava/util/Set;)V
@@ -28,32 +30,59 @@ public final class kotlinx/validation/BinaryCompatibilityValidatorPlugin : org/g
 	public fun apply (Lorg/gradle/api/Project;)V
 }
 
+public abstract class kotlinx/validation/BuildTaskBase : org/gradle/api/DefaultTask {
+	public field outputApiFile Ljava/io/File;
+	public fun <init> ()V
+	public final fun getIgnoredClasses ()Ljava/util/Set;
+	public final fun getIgnoredPackages ()Ljava/util/Set;
+	public final fun getNonPublicMarkers ()Ljava/util/Set;
+	public final fun getOutputApiFile ()Ljava/io/File;
+	public final fun getPublicClasses ()Ljava/util/Set;
+	public final fun getPublicMarkers ()Ljava/util/Set;
+	public final fun getPublicPackages ()Ljava/util/Set;
+	public final fun setIgnoredClasses (Ljava/util/Set;)V
+	public final fun setIgnoredPackages (Ljava/util/Set;)V
+	public final fun setNonPublicMarkers (Ljava/util/Set;)V
+	public final fun setOutputApiFile (Ljava/io/File;)V
+	public final fun setPublicClasses (Ljava/util/Set;)V
+	public final fun setPublicMarkers (Ljava/util/Set;)V
+	public final fun setPublicPackages (Ljava/util/Set;)V
+}
+
+public abstract interface annotation class kotlinx/validation/ExperimentalBCVApi : java/lang/annotation/Annotation {
+}
+
 public abstract interface annotation class kotlinx/validation/ExternalApi : java/lang/annotation/Annotation {
 }
 
-public class kotlinx/validation/KotlinApiBuildTask : org/gradle/api/DefaultTask {
+public class kotlinx/validation/KlibValidationSettings {
+	public fun <init> ()V
+	public final fun getEnabled ()Z
+	public final fun getSignatureVersion ()Lkotlinx/validation/api/klib/KlibSignatureVersion;
+	public final fun getStrictValidation ()Z
+	public final fun setEnabled (Z)V
+	public final fun setSignatureVersion (Lkotlinx/validation/api/klib/KlibSignatureVersion;)V
+	public final fun setStrictValidation (Z)V
+}
+
+public class kotlinx/validation/KotlinApiBuildTask : kotlinx/validation/BuildTaskBase {
 	public field inputDependencies Lorg/gradle/api/file/FileCollection;
-	public field outputApiDir Ljava/io/File;
 	public fun <init> ()V
 	public final fun getInputClassesDirs ()Lorg/gradle/api/file/FileCollection;
 	public final fun getInputDependencies ()Lorg/gradle/api/file/FileCollection;
 	public final fun getInputJar ()Lorg/gradle/api/file/RegularFileProperty;
-	public final fun getOutputApiDir ()Ljava/io/File;
 	public final fun setInputClassesDirs (Lorg/gradle/api/file/FileCollection;)V
 	public final fun setInputDependencies (Lorg/gradle/api/file/FileCollection;)V
-	public final fun setOutputApiDir (Ljava/io/File;)V
 }
 
 public class kotlinx/validation/KotlinApiCompareTask : org/gradle/api/DefaultTask {
-	public field apiBuildDir Ljava/io/File;
+	public field generatedApiFile Ljava/io/File;
+	public field projectApiFile Ljava/io/File;
 	public fun <init> (Lorg/gradle/api/model/ObjectFactory;)V
-	public final fun getApiBuildDir ()Ljava/io/File;
-	public final fun getDummyOutputFile ()Ljava/io/File;
-	public final fun getNonExistingProjectApiDir ()Ljava/lang/String;
-	public final fun getProjectApiDir ()Ljava/io/File;
-	public final fun setApiBuildDir (Ljava/io/File;)V
-	public final fun setNonExistingProjectApiDir (Ljava/lang/String;)V
-	public final fun setProjectApiDir (Ljava/io/File;)V
+	public final fun getGeneratedApiFile ()Ljava/io/File;
+	public final fun getProjectApiFile ()Ljava/io/File;
+	public final fun setGeneratedApiFile (Ljava/io/File;)V
+	public final fun setProjectApiFile (Ljava/io/File;)V
 }
 
 public final class kotlinx/validation/api/ClassBinarySignature {
@@ -79,3 +108,82 @@ public final class kotlinx/validation/api/KotlinSignaturesLoadingKt {
 	public static synthetic fun retainExplicitlyIncludedIfDeclared$default (Ljava/util/List;Ljava/util/Collection;Ljava/util/Collection;Ljava/util/Collection;ILjava/lang/Object;)Ljava/util/List;
 }
 
+public final class kotlinx/validation/api/klib/KlibDump {
+	public static final field Companion Lkotlinx/validation/api/klib/KlibDump$Companion;
+	public fun <init> ()V
+	public final fun copy ()Lkotlinx/validation/api/klib/KlibDump;
+	public final fun getTargets ()Ljava/util/Set;
+	public final fun merge (Ljava/io/File;Ljava/lang/String;)V
+	public final fun merge (Lkotlinx/validation/api/klib/KlibDump;)V
+	public static synthetic fun merge$default (Lkotlinx/validation/api/klib/KlibDump;Ljava/io/File;Ljava/lang/String;ILjava/lang/Object;)V
+	public final fun remove (Ljava/lang/Iterable;)V
+	public final fun retain (Ljava/lang/Iterable;)V
+	public final fun saveTo (Ljava/lang/Appendable;)V
+}
+
+public final class kotlinx/validation/api/klib/KlibDump$Companion {
+	public final fun from (Ljava/io/File;Ljava/lang/String;)Lkotlinx/validation/api/klib/KlibDump;
+	public static synthetic fun from$default (Lkotlinx/validation/api/klib/KlibDump$Companion;Ljava/io/File;Ljava/lang/String;ILjava/lang/Object;)Lkotlinx/validation/api/klib/KlibDump;
+	public final fun fromKlib (Ljava/io/File;Ljava/lang/String;Lkotlinx/validation/api/klib/KlibDumpFilters;)Lkotlinx/validation/api/klib/KlibDump;
+	public static synthetic fun fromKlib$default (Lkotlinx/validation/api/klib/KlibDump$Companion;Ljava/io/File;Ljava/lang/String;Lkotlinx/validation/api/klib/KlibDumpFilters;ILjava/lang/Object;)Lkotlinx/validation/api/klib/KlibDump;
+}
+
+public final class kotlinx/validation/api/klib/KlibDumpFilters {
+	public static final field Companion Lkotlinx/validation/api/klib/KlibDumpFilters$Companion;
+	public final fun getIgnoredClasses ()Ljava/util/Set;
+	public final fun getIgnoredPackages ()Ljava/util/Set;
+	public final fun getNonPublicMarkers ()Ljava/util/Set;
+	public final fun getSignatureVersion ()Lkotlinx/validation/api/klib/KlibSignatureVersion;
+}
+
+public final class kotlinx/validation/api/klib/KlibDumpFilters$Builder {
+	public fun <init> ()V
+	public final fun build ()Lkotlinx/validation/api/klib/KlibDumpFilters;
+	public final fun getIgnoredClasses ()Ljava/util/Set;
+	public final fun getIgnoredPackages ()Ljava/util/Set;
+	public final fun getNonPublicMarkers ()Ljava/util/Set;
+	public final fun getSignatureVersion ()Lkotlinx/validation/api/klib/KlibSignatureVersion;
+	public final fun setSignatureVersion (Lkotlinx/validation/api/klib/KlibSignatureVersion;)V
+}
+
+public final class kotlinx/validation/api/klib/KlibDumpFilters$Companion {
+	public final fun getDEFAULT ()Lkotlinx/validation/api/klib/KlibDumpFilters;
+}
+
+public final class kotlinx/validation/api/klib/KlibDumpFiltersKt {
+	public static final fun KLibDumpFilters (Lkotlin/jvm/functions/Function1;)Lkotlinx/validation/api/klib/KlibDumpFilters;
+}
+
+public final class kotlinx/validation/api/klib/KlibDumpKt {
+	public static final fun inferAbi (Lkotlinx/validation/api/klib/KlibTarget;Ljava/lang/Iterable;Lkotlinx/validation/api/klib/KlibDump;)Lkotlinx/validation/api/klib/KlibDump;
+	public static synthetic fun inferAbi$default (Lkotlinx/validation/api/klib/KlibTarget;Ljava/lang/Iterable;Lkotlinx/validation/api/klib/KlibDump;ILjava/lang/Object;)Lkotlinx/validation/api/klib/KlibDump;
+	public static final fun mergeFromKlib (Lkotlinx/validation/api/klib/KlibDump;Ljava/io/File;Ljava/lang/String;Lkotlinx/validation/api/klib/KlibDumpFilters;)V
+	public static synthetic fun mergeFromKlib$default (Lkotlinx/validation/api/klib/KlibDump;Ljava/io/File;Ljava/lang/String;Lkotlinx/validation/api/klib/KlibDumpFilters;ILjava/lang/Object;)V
+	public static final fun saveTo (Lkotlinx/validation/api/klib/KlibDump;Ljava/io/File;)V
+}
+
+public final class kotlinx/validation/api/klib/KlibSignatureVersion {
+	public static final field Companion Lkotlinx/validation/api/klib/KlibSignatureVersion$Companion;
+	public fun equals (Ljava/lang/Object;)Z
+	public fun hashCode ()I
+	public fun toString ()Ljava/lang/String;
+}
+
+public final class kotlinx/validation/api/klib/KlibSignatureVersion$Companion {
+	public final fun getLATEST ()Lkotlinx/validation/api/klib/KlibSignatureVersion;
+	public final fun of (I)Lkotlinx/validation/api/klib/KlibSignatureVersion;
+}
+
+public final class kotlinx/validation/api/klib/KlibTarget {
+	public static final field Companion Lkotlinx/validation/api/klib/KlibTarget$Companion;
+	public fun equals (Ljava/lang/Object;)Z
+	public final fun getConfigurableName ()Ljava/lang/String;
+	public final fun getTargetName ()Ljava/lang/String;
+	public fun hashCode ()I
+	public fun toString ()Ljava/lang/String;
+}
+
+public final class kotlinx/validation/api/klib/KlibTarget$Companion {
+	public final fun parse (Ljava/lang/String;)Lkotlinx/validation/api/klib/KlibTarget;
+}
+
diff --git a/libraries/tools/abi-validation/build.gradle.kts b/libraries/tools/abi-validation/build.gradle.kts
@@ -62,6 +62,7 @@ val createClasspathManifest = tasks.register("createClasspathManifest") {
 dependencies {
     implementation(gradleApi())
     implementation(libs.kotlinx.metadata)
+    compileOnly(libs.kotlin.compiler.embeddable)
     implementation(libs.ow2.asm)
     implementation(libs.ow2.asmTree)
     implementation(libs.javaDiffUtils)
@@ -78,7 +79,6 @@ dependencies {
 
 tasks.compileKotlin {
     compilerOptions {
-        freeCompilerArgs.add("-Xexplicit-api=strict")
         allWarningsAsErrors.set(true)
         @Suppress("DEPRECATION") // Compatibility with Gradle 7 requires Kotlin 1.4
         languageVersion.set(KotlinVersion.KOTLIN_1_4)
@@ -87,7 +87,9 @@ tasks.compileKotlin {
         // Suppressing "w: Language version 1.4 is deprecated and its support will be removed" message
         // because LV=1.4 in practice is mandatory as it is a default language version in Gradle 7.0+ for users' kts scripts.
         freeCompilerArgs.addAll(
-            "-Xsuppress-version-warnings"
+            "-Xexplicit-api=strict",
+            "-Xsuppress-version-warnings",
+            "-Xopt-in=kotlin.RequiresOptIn"
         )
     }
 }
@@ -162,6 +164,7 @@ testing {
                 implementation(project())
                 implementation(libs.assertJ.core)
                 implementation(libs.kotlin.test)
+                implementation(libs.kotlin.compiler.embeddable)
             }
         }