-
Notifications
You must be signed in to change notification settings - Fork 143
feat(client): support base64 embeddings and use as default #519
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This implementation adds support for Base64-encoded embeddings as the default response format, while maintaining complete backward compatibility with existing `List<Float>` usage. ### 1. Default Behavior Change - **New Default**: Embedding requests now default to Base64 encoding format - **Backward Compatibility**: Existing code using `embedding()` method continues to work unchanged - **Performance**: Base64 encoding reduces network payload size significantly Introduces the EmbeddingValue class to support both float list and base64-encoded embedding data, enabling efficient handling and backward compatibility. Embedding, EmbeddingCreateParams, and related classes are updated to use EmbeddingValue, with automatic decoding and encoding between formats. Adds EmbeddingDefaults for global default encoding configuration, and comprehensive tests for new behaviors and compatibility.
When verifying the operation in the Azure OpenAI environment, I was able to confirm the operation with the following code: package com.openai.example;
import com.openai.azure.AzureOpenAIServiceVersion;
import com.openai.azure.credential.AzureApiKeyCredential;
import com.openai.client.OpenAIClient;
import com.openai.client.OpenAIClientAsync;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.client.okhttp.OpenAIOkHttpClientAsync;
import com.openai.models.embeddings.CreateEmbeddingResponse;
import com.openai.models.embeddings.EmbeddingCreateParams;
import com.openai.models.embeddings.EmbeddingModel;
import com.openai.models.embeddings.EmbeddingValue;
import com.openai.services.blocking.EmbeddingService;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CountDownLatch;
/**
* Sample code using Azure OpenAI Embedding API
* Demonstrates how to retrieve and process embedding data in Base64 and Float formats
*/
public final class EmbeddingsExampleAzure {
private EmbeddingsExampleAzure() {}
// Azure OpenAI endpoint and key configuration
// Please replace with actual values
private static final String AZURE_OPENAI_ENDPOINT = "https://***********.openai.azure.com";
private static final String AZURE_OPENAI_KEY = "***********";
private static OpenAIClient client;
public static void main(String[] args) {
// Initialize Azure OpenAI client
client = OpenAIOkHttpClient.builder()
.baseUrl(AZURE_OPENAI_ENDPOINT)
.credential(AzureApiKeyCredential.create(AZURE_OPENAI_KEY))
.azureServiceVersion(AzureOpenAIServiceVersion.getV2024_02_15_PREVIEW())
.build();
EmbeddingsExampleAzure example = new EmbeddingsExampleAzure();
example.basicSample();
example.multipleDataSample();
example.asyncSample();
}
/**
* Basic embedding retrieval sample
* Demonstrates usage of default format, Float format, and Base64 format
*/
public void basicSample() {
EmbeddingService embeddings = client.embeddings();
String singlePoem = "In the quiet night, stars whisper secrets, dreams take flight.";
System.out.println("=== Basic Embedding Sample ===");
// 1. Default format (Base64 is default)
System.out.println("\n1. Getting embeddings in default format:");
EmbeddingCreateParams embeddingCreateParams = EmbeddingCreateParams.builder()
.input(singlePoem)
.model(EmbeddingModel.TEXT_EMBEDDING_3_SMALL.asString())
.build();
embeddings.create(embeddingCreateParams).data().forEach(embedding -> {
System.out.println("Embedding (default format): " + embedding.toString());
// Use EmbeddingValue to check the original format
EmbeddingValue embeddingValue = embedding.embeddingValue();
if (embeddingValue.isBase64String()) {
System.out.println(
"Received in Base64 format: " + embeddingValue.base64String().substring(0, 50) + "...");
} else if (embeddingValue.isFloatList()) {
System.out.println("Received in Float format: " + embeddingValue.floatList().size() + " elements");
}
// embedding() method always returns List<Float> (Base64 is automatically decoded)
List<Float> floats = embedding.embedding();
System.out.println(
"Retrieved as Float array: " + floats.size() + " elements, first 5: " + floats.subList(0, Math.min(5, floats.size())));
});
System.out.println("\n------------------------------------------------");
// 2. Explicitly specify Float format
System.out.println("\n2. Explicitly specifying Float format:");
EmbeddingCreateParams embeddingCreateParams2 = EmbeddingCreateParams.builder()
.input(singlePoem)
.model(EmbeddingModel.TEXT_EMBEDDING_3_SMALL.asString())
.encodingFormat(EmbeddingCreateParams.EncodingFormat.FLOAT)
.build();
embeddings.create(embeddingCreateParams2).data().forEach(embedding -> {
EmbeddingValue embeddingValue = embedding.embeddingValue();
if (embeddingValue.isFloatList()) {
System.out.println("Received in Float format: " + embeddingValue.floatList().size() + " elements");
System.out.println("First 5 values: "
+ embeddingValue
.floatList()
.subList(
0,
Math.min(5, embeddingValue.floatList().size())));
}
// Can also convert to Base64 format
String base64 = embeddingValue.asBase64String();
System.out.println("Converted to Base64 format: " + base64.substring(0, 50) + "...");
});
System.out.println("\n------------------------------------------------");
// 3. Explicitly specify Base64 format
System.out.println("\n3. Explicitly specifying Base64 format:");
EmbeddingCreateParams embeddingCreateParams3 = EmbeddingCreateParams.builder()
.input(singlePoem)
.model(EmbeddingModel.TEXT_EMBEDDING_3_SMALL.asString())
.encodingFormat(EmbeddingCreateParams.EncodingFormat.BASE64)
.build();
embeddings.create(embeddingCreateParams3).data().forEach(embedding -> {
EmbeddingValue embeddingValue = embedding.embeddingValue();
if (embeddingValue.isBase64String()) {
System.out.println(
"Received in Base64 format: " + embeddingValue.base64String().substring(0, 50) + "...");
}
// Automatically convert to Float array
List<Float> floats = embeddingValue.asFloatList();
System.out.println(
"Auto-converted to Float array: " + floats.size() + " elements, first 5: " + floats.subList(0, Math.min(5, floats.size())));
});
System.out.println("\n================================================");
}
/**
* Multiple data embedding retrieval sample
*/
public void multipleDataSample() {
EmbeddingService embeddings = client.embeddings();
System.out.println("\n=== Multiple Data Embedding Sample ===");
getPoems().forEach(poem -> {
System.out.println("\nPoem (start): " + poem);
EmbeddingCreateParams embeddingCreateParams = EmbeddingCreateParams.builder()
.input(poem)
.model(EmbeddingModel.TEXT_EMBEDDING_3_SMALL.asString())
.build(); // Use default format (Base64)
embeddings.create(embeddingCreateParams).data().forEach(embedding -> {
List<Float> floats = embedding.embedding();
System.out.println("Embedding (default): " + floats.size() + " dimensions");
// Check original format
EmbeddingValue embeddingValue = embedding.embeddingValue();
if (embeddingValue.isBase64String()) {
System.out.println("Original format: Base64");
} else {
System.out.println("Original format: Float array");
}
});
System.out.println("Poem (end)");
});
System.out.println("\n================================================");
}
/**
* Asynchronous embedding retrieval sample
*/
public void asyncSample() {
System.out.println("\n=== Asynchronous Embedding Sample ===");
CountDownLatch latch = new CountDownLatch(1);
try {
OpenAIClientAsync asyncClient = OpenAIOkHttpClientAsync.builder()
.baseUrl(AZURE_OPENAI_ENDPOINT)
.credential(AzureApiKeyCredential.create(AZURE_OPENAI_KEY))
.azureServiceVersion(AzureOpenAIServiceVersion.getV2024_02_15_PREVIEW())
.build();
CompletableFuture<CreateEmbeddingResponse> completableFuture = asyncClient
.embeddings()
.create(EmbeddingCreateParams.builder()
.input("The quick brown fox jumped over the lazy dog")
.model(EmbeddingModel.TEXT_EMBEDDING_3_SMALL)
.encodingFormat(EmbeddingCreateParams.EncodingFormat.FLOAT)
.user("user-1234")
.build());
completableFuture
.thenAccept(response -> {
response.validate();
response.data().forEach(embedding -> {
System.out.println("Asynchronous embedding retrieval completed:");
System.out.println("Embedding info: " + embedding.toString());
EmbeddingValue embeddingValue = embedding.embeddingValue();
if (embeddingValue.isFloatList()) {
System.out.println(
"Float format: " + embeddingValue.floatList().size() + " dimensions");
}
// Visitor pattern usage example
String result = embeddingValue.accept(new EmbeddingValue.Visitor<String>() {
@Override
public String visitFloatList(List<Float> floatList) {
return "Processing Float array: " + floatList.size() + " elements";
}
@Override
public String visitBase64String(String base64String) {
return "Processing Base64 string: " + base64String.length() + " characters";
}
});
System.out.println("Visitor pattern result: " + result);
latch.countDown();
});
})
.exceptionally(ex -> {
System.err.println("Error: " + ex.getMessage());
latch.countDown();
return null;
});
latch.await();
System.out.println("Asynchronous processing completed");
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
System.err.println("Processing was interrupted");
}
System.out.println("\n================================================");
System.out.println("All samples execution completed");
}
/**
* Get sample poems list
*/
private List<String> getPoems() {
List<String> poems = new ArrayList<>();
poems.add("In the quiet night, stars whisper secrets, dreams take flight.");
poems.add("Beneath the moon's glow, shadows dance, hearts begin to know.");
poems.add("Waves crash on the shore, time stands still, love forevermore.");
poems.add("Autumn leaves fall, painting the ground, nature's final call.");
poems.add("Morning dew glistens, a new day dawns, hope always listens.");
poems.add("Mountains stand tall, silent guardians, witnessing it all.");
poems.add("In a field of green, flowers bloom bright, a serene scene.");
poems.add("Winter's chill bites, fireside warmth, cozy, long nights.");
poems.add("Spring's gentle breeze, life awakens, hearts find ease.");
poems.add("Sunset hues blend, day meets night, a perfect end.");
return poems;
}
} |
Deleted invalid EmbeddingDefaults.kt
openai-java-core/src/main/kotlin/com/openai/models/embeddings/Embedding.kt
Outdated
Show resolved
Hide resolved
openai-java-core/src/main/kotlin/com/openai/models/embeddings/Embedding.kt
Outdated
Show resolved
Hide resolved
openai-java-core/src/main/kotlin/com/openai/models/embeddings/EmbeddingCreateParams.kt
Outdated
Show resolved
Hide resolved
openai-java-core/src/main/kotlin/com/openai/models/embeddings/EmbeddingValue.kt
Outdated
Show resolved
Hide resolved
openai-java-core/src/main/kotlin/com/openai/models/embeddings/EmbeddingValue.kt
Outdated
Show resolved
Hide resolved
openai-java-core/src/main/kotlin/com/openai/models/embeddings/EmbeddingDefaults.kt
Outdated
Show resolved
Hide resolved
openai-java-core/src/main/kotlin/com/openai/models/embeddings/EmbeddingValue.kt
Outdated
Show resolved
Hide resolved
openai-java-core/src/main/kotlin/com/openai/models/embeddings/EmbeddingValue.kt
Outdated
Show resolved
Hide resolved
openai-java-core/src/test/kotlin/com/openai/models/embeddings/EmbeddingDebugTest.kt
Outdated
Show resolved
Hide resolved
Modified the field name for floats and base64
Deleted DebutTest
* validate() methods validate anything the "shape" of the data being correct * The code has been refactored to use Kotlin's apply scope function for improved conciseness and consistency.
Modified default encoding
Modified the implementation of default encoding
Modified format
Delete plan2.md
Dear @TomerAberbach Today I have promptly addressed and implemented corrections for the points that could be immediately resolved based on the comments received from you. Could I kindly ask you to review the entire content once again? Regarding the very first point, there are several possible approaches we could take. For instance, the decision may vary depending on whether we prioritize existing users of the API or focus on the benefits for future users and the maintainability of the system moving forward. I would greatly appreciate it if you could consider which approach would be more appropriate and advise us on the preferred course of action. |
Thanks for getting this PR started! I ended up implementing the rest myself to save some back-and-forth. Had to clean up some things and make it so that the stored data model was Verified the implementation is correct against the API also: EmbeddingValue value1 = client.embeddings()
.create(EmbeddingCreateParams.builder()
.input("Hello World")
.model(EmbeddingModel.TEXT_EMBEDDING_ADA_002)
.build())
.data()
.get(0)
.embeddingValue();
EmbeddingValue value2 = client.embeddings()
.create(EmbeddingCreateParams.builder()
.input("Hello World")
.model(EmbeddingModel.TEXT_EMBEDDING_ADA_002)
.encodingFormat(EmbeddingCreateParams.EncodingFormat.FLOAT)
.build())
.data()
.get(0)
.embeddingValue();
// Both print `true`!
System.out.println(value1.asBase64().equals(value2.asBase64()));
System.out.println(value1.asFloats().equals(value2.asFloats())); |
* docs: fix missing readme comment * feat(client): support base64 embeddings and use as default (#519) * Add EmbeddingValue union type and Base64 support for embeddings This implementation adds support for Base64-encoded embeddings as the default response format, while maintaining complete backward compatibility with existing `List<Float>` usage. ### 1. Default Behavior Change - **New Default**: Embedding requests now default to Base64 encoding format - **Backward Compatibility**: Existing code using `embedding()` method continues to work unchanged - **Performance**: Base64 encoding reduces network payload size significantly Introduces the EmbeddingValue class to support both float list and base64-encoded embedding data, enabling efficient handling and backward compatibility. Embedding, EmbeddingCreateParams, and related classes are updated to use EmbeddingValue, with automatic decoding and encoding between formats. Adds EmbeddingDefaults for global default encoding configuration, and comprehensive tests for new behaviors and compatibility. * Deleted invalid EmbeddingDefaults.kt Deleted invalid EmbeddingDefaults.kt * fix: actually add system properties * Modified the field name for floats and base64 Modified the field name for floats and base64 * Deleted DebutTest Deleted DebutTest * Modified validate method and refactored apply scope function * validate() methods validate anything the "shape" of the data being correct * The code has been refactored to use Kotlin's apply scope function for improved conciseness and consistency. * Modified default encoding Modified default encoding * Modified the implementation of default encoding Modified the implementation of default encoding * Modified format Modified format * Delete plan2.md Delete plan2.md * fix: set the default correctly * fix: rename some things * chore: EmbeddingValue refactor * refactor: embedding data model * chore: delete test with no asserts * chore: test changes * fix: little-endian --------- Co-authored-by: Yoshio Terada <[email protected]> Co-authored-by: Tomer Aberbach <[email protected]> * release: 2.19.0 --------- Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com> Co-authored-by: Yoshio Terada <[email protected]> Co-authored-by: Yoshio Terada <[email protected]> Co-authored-by: Tomer Aberbach <[email protected]>
@TomerAberbach san Thank you so much for your support. I much appreciate. And I glad to hear that it will be merged and release soon. |
Already released in v2.19.0 :) |
This implementation adds support for Base64-encoded embeddings as the default response format, while maintaining complete backward compatibility with existing
List<Float>
usage.1. Default Behavior Change
embedding()
method continues to work unchangedIntroduces the EmbeddingValue class to support both float list and base64-encoded embedding data, enabling efficient handling and backward compatibility. Embedding, EmbeddingCreateParams, and related classes are updated to use EmbeddingValue, with automatic decoding and encoding between formats. Adds EmbeddingDefaults for global default encoding configuration, and comprehensive tests for new behaviors and compatibility.
Improvement from previous PR
This PR is the fix of the issue #211
This issue is related to the PR of #303