feature: Implement Glossaries for Firestore Translate Text Extension

README.md

@@ -14,6 +14,4 @@ You can also browse official Firebase extensions on the [Extensions Marketplace]
 
 ## Documentation
 
-Documentation for the [Extensions by Firebase](https://firebase.google.com/docs/extensions) section are now stored in this repository.
-
-They can be found under [Docs](https://github.com/firebase/extensions/docs)
+Documentation for the [Extensions by Firebase](https://firebase.google.com/docs/extensions) section are now stored in this repository.

firestore-translate-text/POSTINSTALL.md

@@ -2,20 +2,19 @@
 
 You can test out this extension right away!
 
-1.  Go to your [Cloud Firestore dashboard](https://console.firebase.google.com/project/${param:PROJECT_ID}/firestore/data) in the Firebase console.
+1. Go to your [Cloud Firestore dashboard](https://console.firebase.google.com/project/${param:PROJECT_ID}/firestore/data) in the Firebase console.
 
-1.  If it doesn't exist already, create a collection called `${param:COLLECTION_PATH}`.
+2. If it doesn't exist already, create a collection called `${param:COLLECTION_PATH}`.
 
-1.  Create a document with a field named `${param:INPUT_FIELD_NAME}`, then make its value a word or phrase that you want to translate.
+3. Create a document with a field named `${param:INPUT_FIELD_NAME}`, then make its value a word or phrase that you want to translate.
 
-1.  In a few seconds, you'll see a new field called `${param:OUTPUT_FIELD_NAME}` pop up in the same document you just created. It will contain the translations for each language you specified during installation. 
+4. In a few seconds, you'll see a new field called `${param:OUTPUT_FIELD_NAME}` pop up in the same document you just created. It will contain the translations for each language you specified during installation.
 
 ### Using the extension
 
 This extension translates the input string(s) into your specified target language(s); the source language of the string is automatically detected. If the `${param:INPUT_FIELD_NAME}` field of the document is updated,
  then the translations will be automatically updated as well.
 
-
 #### Input field as a string
 
 Write the string "My name is Bob" to the field `${param:INPUT_FIELD_NAME}` in `${param:COLLECTION_PATH}` will result in the following translated output in `${param:OUTPUT_FIELD_NAME}`:

firestore-translate-text/PREINSTALL.md

@@ -25,10 +25,12 @@ admin.firestore().collection('translations').add({
   second: "Hello, friend"
 })
 ```
+
 #### Multiple languages
 
 To translate text into multiple languages, set the `languages` parameter to a comma-separated list
 of languages, such as `en,fr,de`. See the [supported languages list](https://cloud.google.com/translate/docs/languages).
+
 #### Additional setup
 
 Before installing this extension, make sure that you've [set up a Cloud Firestore database](https://firebase.google.com/docs/firestore/quickstart) in your Firebase project.
@@ -43,10 +45,50 @@ A large language model like Gemini 1.5 Pro may have more contextual understandin
 
 It is important to note that Gemini should only be used with sanitized input, as prompt injection is a possibility.
 
-##### Notes:
+##### Notes
+
 - Using the Gemini API may have a different pricing model than the Cloud Translation API.
 
+### How to Use Glossaries with the Cloud Translation API
+
+#### Enabling Glossaries
+
+1. **Enable Translation Hub**: Before using glossaries, make sure that the [Translation Hub](https://console.cloud.google.com/translation/hub) is enabled for your project.
+2. **Source Language Code**: When using glossaries, you must specify the source language. If no glossary is used, the source language can be automatically detected.
+3. **Case Sensitivity**: Glossary names are case-sensitive and must be entered precisely as created.
+
+#### Steps to Create and Use a Glossary
+
+1. **Create a Glossary**:
+   - Use the [Google Cloud Translation API glossary creation guide](https://cloud.google.com/translate/docs/advanced/glossary) to create a glossary.
+   - Store the glossary in the correct Google Cloud Storage bucket and ensure that the bucket's location matches your project's region.
+   - Glossaries must be unique to the project and region.
+
+2. **Specify the Glossary in the Extension**:
+   - Provide the `GLOSSARY_ID` parameter during installation. This should match the ID of the glossary you created.
+   - If using a glossary, also provide the `SOURCE_LANGUAGE_CODE` parameter to define the source language for your translations.
+
+#### Example Usage
+
+- Glossary ID: `city_names_glossary`
+- Source Language Code: `en`
+
+For example, if translating the phrase *"Paris is beautiful"* and your glossary specifies `Paris` to remain untranslated, the extension will ensure it remains in the source form.
+
+#### Common Errors and Troubleshooting
+
+- **Invalid Glossary ID**: Ensure the glossary ID is correct and case-sensitive.
+- **Missing Source Language Code**: If using a glossary, a source language code is mandatory.
+- **Glossary Not Found**: Confirm that the glossary exists in the correct project and region.
+
+#### Links and Resources
+
+- [Glossary Documentation](https://cloud.google.com/translate/docs/advanced/glossary)
+- [Supported Languages List](https://cloud.google.com/translate/docs/languages)
+- [Service Account Key Documentation](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
+
 #### Billing
+
 To install an extension, your project must be on the [Blaze (pay as you go) plan](https://firebase.google.com/pricing)
 
 - You will be charged a small amount (typically around $0.01/month) for the Firebase resources required by this extension (even if it is not used).

firestore-translate-text/README.md

@@ -4,8 +4,6 @@
 
 **Description**: Translates strings written to a Cloud Firestore collection into multiple languages (uses Cloud Translation API).
 
-
-
 **Details**: Use this extension to translate strings (for example, text messages) written to a Cloud Firestore collection.
 
 This extension listens to your specified Cloud Firestore collection. If you add a string to a specified field in any document within that collection, this extension:
@@ -33,10 +31,12 @@ admin.firestore().collection('translations').add({
   second: "Hello, friend"
 })
 ```
+
 #### Multiple languages
 
 To translate text into multiple languages, set the `languages` parameter to a comma-separated list
 of languages, such as `en,fr,de`. See the [supported languages list](https://cloud.google.com/translate/docs/languages).
+
 #### Additional setup
 
 Before installing this extension, make sure that you've [set up a Cloud Firestore database](https://firebase.google.com/docs/firestore/quickstart) in your Firebase project.
@@ -51,67 +51,98 @@ A large language model like Gemini 1.5 Pro may have more contextual understandin
 
 It is important to note that Gemini should only be used with sanitized input, as prompt injection is a possibility.
 
-##### Notes:
-- Using the Gemini API may have a different pricing model than the Cloud Translation API.
+##### Notes
 
-#### Billing
-To install an extension, your project must be on the [Blaze (pay as you go) plan](https://firebase.google.com/pricing)
+- Using the Gemini API may have a different pricing model than the Cloud Translation API.
 
-- You will be charged a small amount (typically around $0.01/month) for the Firebase resources required by this extension (even if it is not used).
-- This extension uses other Firebase and Google Cloud Platform services, which have associated charges if you exceed the service’s no-cost tier:
-  - Cloud Translation API
-  - Cloud Firestore
-  - Cloud Functions (Node.js 10+ runtime. [See FAQs](https://firebase.google.com/support/faq#extensions-pricing))
+### How to Use Glossaries with the Cloud Translation API
 
+#### Enabling Glossaries
 
+1. **Glossary Requirement**: Glossaries enable domain-specific translations and are case-sensitive. Ensure that the glossary's name matches exactly, as mismatches will result in errors.
+2. **Enable Translation Hub**: Before using glossaries, make sure that the [Translation Hub](https://console.cloud.google.com/translation/hub) is enabled for your project.
+3. **Source Language Code**: When using glossaries, you must specify the source language. If no glossary is used, the source language can be automatically detected.
+4. **Case Sensitivity**: Glossary names are case-sensitive and must be entered precisely as created.
 
+#### Steps to Create and Use a Glossary
 
-**Configuration Parameters:**
+1. **Create a Glossary**:
+   - Use the [Google Cloud Translation API glossary creation guide](https://cloud.google.com/translate/docs/advanced/glossary) to create a glossary.
+   - Store the glossary in the correct Google Cloud Storage bucket and ensure that the bucket's location matches your project's region.
+   - Glossaries must be unique to the project and region.
 
-* Target languages for translations, as a comma-separated list: Into which target languages do you want to translate new strings? The languages are identified using ISO-639-1 codes in a comma-separated list, for example: en,es,de,fr. For these codes, visit the [supported languages list](https://cloud.google.com/translate/docs/languages).
+2. **Specify the Glossary in the Extension**:
+   - Provide the `GLOSSARY_ID` parameter during installation. This should match the ID of the glossary you created.
+   - If using a glossary, also provide the `SOURCE_LANGUAGE_CODE` parameter to define the source language for your translations.
 
+3. **Set Up Service Account**:
+   - The extension uses a service account for authorization. If needed, provide the `GOOGLE_APPLICATION_CREDENTIALS` secret containing the service account key JSON file.
 
-* Collection path: What is the path to the collection that contains the strings that you want to translate?
+#### Example Usage
 
+- Glossary ID: `city_names_glossary`
+- Source Language Code: `en`
 
-* Input field name: What is the name of the field that contains the string that you want to translate?
+For example, if translating the phrase *"Paris is beautiful"* and your glossary specifies `Paris` to remain untranslated, the extension will ensure it remains in the source form.
 
+#### Common Errors and Troubleshooting
 
-* Translations output field name: What is the name of the field where you want to store your translations?
+- **Invalid Glossary ID**: Ensure the glossary ID is correct and case-sensitive.
+- **Missing Source Language Code**: If using a glossary, a source language code is mandatory.
+- **Glossary Not Found**: Confirm that the glossary exists in the correct project and region.
 
+#### Links and Resources
 
-* Languages field name: What is the name of the field that contains the languages that you want to translate into? This field is optional. If you don't specify it, the extension will use the languages specified in the LANGUAGES parameter.
+- [Glossary Documentation](https://cloud.google.com/translate/docs/advanced/glossary)
+- [Supported Languages List](https://cloud.google.com/translate/docs/languages)
+- [Service Account Key Documentation](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
 
+#### Billing
 
-* Translation Model: This extension provides the option to use Gemini 1.5 Pro for translations, which may provide more accurate and context-aware translations. The extension accesses the Gemini API using a Google AI API key that you can provide as a secret during installation.
+To install an extension, your project must be on the [Blaze (pay as you go) plan](https://firebase.google.com/pricing)
 
+- You will be charged a small amount (typically around $0.01/month) for the Firebase resources required by this extension (even if it is not used).
+- This extension uses other Firebase and Google Cloud Platform services, which have associated charges if you exceed the service’s no-cost tier:
+  - Cloud Translation API
+  - Cloud Firestore
+  - Cloud Functions (Node.js 10+ runtime. [See FAQs](https://firebase.google.com/support/faq#extensions-pricing))
 
-* Google AI API key: If you selected AI Translations Using Gemini to perform translations, please provide a Google AI API key, which you can create here: https://ai.google.dev/gemini-api/docs/api-key
+**Configuration Parameters:**
 
+- Target languages for translations, as a comma-separated list: Into which target languages do you want to translate new strings? The languages are identified using ISO-639-1 codes in a comma-separated list, for example: en,es,de,fr. For these codes, visit the [supported languages list](https://cloud.google.com/translate/docs/languages).
 
-* Translate existing documents?: Should existing documents in the Firestore collection be translated as well?  If you've added new languages since a document was translated, this will fill those in as well.
+- Collection path: What is the path to the collection that contains the strings that you want to translate?
 
+- Input field name: What is the name of the field that contains the string that you want to translate?
 
+- Translations output field name: What is the name of the field where you want to store your translations?
 
+- Languages field name: What is the name of the field that contains the languages that you want to translate into? This field is optional. If you don't specify it, the extension will use the languages specified in the LANGUAGES parameter.
 
-**Cloud Functions:**
+- Translation Model: This extension provides the option to use Gemini 1.5 Pro for translations, which may provide more accurate and context-aware translations. The extension accesses the Gemini API using a Google AI API key that you can provide as a secret during installation.
 
-* **fstranslate:** Listens for writes of new strings to your specified Cloud Firestore collection, translates the strings, then writes the translated strings back to the same document.
+- Google AI API key: If you selected AI Translations Using Gemini to perform translations, please provide a Google AI API key, which you can create here: <https://ai.google.dev/gemini-api/docs/api-key>
 
-* **fstranslatebackfill:** Searches your specified Cloud Firestore collection for existing documents, translates the strings into any missing languages, then writes the translated strings back to the same document.
+- Glossary ID (Cloud Translation Only): (Optional) Specify the ID of the glossary you want to use for domain-specific translations. This parameter is applicable only when using Cloud Translation. **Note**: The glossary ID is case-sensitive. Ensure that the ID matches exactly as defined. Additionally, the Translation Hub must be enabled in your Google Cloud project to use glossaries. For more details on creating a glossary, refer to the [glossary documentation](https://cloud.google.com/translate/docs/advanced/glossary).
 
+- Service Account Key JSON Path (Cloud Translation Only): (Optional) Provide the path to the service account key JSON file used for authentication with the Google Translation API. This parameter is applicable only when using Cloud Translation. If not specified, the extension will use the default credentials associated with the project. For more information on creating and using service accounts, refer to the [service account documentation](https://cloud.google.com/iam/docs/creating-managing-service-account-keys).
 
+- Source Language Code (Cloud Translation Only, required if using a glossary): The language code of the source text (e.g., "en" for English). This field is required only when using glossaries with Cloud Translation. Leave this blank if no glossary is used to allow auto-detection of the source language. **Note**: The Translation Hub must be enabled to use glossaries with a source language. Refer to the [supported languages list](https://cloud.google.com/translate/docs/languages).
 
-**APIs Used**:
+- Translate existing documents?: Should existing documents in the Firestore collection be translated as well?  If you've added new languages since a document was translated, this will fill those in as well.
 
-* translate.googleapis.com (Reason: To use Google Translate to translate strings into your specified target languages.)
+**Cloud Functions:**
 
+- **fstranslate:** Listens for writes of new strings to your specified Cloud Firestore collection, translates the strings, then writes the translated strings back to the same document.
 
+- **fstranslatebackfill:** Searches your specified Cloud Firestore collection for existing documents, translates the strings into any missing languages, then writes the translated strings back to the same document.
 
-**Access Required**:
+**APIs Used**:
 
+- translate.googleapis.com (Reason: To use Google Translate to translate strings into your specified target languages.)
 
+**Access Required**:
 
 This extension will operate with the following project IAM roles:
 
-* datastore.user (Reason: Allows the extension to write translated strings to Cloud Firestore.)
+- datastore.user (Reason: Allows the extension to write translated strings to Cloud Firestore.)

firestore-translate-text/extension.yaml

@@ -51,6 +51,8 @@ apis:
 roles:
   - role: datastore.user
     reason: Allows the extension to write translated strings to Cloud Firestore.
+  - role: cloudtranslate.editor
+    reason: Allows the extensions to use the latest Cloud Tranlsate Api.
 
 resources:
   - name: fstranslate
@@ -154,6 +156,33 @@ params:
     type: secret
     required: false
 
+  - param: GLOSSARY_ID
+    label: Glossary ID (Cloud Translation Only)
+    description: >
+      (Optional) Specify the ID of the glossary you want to use for
+      domain-specific translations. This parameter is applicable only when using
+      Cloud Translation. **Note**: The glossary ID is case-sensitive. Ensure
+      that the ID matches exactly as defined. Additionally, the Translation Hub
+      must be enabled in your Google Cloud project to use glossaries. For more
+      details on creating a glossary, refer to the [glossary
+      documentation](https://cloud.google.com/translate/docs/advanced/glossary).
+    default: ""
+    required: false
+
+  - param: SOURCE_LANGUAGE_CODE
+    label:
+      Source Language Code (Cloud Translation Only, Required if using a
+      glossary)
+    description: >
+      The language code of the source text (e.g., "en" for English). This field
+      is required only when using glossaries with Cloud Translation. Leave this
+      blank if no glossary is used to allow auto-detection of the source
+      language. **Note**: The Translation Hub must be enabled to use glossaries
+      with a source language. Refer to the [supported languages
+      list](https://cloud.google.com/translate/docs/languages).
+    default: ""
+    required: false
+
   - param: DO_BACKFILL
     label: Translate existing documents?
     description: >

firestore-translate-text/functions/__tests__/config.test.ts

@@ -111,3 +111,17 @@ describe("extension config", () => {
     });
   });
 });
+
+it("should correctly read GLOSSARY_ID from config", () => {
+  process.env.GLOSSARY_ID = "test_glossary";
+  const config = require("../config").default;
+
+  expect(config.glossaryId).toBe("test_glossary");
+});
+
+it("should fallback to default source language if not provided", () => {
+  delete process.env.SOURCE_LANGUAGE_CODE;
+  const config = require("../config").default;
+
+  expect(config.sourceLanguageCode).toBeUndefined(); // or check for default behavior
+});

firestore-translate-text/functions/__tests__/jest.setup.ts

@@ -6,7 +6,6 @@ import {
 } from "./mocks/firestore";
 import {
   testTranslations,
-  mockTranslate,
   mockTranslateClassMethod,
   mockTranslateClass,
   mockTranslateModuleFactory,
@@ -20,8 +19,6 @@ global.testTranslations = testTranslations;
 
 global.mockDocumentSnapshotFactory = mockDocumentSnapshotFactory;
 
-global.mockTranslate = mockTranslate;
-
 global.mockTranslateClassMethod = mockTranslateClassMethod;
 
 global.mockTranslateClass = mockTranslateClass;

firestore-translate-text/functions/__tests__/mocks/firestore.ts

@@ -1,4 +1,25 @@
 import * as functionsTestInit from "firebase-functions-test";
+import * as admin from "firebase-admin";
+import {
+  DocumentSnapshot,
+  BulkWriter,
+  WriteResult,
+} from "firebase-admin/firestore";
+import { jest } from "@jest/globals";
+
+// Mock DocumentSnapshot
+export const mockDocumentSnapshot = (data: object): DocumentSnapshot => {
+  return {
+    data: () => data,
+    exists: true,
+    id: "mockId",
+    ref: {
+      path: "mockPath",
+      update: jest.fn(),
+    } as any,
+    get: jest.fn((field: string) => (data as any)[field]),
+  } as unknown as DocumentSnapshot;
+};
 
 export const snapshot = (
   input = { input: "hello" },