diff --git a/DEVELOPER.md b/DEVELOPER.md index c07b2b0..ffee29e 100644 --- a/DEVELOPER.md +++ b/DEVELOPER.md @@ -1,7 +1,7 @@ # DEVELOPER.md This document provides instructions for setting up your development environment -and contributing to the Knowledge Catalog (formerly known as Dataplex) Gemini CLI Extension project. +and contributing to the Knowledge Catalog (formerly known as Dataplex) Agent skills project. ## Prerequisites @@ -16,8 +16,6 @@ Before you begin, ensure you have the following: ### Running from Local Source -The core logic for this extension is handled by a pre-built `toolbox` binary. The development process involves installing the extension locally into the Gemini CLI to test changes. - 1. **Clone the Repository:** ```bash @@ -25,30 +23,16 @@ The core logic for this extension is handled by a pre-built `toolbox` binary. Th cd knowledge-catalog ``` -2. **Download the Toolbox Binary:** The required version of the `toolbox` binary - is specified in `toolbox_version.txt`. Download it for your platform. - - ```bash - # Read the required version - VERSION=$(cat toolbox_version.txt) - - # Example for macOS/amd64 - curl -L -o toolbox https://storage.googleapis.com/mcp-toolbox-for-databases/geminicli/v$VERSION/darwin/amd64/toolbox - chmod +x toolbox - ``` - Adjust the URL for your operating system (`linux/amd64`, `darwin/arm64`, `windows/amd64`). - -3. **Link the Extension Locally:** Use the Gemini CLI to install the +2. **Install the Extension Locally:** Use the Gemini CLI to install the extension from your local directory. ```bash - gemini extensions link . + gemini extensions install . ``` - The CLI will prompt you to confirm the linking. Accept it to proceed. + The CLI will prompt you to confirm the installation. Accept it to proceed. -4. **Testing Changes:** After linking, start the Gemini CLI (`gemini`). - You can now interact with the `knowledge-catalog` tools to manually test your changes - against your connected database. +3. **Testing Changes:** After installation, start the Gemini CLI (`gemini`). + You can now interact with the `knowledge-catalog` skills to manually test your changes. ## Testing @@ -58,9 +42,9 @@ A GitHub Actions workflow (`.github/workflows/presubmit-tests.yml`) is triggered for every pull request. This workflow primarily verifies that the extension can be successfully installed by the Gemini CLI. -Currently, there are no automated unit or integration test suites -within this repository. All functional testing must be performed manually. All tools -are currently tested in the [MCP Toolbox GitHub](https://github.com/googleapis/mcp-toolbox). +All skills are currently tested in the [MCP Toolbox GitHub](https://github.com/googleapis/mcp-toolbox). + +The skills themselves are validated using the `skills-validate.yml` workflow. ### Other GitHub Checks @@ -73,7 +57,6 @@ are currently tested in the [MCP Toolbox GitHub](https://github.com/googleapis/m * **Dependency Updates:** [Renovate](https://github.com/apps/forking-renovate) is configured to automatically create pull requests for dependency updates. - ## Maintainer Information ### Team diff --git a/KNOWLEDGE_CATALOG.md b/KNOWLEDGE_CATALOG.md index 282e7f5..5188b89 100644 --- a/KNOWLEDGE_CATALOG.md +++ b/KNOWLEDGE_CATALOG.md @@ -8,11 +8,11 @@ Your primary objective is to help discover, organize and manage metadata related To install this extension, the Gemini CLI version must be v0.6.0 or above. The version can be found by running: `gemini --version`. -## Knowledge Catalog(formerly known as Dataplex) MCP Server (Data Plane: Connecting and Querying) +## Knowledge Catalog (formerly known as Dataplex) Agent Skills (Data Plane: Connecting and Querying) This section covers connecting to a Knowledge Catalog instance. -1. **Verify Environment Variables**: The extension requires the following environment variables to be set before the Gemini CLI is started: +1. **Verify Environment Variables**: The extension requires the following environment variables to be set before the agent is started: * `DATAPLEX_PROJECT`: The GCP project ID. @@ -24,6 +24,20 @@ This section covers connecting to a Knowledge Catalog instance. --- +# Usage Guidelines + +## Reusing Project Values + +Users may have set project environment variables: + +* `DATAPLEX_PROJECT`: The GCP project ID. + +Instead of prompting the user for these values for specific skill calls, prompt the user to verify the reuse of a specific setting value. +Make sure to not use the environment variable name like `DATAPLEX_PROJECT`, `${DATAPLEX_PROJECT}`, or `$DATAPLEX_PROJECT`. +The value can be verified by the user using the `gemini extensions config knowledge-catalog` command or by checking their local settings. + +--- + # Data Model ## Entries @@ -145,9 +159,9 @@ Entries are organized within Entry Groups, which are logical groupings of Entrie Entries can be linked together using EntryLinks to represent relationships between data assets (e.g. foreign keys). -# Tool instructions +# Skill instructions -## Tool: search_entries +## Skill: search_entries ## General @@ -341,12 +355,12 @@ This abbreviated syntax works for the qualified predicates except for `label` in 1. Explain that no search result was found 2. Suggest to provide a more specific search query. -## Tool: lookup_entry +## Skill: lookup_entry ### Request -1. Always try to limit the size of the response by specifying `aspect_types` parameter. Make sure to include to select view=CUSTOM when using aspect_types parameter. If you do not know the name of the aspect type, use the `search_aspect_types` tool. -2. If you do not know the name of the entry, use `search_entries` tool +1. Always try to limit the size of the response by specifying `aspect_types` parameter. Make sure to include to select view=CUSTOM when using aspect_types parameter. If you do not know the name of the aspect type, use the `search_aspect_types` skill. +2. If you do not know the name of the entry, use `search_entries` skill ### Response diff --git a/README.md b/README.md index 77fcc32..df74fa4 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,10 @@ -# Gemini CLI Extension - Knowledge Catalog (formerly known as Dataplex) +# Agent Skills: Knowledge Catalog (formerly known as Dataplex) > [!NOTE] -> This extension is currently in beta (pre-v1.0), and may see breaking changes until the first stable release (v1.0). +> Currently in beta (pre-v1.0), and may see breaking changes until the first stable release (v1.0). -This Gemini CLI extension provides a set of tools to interact with [Knowledge Catalog](https://cloud.google.com/dataplex/docs) instances. It allows you to manage your data lakes, zones, and assets directly from the [Gemini CLI](https://google-gemini.github.io/gemini-cli/), using natural language prompts. +This repository provides a set of agent skills to interact with [Knowledge Catalog](https://cloud.google.com/dataplex/docs) (formerly known as Dataplex) instances. These skills can be used with various AI agents, including [Gemini CLI](https://google-gemini.github.io/gemini-cli/), Claude Code, and Codex, to discover, manage, monitor, and govern data and AI artifacts across your data platform using natural language prompts. -Learn more about [Gemini CLI Extensions](https://github.com/google-gemini/gemini-cli/blob/main/docs/extensions/index.md). > [!IMPORTANT] > **We Want Your Feedback!** > Please share your thoughts with us by filling out our feedback [form][form]. @@ -13,74 +12,198 @@ Learn more about [Gemini CLI Extensions](https://github.com/google-gemini/gemini [form]: https://docs.google.com/forms/d/e/1FAIpQLSfEGmLR46iipyNTgwTmIDJqzkAwDPXxbocpXpUbHXydiN1RTw/viewform?usp=pp_url&entry.157487=dataplex -## Why Use the Knowledge Catalog Extension? +## Table of Contents -* **Natural Language Management:** Stop wrestling with complex commands. Explore schemas and query data by describing what you want in plain English. -* **Seamless Workflow:** As a Google-developed extension, it integrates seamlessly into the Gemini CLI environment. No need to constantly switch contexts for common database tasks. -* **Code Generation:** Accelerate development by asking Gemini to generate data classes and other code snippets based on your table schemas. +- [Why Use Knowledge Catalog Agent Skills?](#why-use-knowledge-catalog-agent-skills) +- [Prerequisites](#prerequisites) +- [Getting Started](#getting-started) + - [Configuration](#configuration) + - [Installation & Usage](#installation--usage) + - [Gemini CLI](#gemini-cli) + - [Claude Code](#claude-code) + - [Codex](#codex) + - [Antigravity](#antigravity) +- [Usage Examples](#usage-examples) +- [Supported Skills](#supported-skills) +- [Additional Agent Skills](#additional-agent-skills) +- [Troubleshooting](#troubleshooting) + +## Why Use Knowledge Catalog Agent Skills? +* **Natural Language Management:** Stop wrestling with complex commands. Explore schemas and query data by describing what you want in plain English. +* **Seamless Workflow:** Integrates seamlessly into your AI agent's environment. No need to constantly switch contexts for common database tasks. +* **Code Generation:** Accelerate development by asking your agent to generate data classes and other code snippets based on your table schemas. ## Prerequisites Before you begin, ensure you have the following: -* [Gemini CLI](https://github.com/google-gemini/gemini-cli) installed with version **+v0.6.0**. -* Setup Gemini CLI [Authentication](https://github.com/google-gemini/gemini-cli/tree/main?tab=readme-ov-file#-authentication-options). -* A Google Cloud project with the **Dataplex API** enabled. -* Ensure [Application Default Credentials](https://cloud.google.com/docs/authentication/gcloud) are available in your environment. -* IAM Permissions: +- One of these AI agents installed + - [Gemini CLI](https://github.com/google-gemini/gemini-cli) version **v0.6.0** or higher + - [Claude Code](https://claude.com/product/claude-code) version **v2.1.94** or higher + - [Codex](https://developers.openai.com/codex) **v0.117.0** or higher + - [Antigravity](https://antigravity.google) **v1.14.2** or higher +- A Google Cloud project with the **Dataplex API** enabled. +- Ensure [Application Default Credentials](https://cloud.google.com/docs/authentication/gcloud) are available in your environment. +- IAM Permissions: * Dataplex Data Reader (`roles/dataplex.dataReader`): For reading data from the underlying assets (e.g., to run analytics queries). * Service Usage Consumer (`roles/serviceusage.serviceUsageConsumer`) ## Getting Started -### Installation +### Configuration + +Please keep these env vars handy during the installation process: + +* `DATAPLEX_PROJECT`: The GCP project ID. + +### Installation & Usage + +To start interacting with your database, install the skills for your preferred AI agent, then launch the agent and use natural language to ask questions or perform tasks. + +For the latest version, check the [releases page][releases]. + +[releases]: https://github.com/gemini-cli-extensions/knowledge-catalog/releases -To install the extension, use the command: + + +
+Gemini CLI + +**1. Install the extension:** ```bash gemini extensions install https://github.com/gemini-cli-extensions/knowledge-catalog ``` -### Configuration +During the installation, enter your environment vars as described in the [configuration section](#configuration). -You will be prompted to configure the following settings during installation. These settings are saved in an `.env` file within the extension's directory. +**2. (Optional) Manage Configuration:** +To view or update your configuration in Gemini CLI: -* `DATAPLEX_PROJECT`: The GCP project ID. +- Terminal: `gemini extensions config knowledge-catalog [setting name] [--scope ]` +- Gemini CLI: `/extensions list` + +**3. Start the agent:** + +```bash +gemini +``` + +_(Tip: Run `/extensions list` to verify your configuration and active extensions.)_ -To view or update your configuration: +
-**List Settings:** -* Terminal: `gemini extensions list` -* Gemini CLI: `/extensions list` +
+Claude Code -**Update Settings:** -* Terminal: `gemini extensions config knowledge-catalog [setting name] [--scope ]` - * `setting name`: (Optional) The single setting to configure. - * `scope`: (Optional) The scope of the setting in (`user` or `workspace`). Defaults to `user`. -* Currently, you must restart the Gemini CLI for changes to take effect. We recommend using `gemini --resume` to resume your session. +**1. Set env vars:** +In your terminal, set your environment vars as described in the [configuration section](#configuration). -Alternatively, you can manually set these environment variables before starting the Gemini CLI: +**2. Start the agent:** ```bash -export DATAPLEX_PROJECT="" +claude ``` -> [!NOTE] -> * Ensure [Application Default Credentials](https://cloud.google.com/docs/authentication/gcloud) are available in your environment. -> * See [Troubleshooting](#troubleshooting) for debugging your configuration. +**3. Add the marketplace:** -### Start Gemini CLI +```bash +/plugin marketplace add https://github.com/gemini-cli-extensions/knowledge-catalog.git#0.4.0 +``` -To start the Gemini CLI, use the following command: +**4. Install the plugin:** ```bash -gemini +/plugin install knowledge-catalog@knowledge-catalog-marketplace +``` + +_(Tip: Run `/plugin list` inside Claude Code to verify the plugin is active, or `/reload-plugins` if you just installed it.)_ + +
+ +
+Codex + +**1. Clone the Repo:** + +```bash +git clone --branch 0.4.0 git@github.com:gemini-cli-extensions/knowledge-catalog.git +``` + +**2. Install the plugin:** + +```bash +mkdir -p ~/.codex/plugins +cp -R /absolute/path/to/knowledge-catalog ~/.codex/plugins/knowledge-catalog +``` + +**3. Set env vars:** +Enter your environment vars as described in the [configuration section](#configuration). + +**4. Create or update marketplace.json:** +`~/.agents/plugins/marketplace.json` + +```json +{ + "name": "my-data-cloud-google-marketplace", + "interface": { + "displayName": "Google Data Cloud Skills" + }, + "plugins": [ + { + "name": "knowledge-catalog", + "source": { + "source": "local", + "path": "./plugins/knowledge-catalog" + }, + "policy": { + "installation": "AVAILABLE", + "authentication": "ON_INSTALL" + }, + "category": "Data Governance" + } + ] +} +``` + +_(Tip: Run `codex plugin list` or use the `/plugins` interactive menu to verify your installed plugins.)_ + +
+ +
+Antigravity + +**1. Clone the Repo:** + +```bash +git clone --branch 0.4.0 https://github.com/gemini-cli-extensions/knowledge-catalog.git +``` + +**2. Install the skills:** + +Choose a location for the skills: +- **Global (all workspaces):** `~/.gemini/antigravity/skills/` +- **Workspace-specific:** `/.agents/skills/` + +Copy the skill folders from the cloned repository's `skills/` directory to your chosen location: + +```bash +cp -R knowledge-catalog/skills/* ~/.gemini/antigravity/skills/ ``` +**3. Set env vars:** +Set your environment vars as described in the [configuration section](#configuration). + +_(Tip: Antigravity automatically discovers skills in these directories at the start of a session.)_ + +
+ + + ## Usage Examples -Interact with Knowledge Catalog using natural language right from your IDE: +Interact with Knowledge Catalog using natural language: * **Explore Catalog and Metadata:** * "Find all catalog entries related to 'customer orders'." @@ -90,23 +213,23 @@ Interact with Knowledge Catalog using natural language right from your IDE: * **Perform Ad-hoc Analysis:** * "Calculate the total 'customer orders' this month." -## Supported Tools +## Supported Skills + +The following skills are available in this repository: -* `search_entries`: Use this tool to search for entries in Knowledge Catalog based on the provided search query. -* `lookup_entry`: Use this tool to retrieve a specific entry from Knowledge Catalog. -* `search_aspect_types`: Use this tool to find aspect types relevant to the query. +- [Knowledge Catalog Discovery](./skills/knowledge-catalog-discovery/SKILL.md) - Use these skills when you need to discover and explore data assets in the Knowledge Catalog. It allows you to search for entries, lookup specific metadata, and explore aspect types to understand your data platform's assets. -## Additional Extensions +## Additional Agent Skills -Find additional extensions to support your entire software development lifecycle at [github.com/gemini-cli-extensions](https://github.com/gemini-cli-extensions). +Find additional skills to support your entire software development lifecycle at [github.com/gemini-cli-extensions](https://github.com/gemini-cli-extensions). ## Troubleshooting -Use `gemini --debug` to enable debugging. +Use the debug mode of your agent (e.g., `gemini --debug`) to enable debugging. Common issues: * "failed to find default credentials: google: could not find default credentials.": Ensure [Application Default Credentials](https://cloud.google.com/docs/authentication/gcloud) are available in your environment. See [Set up Application Default Credentials](https://cloud.google.com/docs/authentication/external/set-up-adc) for more information. * "✖ Error during discovery for server: MCP error -32000: Connection closed": The database connection has not been established. Ensure your configuration is set via environment variables. -* "✖ MCP ERROR: Error: spawn /Users/USER/.gemini/extensions/knowledge-catalog/toolbox ENOENT": The Toolbox binary did not download correctly. Ensure you are using Gemini CLI v0.6.0+. +* "✖ MCP ERROR: Error: spawn .../toolbox ENOENT": The Toolbox binary did not download correctly. Ensure you are using the latest version of your agent. * "cannot execute binary file": The Toolbox binary did not download correctly. Ensure the correct binary for your OS/Architecture has been downloaded. See [Installing the server](https://mcp-toolbox.dev/documentation/introduction/#install-toolbox) for more information.