Merge pull request #1854 from Thushani-Jayasekera/pe-dev-thushani

Add docs for choreo ai gateway feature

Author: Thushani-Jayasekera

en/developer-docs/docs/develop-components/sharing-and-reusing/create-a-connection-to-a-genai-service.md

@@ -0,0 +1,50 @@
+# Consume a GenAI Service
+
+## Overview
+
+Choreo is a cloud-native platform that enables developers to create, deploy, and consume services efficiently. The **Choreo Internal Marketplace** streamlines API discovery and integration, making it easier to incorporate services into your applications.
+
+This guide is for application developers who want to consume GenAI Services published in the Internal Marketplace.
+
+## Prerequisites
+
+Before you begin, ensure that:
+
+- You have access to a published GenAI Service in the Internal Marketplace.
+
+- If you don’t yet have one, refer to the [PE documentation](https://docs.choreo.dev/en/pe-docs/docs/db-and-services/ai-gateway/integrate-and-manage-gen-ai-services) to create and deploy a sample GenAI Service.
+
+## Discover GenAI Services
+
+1. Navigate to the [Choreo Internal Marketplace](https://console.choreo.dev).
+
+2. Sign in with your credentials.
+3. Browse or search for the desired GenAI Service.
+
+4. Select the service to view its details.
+
+## Create a Connection to a GenAI Service
+
+To use a GenAI Service in your Choreo component, you first need to create a connection.
+
+1. In the left navigation menu, click Connections.
+
+2. This page lists all existing connections.
+
+3. Click **+Create**.
+
+4. The Marketplace view opens, where you can browse and search for services.
+
+4. Click the **Services** tab.
+
+5. Use search or filters to find the required service efficiently.
+
+6. Select the GenAI Service you want to connect to.
+
+7. Provide a name and description for the connection.
+
+8. Choose an Access Mode and Authentication Scheme.
+
+9. Click **Create**.
+
+Once created, the connection details are displayed for each environment. An inline usage guide is also provided, showing how to use the connection in your component.

en/developer-docs/docs/integrate-and-manage-external-services/integrate-and-manage-gen-ai-services.md

@@ -139,6 +139,7 @@ nav:
       - Integrate Unit Tests into the Build Pipeline: develop-components/integrate-unit-tests-into-the-build-pipeline.md
       - Share and Reuse Components:
           - Create a Connection: develop-components/sharing-and-reusing/create-a-connection.md
+          - Create a connection to a GenAI Service: develop-components/sharing-and-reusing/create-a-connection-to-a-genai-service.md
           - Use a Connection in Your Service: develop-components/sharing-and-reusing/use-a-connection-in-your-service.md
           - Use a Connection in Your Web Application: develop-components/sharing-and-reusing/use-a-connection-in-your-web-application.md
           - Use a Database Connection in Your Component: develop-components/sharing-and-reusing/use-a-database-connection-in-your-component.md
@@ -151,7 +152,6 @@ nav:
       - Connect to Remote Dependencies While Developing: develop-components/connect-to-remote-dependencies-while-developing.md
   - Integrate and Manage External Services:
       - Integrate and Manage Third-Party Services: integrate-and-manage-external-services/integrate-and-manage-third-party-services.md
-      - Integrate and Manage Generative AI Services: integrate-and-manage-external-services/integrate-and-manage-gen-ai-services.md
   - Choreo-Managed Databases and Caches:
       - Overview: manage-databases-and-caches/choreo-managed-databases-and-caches.md
       - PostgreSQL: manage-databases-and-caches/choreo-managed-postgresql-databases.md

en/developer-docs/mkdocs.yml

@@ -0,0 +1,45 @@
+# Choreo AI Gateway
+
+As organizations accelerate AI adoption, managing AI APIs effectively has become essential. The Choreo AI Gateway simplifies this process by providing a unified way to create, manage, and expose AI APIs with enterprise-grade security, observability, and scalability.
+
+![Choreo AI Gateway](../../assets/img/ai-gateway/ai-gateway.png)
+
+## Supported Providers
+
+The AI Gateway integrates seamlessly with leading AI platforms, including:
+
+- OpenAI
+
+- Azure OpenAI
+
+- Anthropic Claude
+
+- Mistral
+
+- AWS Bedrock
+
+## Key Features
+
+The AI Gateway offers a robust set of capabilities to ensure secure and efficient AI integration:
+
+- AI Vendor Key Configuration: 
+
+  Securely authenticate with AI providers by configuring API keys obtained from each vendor.
+
+- Token-Based Rate Limiting:
+
+  Protect backend services and manage costs by applying rate limits based on token usage rather than just request counts.
+  
+- AI API Observability (Coming Soon):
+
+  Gain insights into AI API performance and usage with built-in analytics and monitoring.
+
+With these features, organizations can integrate, monitor, and scale AI services efficiently, unlocking the full potential of AI-driven applications.
+
+### Token-Based Rate Limiting
+
+AI services typically charge based on tokens rather than requests, making cost control critical. Choreo AI Gateway enables token-based rate limiting at the API level to:
+
+- Define quotas based on LLM token consumption.
+- Prevent unexpected costs by controlling overuse.
+- Enforce fair usage policies across AI-powered APIs.

en/pe-docs/docs/assets/img/ai-gateway/add-configs.png

@@ -0,0 +1,163 @@
+# Integrate and Manage Generative AI Services
+
+## Overview
+
+Generative AI (GenAI) services leverage advanced machine learning models to produce original content—such as text, images, music, or code—by learning patterns from existing data. Powered by deep neural networks, these services can generate human-like outputs across multiple formats, making them well-suited for tasks like content creation, image generation, and conversational automation.
+
+Choreo provides native support for integrating GenAI services into applications. Using the Choreo AI Gateway, Platform Engineers and Admins can:
+
+- Register GenAI services in the Choreo AI Gateway.
+
+- Manage access and authentication for GenAI services.
+
+- Enforce governance, security, and cost controls for GenAI services at scale.
+
+## Register a GenAI Service
+
+When you register a GenAI service in Choreo, all requests are routed through the AI Gateway, which enforces:
+
+- Security – Authentication, authorization, and encrypted transport.
+
+- Rate Limiting – Token-based throttling.
+
+- Observability (Coming Soon) – Metrics, logging, and monitoring.
+
+This ensures consistent, secure, and cost-controlled access to the underlying provider.
+
+## Registration Scopes
+
+You can register GenAI services at two levels:
+
+- Organization-level registration – Service is available across all projects in the organization.
+
+- Project-level registration – Service is restricted to the project where it is registered.
+
+Once registered, the service is published to the Internal Marketplace, where developers can discover and consume it via Connections.
+
+## Prerequisites
+
+Before registering a GenAI service, obtain the following from your provider:
+
+- API key
+
+- Service URL
+
+- Any additional credentials (e.g., client secrets, subscription keys)
+
+!!! note
+
+    - Choose the correct registration scope:
+
+        - Use organization-level if the service will be shared across multiple projects.
+        - Use project-level if usage is limited to a single project.
+
+## Registration Steps
+
+### Step 1: Select a Service Provider
+
+1. Sign in to the Choreo Console.
+
+!!! info 
+    **AI Gateway is only available in the US region.**
+
+    - 🇺🇸 **US Region**: [console.choreo.dev](https://console.choreo.dev)
+
+2. Select your Organization or Project, depending on the required scope.
+
+3. In the left navigation menu, go to DB & Services → GenAI Services.
+
+4. Choose a provider from the list:
+
+- OpenAI
+
+- Azure OpenAI
+
+- Anthropic Claude
+
+- Mistral
+
+- AWS Bedrock
+
+5. Click Next.
+
+### Step 2: Provide Service Details
+
+Under **Register a GenAI Service**, enter:
+
+- Name – The service name.
+
+- Context Path – A unique identifier for the API within the AI Gateway.
+
+- Service URL – The provider’s endpoint.
+
+- Summary – A short description of the service.
+
+Click **Next**.
+
+### Step 3: Add Environment Configurations
+
+![Add Environment Configurations](../../assets/img/ai-gateway/add-configs.png)
+
+### Configure Backend Security
+
+In **Backend Security**, set the Authorization header.
+
+- Provide the API key or credentials (e.g., Authorization: Bearer <token>).
+
+- Key formats may vary by provider.
+
+### Configure Token-Based Rate Limiting
+
+LLMs and GenAI services are billed based on token usage, not just request count. Token-based rate limiting helps:
+
+- Control costs.
+
+- Prevent excessive usage.
+
+- Enforce fair quotas across consumers.
+
+Set the following parameters:
+
+- Time Unit – Enforcement window (per minute, hour, or day).
+
+- Max Prompt Token Count – Maximum input tokens per request.
+
+- Max Completion Token Count – Maximum output tokens per request.
+
+- Max Total Token Count – Combined input + output tokens per request.
+
+Click **Register**.
+
+The service is now available in the Internal Marketplace, where developers can discover and connect to it.
+
+See the [Developer Docs](https://wso2.com/choreo/docs/develop-components/sharing-and-reusing/create-a-connection-to-a-genai-service/) for instructions on how to consume a GenAI service published in the Internal Marketplace.
+
+## Manage Registered Services
+
+### View or Update Service Details
+
+1. Sign in to the [Choreo Console](https://console.choreo.dev).
+
+2. Navigate to DB & Services → GenAI Services.
+
+3. Select a service to view or edit details:
+
+- General Details – Name, description, and labels.
+
+- Environment Configurations – Backend security and rate limiting.
+
+- Deployment Status – View deployment state for each environment.
+
+![Deployment Status](../../assets/img/ai-gateway/view-configs.png)
+
+### Remove a GenAI Service
+
+1. Sign in to the [Choreo Console](https://console.choreo.dev).
+
+2. Navigate to DB & Services → GenAI Services.
+
+3. Select the service.
+
+4. Click Remove from Marketplace.
+
+The service is removed from the Internal Marketplace and can’t be used for new Connections. Existing Connections remain functional until manually removed.

en/pe-docs/docs/assets/img/ai-gateway/ai-gateway.png

@@ -135,7 +135,9 @@ nav:
           - Create Choreo-Managed Kafka Services: db-and-services/message-brokers/create-choreo-managed-kafka-services.md
           - Configure a Kafka Service: db-and-services/message-brokers/configure-a-kafka-service.md
           - Monitor a Kafka Service: db-and-services/message-brokers/monitor-a-kafka-service.md
-      - Gen AI Services: db-and-services/external-services/integrate-and-manage-gen-ai-services.md
+      - Gen AI Services:
+          - Choreo AI Gateway: db-and-services/ai-gateway/choreo-ai-gateway.md
+          - Integrate and Manage Generative AI Services: db-and-services/ai-gateway/integrate-and-manage-gen-ai-services.md
   - K8s Operations:
       - Runtime:
           - View Kubernetes Runtime Details: k8s-operations/runtime/view-runtime-details.md