diff --git a/README.md b/README.md index 280c994..df06102 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,22 @@ This server connects agents to your Elasticsearch data using the Model Context P * `esql`: Perform an ES|QL query * `get_shards`: Get shard information for all or specific indices +## Dynamic URL and Authentication + +When using the HTTP or SSE transport protocols, you can dynamically specify the Elasticsearch cluster URL and authentication credentials on a per-request basis. This is useful for: + +* Connecting to multiple Elasticsearch clusters without restarting the MCP server +* Using different credentials for different operations +* Multi-tenant scenarios where each request targets a different cluster + +The dynamic configuration is provided via HTTP headers: +* **`X-Elasticsearch-URL`**: Override the configured Elasticsearch URL (e.g., `https://my-cluster.es.io:9200`) +* **`Authorization`**: Override the authentication credentials + * API Key format: `ApiKey ` + * Basic auth format: `Basic ` (encode with: `echo -n "username:password" | base64`) + +When these headers are not provided, the server falls back to the configured defaults from the environment variables or config file. + ## Prerequisites * An Elasticsearch instance @@ -60,13 +76,19 @@ Options: The MCP server needs environment variables to be set: -* `ES_URL`: the URL of your Elasticsearch cluster +* `ES_URL`: the URL of your Elasticsearch cluster (defaults to `http://localhost:9200` if not provided) * For authentication use either an API key or basic authentication: * API key: `ES_API_KEY` * Basic auth: `ES_USERNAME` and `ES_PASSWORD` * Optionally, `ES_SSL_SKIP_VERIFY` set to `true` skips SSL/TLS certificate verification when connecting to Elasticsearch. The ability to provide a custom certificate will be added in a later version. +**Note:** When using the HTTP or SSE protocols, the Elasticsearch URL and authentication can be dynamically provided via HTTP headers: +* `X-Elasticsearch-URL`: Override the configured Elasticsearch cluster URL +* `Authorization`: Provide authentication credentials + * API Key: `ApiKey ` + * Basic auth: `Basic ` + The MCP server is started in stdio mode with this command: ```bash @@ -101,25 +123,85 @@ Note: streamable-HTTP is recommended, as [SSE is deprecated](https://modelcontex The MCP server needs environment variables to be set: -* `ES_URL`, the URL of your Elasticsearch cluster +* `ES_URL`, the URL of your Elasticsearch cluster (defaults to `http://localhost:9200` if not provided) * For authentication use either an API key or basic authentication: * API key: `ES_API_KEY` * Basic auth: `ES_USERNAME` and `ES_PASSWORD` * Optionally, `ES_SSL_SKIP_VERIFY` set to `true` skips SSL/TLS certificate verification when connecting to Elasticsearch. The ability to provide a custom certificate will be added in a later version. +#### Dynamic Configuration via HTTP Headers + +When using HTTP or SSE protocols, you can override the Elasticsearch URL and authentication on a per-request basis using HTTP headers: + +* **`X-Elasticsearch-URL`**: Specify a different Elasticsearch cluster URL for the request +* **`Authorization`**: Provide authentication credentials (supports `ApiKey ` and `Basic ` formats) + +This allows you to connect to multiple Elasticsearch clusters or use different credentials without restarting the MCP server. + The MCP server is started in http mode with this command: ```bash docker run --rm -e ES_URL -e ES_API_KEY -p 8080:8080 docker.elastic.co/mcp/elasticsearch http ``` -If for some reason your execution environment doesn't allow passing parameters to the container, they can be passed -using the `CLI_ARGS` environment variable: `docker run --rm -e ES_URL -e ES_API_KEY -e CLI_ARGS=http -p 8080:8080...` +To enable both streamable-HTTP and SSE transports, use the `--sse` flag: + +```bash +docker run --rm -e ES_URL -e ES_API_KEY -p 8080:8080 docker.elastic.co/mcp/elasticsearch http --sse +``` + +#### Environment Variable Configuration + +For containerized environments (Docker, Kubernetes, etc.) where passing command-line arguments may be difficult, you can use environment variables: + +**Option 1: Using specific environment variables** +```bash +docker run --rm \ + -e ES_URL=https://my-cluster.es.io:9200 \ + -e ES_API_KEY=your-api-key \ + -e CONTAINER_MODE=true \ + -e ENABLE_SSE=true \ + -e HTTP_ADDRESS=0.0.0.0:8080 \ + -p 8080:8080 \ + docker.elastic.co/mcp/elasticsearch http +``` + +**Option 2: Using CLI_ARGS for complex configurations** +```bash +docker run --rm \ + -e ES_URL=https://my-cluster.es.io:9200 \ + -e ES_API_KEY=your-api-key \ + -e CLI_ARGS="--container-mode http --sse" \ + -p 8080:8080 \ + docker.elastic.co/mcp/elasticsearch +``` + +**Available environment variables:** +- `CONTAINER_MODE` - Set to `true` to enable container mode (binds to 0.0.0.0, rewrites localhost) +- `ENABLE_SSE` - Set to `true` to enable SSE transport on `/mcp/sse` +- `HTTP_ADDRESS` - Override the listen address (e.g., `0.0.0.0:8080`) +- `CLI_ARGS` - Pass complete command-line arguments as a string + +**Available Endpoints:** +- Streamable-HTTP: `http://:8080/mcp` +- SSE (when `--sse` is used): `http://:8080/mcp/sse` +- Health check: `http://:8080/ping` +- Readiness probe: `http://:8080/_health/ready` +- Liveness probe: `http://:8080/_health/live` + +#### Direct Connection Support + +Many MCP clients can connect directly to the streamable-HTTP and SSE endpoints: +- **Cursor**: Supports direct connections to both streamable-HTTP and SSE endpoints +- **Google Gemini**: Supports direct connections to both streamable-HTTP and SSE endpoints +- **Claude Desktop (free edition)**: Only supports stdio protocol, requires a proxy (see configuration below) + +If your MCP client supports direct HTTP/SSE connections, you can configure it to use the endpoints above directly without needing `mcp-proxy`. -The streamable-HTTP endpoint is at `http::8080/mcp`. There's also a health check at `http::8080/ping` +#### Configuration for Claude Desktop -Configuration for Claude Desktop (free edition that only supports the stdio protocol). +Claude Desktop (free edition) only supports the stdio protocol and requires a proxy. 1. Install `mcp-proxy` (or an equivalent), that will bridge stdio to streamable-http. The executable will be installed in `~/.local/bin`: @@ -144,3 +226,233 @@ Configuration for Claude Desktop (free edition that only supports the stdio prot } } ``` + +3. To dynamically specify the Elasticsearch URL per request, add the `X-Elasticsearch-URL` header: + + **With API Key:** + ```json + { + "mcpServers": { + "elasticsearch-mcp-server": { + "command": "//.local/bin/mcp-proxy", + "args": [ + "--transport=streamablehttp", + "--header", "Authorization", "ApiKey ", + "--header", "X-Elasticsearch-URL", "https://my-cluster.es.io:9200", + "http://:/mcp" + ] + } + } + } + ``` + + **With Username/Password (Basic Auth):** + ```json + { + "mcpServers": { + "elasticsearch-mcp-server": { + "command": "//.local/bin/mcp-proxy", + "args": [ + "--transport=streamablehttp", + "--header", "Authorization", "Basic ", + "--header", "X-Elasticsearch-URL", "https://my-cluster.es.io:9200", + "http://:/mcp" + ] + } + } + } + ``` + + Note: For Basic auth, encode your credentials as base64. In shell: `echo -n "username:password" | base64` + +4. To use the SSE transport (deprecated, but still supported), start the server with `--sse` and use the `/mcp/sse` endpoint: + + ```json + { + "mcpServers": { + "elasticsearch-mcp-server": { + "command": "//.local/bin/mcp-proxy", + "args": [ + "--transport=sse", + "--header", "Authorization", "ApiKey ", + "http://:/mcp/sse" + ] + } + } + } + ``` + +### Configuration for Other MCP Clients + +Clients like **Cursor** and **Google Gemini** support direct connections to streamable-HTTP and SSE endpoints without requiring a proxy. + +#### Cursor Configuration + +Add to your Cursor MCP settings: + +```json +{ + "mcpServers": { + "elasticsearch": { + "url": "http://:/mcp", + "transport": "streamable-http", + "headers": { + "Authorization": "ApiKey ", + "X-Elasticsearch-URL": "https://my-cluster.es.io:9200" + } + } + } +} +``` + +#### Google Gemini Configuration + +Add to your Gemini MCP configuration: + +**For Streamable-HTTP:** +```json +{ + "mcpServers": { + "elasticsearch": { + "httpUrl": "http://:/mcp", + "headers": { + "Authorization": "ApiKey ", + "X-Elasticsearch-URL": "https://my-cluster.es.io:9200" + } + } + } +} +``` + +**For SSE (start server with `--sse`):** +```json +{ + "mcpServers": { + "elasticsearch": { + "url": "http://:/mcp/sse", + "headers": { + "Authorization": "ApiKey ", + "X-Elasticsearch-URL": "https://my-cluster.es.io:9200" + } + } + } +} +``` + +**Note**: Replace `` and `` with your server's address (default port is 8080). The dynamic headers (`Authorization` and `X-Elasticsearch-URL`) are optional if you've configured defaults via environment variables. + +### Kubernetes Deployment + +For deploying to Kubernetes (including OpenShift, EKS, GKE, AKS, etc.), use environment variables for configuration: + +#### Example Deployment YAML + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: elasticsearch-mcp +spec: + replicas: 1 + selector: + matchLabels: + app: elasticsearch-mcp + template: + metadata: + labels: + app: elasticsearch-mcp + spec: + containers: + - name: mcp-server + image: docker.elastic.co/mcp/elasticsearch:latest + command: ["elasticsearch-core-mcp-server"] + args: ["http", "--sse"] + ports: + - containerPort: 8080 + name: http + env: + - name: ES_URL + value: "https://elasticsearch.example.com:9200" + - name: ES_API_KEY + valueFrom: + secretKeyRef: + name: elasticsearch-credentials + key: api-key + - name: CONTAINER_MODE + value: "true" + - name: ENABLE_SSE + value: "true" + - name: HTTP_ADDRESS + value: "0.0.0.0:8080" + livenessProbe: + httpGet: + path: /_health/live + port: 8080 + initialDelaySeconds: 10 + periodSeconds: 30 + readinessProbe: + httpGet: + path: /_health/ready + port: 8080 + initialDelaySeconds: 5 + periodSeconds: 10 +--- +apiVersion: v1 +kind: Service +metadata: + name: elasticsearch-mcp +spec: + selector: + app: elasticsearch-mcp + ports: + - port: 8080 + targetPort: 8080 + name: http +``` + +**Key points for Kubernetes deployment:** +- Pass the command via `command` and `args` fields (e.g., `args: ["http", "--sse"]`) +- Alternatively, use `CLI_ARGS` environment variable if your deployment restricts command/args +- Use `CONTAINER_MODE=true` to bind to `0.0.0.0` instead of `127.0.0.1` +- Store sensitive credentials in Kubernetes Secrets and reference them via `secretKeyRef` +- Use the built-in health probes at `/_health/live` and `/_health/ready` +- For dynamic URL/auth support, configure via headers in your client or ingress/gateway + +#### Alternative: Using Only Environment Variables + +If your Kubernetes deployment restricts modifying the container command, use the `CLI_ARGS` approach: + +```yaml + containers: + - name: mcp-server + image: docker.elastic.co/mcp/elasticsearch:latest + # No command/args specified - uses container's default ENTRYPOINT + env: + - name: CLI_ARGS + value: "http --sse" + - name: ES_URL + value: "https://elasticsearch.example.com:9200" + - name: ES_API_KEY + valueFrom: + secretKeyRef: + name: elasticsearch-credentials + key: api-key + # ... rest of env vars same as above +``` + +Or use individual environment variables for maximum compatibility: + +```yaml + containers: + - name: mcp-server + image: docker.elastic.co/mcp/elasticsearch:latest + args: ["http"] # Just the subcommand + env: + - name: ENABLE_SSE + value: "true" + - name: CONTAINER_MODE + value: "true" + - name: HTTP_ADDRESS + value: "0.0.0.0:8080" + # ... rest of configuration +``` diff --git a/elastic-mcp.json5 b/elastic-mcp.json5 index 8d001a5..399ed71 100644 --- a/elastic-mcp.json5 +++ b/elastic-mcp.json5 @@ -2,7 +2,9 @@ { // Configure the target Elasticsearch server "elasticsearch": { - "url": "${ES_URL}", + // URL is optional and defaults to http://localhost:9200 + // Can also be overridden per-request via X-Elasticsearch-URL header (HTTP/SSE only) + "url": "${ES_URL:http://localhost:9200}", "api_key": "${ES_API_KEY:}", "username": "${ES_USERNAME:}", "password": "${ES_PASSWORD:}", diff --git a/src/cli.rs b/src/cli.rs index 02219ea..200b4a0 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -52,7 +52,7 @@ pub struct HttpCommand { pub address: Option, /// Also start an SSE server on '/sse' - #[clap(long)] + #[clap(long, env = "ENABLE_SSE")] pub sse: bool, } diff --git a/src/lib.rs b/src/lib.rs index 36edab4..9f31043 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -97,7 +97,7 @@ pub async fn setup_services(config: &Option, container_mode: bool) -> a // Built-in default configuration, based on env variables. r#"{ "elasticsearch": { - "url": "${ES_URL}", + "url": "${ES_URL:http://localhost:9200}", "api_key": "${ES_API_KEY:}", "username": "${ES_USERNAME:}", "password": "${ES_PASSWORD:}", diff --git a/src/servers/elasticsearch/base_tools.rs b/src/servers/elasticsearch/base_tools.rs index 2026ca3..9221184 100644 --- a/src/servers/elasticsearch/base_tools.rs +++ b/src/servers/elasticsearch/base_tools.rs @@ -39,9 +39,9 @@ pub struct EsBaseTools { } impl EsBaseTools { - pub fn new(es_client: Elasticsearch) -> Self { + pub fn new(es_client: Elasticsearch, ssl_skip_verify: bool) -> Self { Self { - es_client: EsClientProvider::new(es_client), + es_client: EsClientProvider::new(es_client, ssl_skip_verify), tool_router: Self::tool_router(), } } diff --git a/src/servers/elasticsearch/mod.rs b/src/servers/elasticsearch/mod.rs index 9043214..3cfdacf 100644 --- a/src/servers/elasticsearch/mod.rs +++ b/src/servers/elasticsearch/mod.rs @@ -39,8 +39,9 @@ use std::collections::HashMap; #[derive(Debug, Serialize, Deserialize)] pub struct ElasticsearchMcpConfig { - /// Cluster URL - pub url: String, + /// Cluster URL (optional if dynamic URL will be provided via X-Elasticsearch-URL header) + #[serde(default = "default_es_url", deserialize_with = "none_if_empty_string")] + pub url: Option, /// API key #[serde(default, deserialize_with = "none_if_empty_string")] @@ -68,41 +69,112 @@ pub struct ElasticsearchMcpConfig { // TODO: search as resources? } +fn default_es_url() -> Option { + Some("http://localhost:9200".to_string()) +} + // A wrapper around an ES client that provides a client instance configured -/// for a given request context (i.e. auth credentials) +/// for a given request context (i.e. auth credentials and URL) #[derive(Clone)] -pub struct EsClientProvider(Elasticsearch); +pub struct EsClientProvider { + client: Elasticsearch, + ssl_skip_verify: bool, +} impl EsClientProvider { - pub fn new(client: Elasticsearch) -> Self { - EsClientProvider(client) + pub fn new(client: Elasticsearch, ssl_skip_verify: bool) -> Self { + EsClientProvider { client, ssl_skip_verify } } - /// If the incoming request is a http request and has an `Authorization` header, use it - /// to authenticate to the remote ES instance. + /// If the incoming request is a http request and has headers for URL or Authorization, + /// use them to create a new client instance for the remote ES instance. pub fn get(&self, context: RequestContext) -> Cow<'_, Elasticsearch> { - let client = &self.0; + let client = &self.client; + + let parts = context.extensions.get::(); + + // Check for custom URL header + let custom_url = parts + .and_then(|p| p.headers.get("X-Elasticsearch-URL")) + .and_then(|h| h.to_str().ok()); - let Some(mut auth) = context - .extensions - .get::() + // Check for authorization header + let mut auth = parts .and_then(|p| p.headers.get(header::AUTHORIZATION)) - .and_then(|h| h.to_str().ok()) - else { - // No auth + .and_then(|h| h.to_str().ok()); + + // If neither URL nor auth is provided, return the default client + if custom_url.is_none() && auth.is_none() { + tracing::debug!("Using default Elasticsearch client configuration"); return Cow::Borrowed(client); - }; + } + + if custom_url.is_some() || auth.is_some() { + tracing::debug!( + "Dynamic configuration detected: custom_url={}, has_auth={}", + custom_url.is_some(), + auth.is_some() + ); + } // MCP inspector insists on sending a bearer token and prepends "Bearer" to the value provided - if auth.starts_with("Bearer ApiKey ") || auth.starts_with("Bearer Basic ") { - auth = auth.trim_start_matches("Bearer "); + if let Some(auth_str) = auth { + if auth_str.starts_with("Bearer ApiKey ") || auth_str.starts_with("Bearer Basic ") { + auth = Some(auth_str.trim_start_matches("Bearer ")); + } + } + + // If we have a custom URL, we need to build a new transport from scratch + if let Some(url_str) = custom_url { + match self.build_client_with_url(url_str, auth) { + Ok(new_client) => { + tracing::info!("Using dynamic Elasticsearch URL: {}", url_str); + return Cow::Owned(new_client); + } + Err(e) => { + tracing::error!( + "Failed to create client with custom URL '{}': {}. Falling back to default configuration.", + url_str, e + ); + // Fall back to just updating auth if we can't parse the URL + } + } + } + + // If we only have auth (no custom URL), just clone with new auth + if let Some(auth_str) = auth { + tracing::debug!("Using dynamic authentication with default URL"); + let transport = client + .transport() + .clone_with_auth(Some(Credentials::AuthorizationHeader(auth_str.to_string()))); + return Cow::Owned(Elasticsearch::new(transport)); } - let transport = client - .transport() - .clone_with_auth(Some(Credentials::AuthorizationHeader(auth.to_string()))); + // Fallback to default client + Cow::Borrowed(client) + } - Cow::Owned(Elasticsearch::new(transport)) + /// Build a new Elasticsearch client with a custom URL and optional auth + fn build_client_with_url(&self, url_str: &str, auth: Option<&str>) -> anyhow::Result { + let url = Url::parse(url_str)?; + let pool = elasticsearch::http::transport::SingleNodeConnectionPool::new(url); + let mut transport = elasticsearch::http::transport::TransportBuilder::new(pool); + + if let Some(auth_str) = auth { + transport = transport.auth(Credentials::AuthorizationHeader(auth_str.to_string())); + } + + if self.ssl_skip_verify { + transport = transport.cert_validation(CertificateValidation::None); + } + + transport = transport.header( + USER_AGENT, + HeaderValue::from_str(&format!("elastic-mcp/{}", env!("CARGO_PKG_VERSION")))?, + ); + + let transport = transport.build()?; + Ok(Elasticsearch::new(transport)) } } @@ -185,16 +257,26 @@ impl ElasticsearchMcp { None }; - let url = config.url.as_str(); - if url.is_empty() { + // Use the configured URL or fall back to default + let url_str = config.url.as_ref() + .ok_or_else(|| anyhow::Error::msg("Elasticsearch URL is not configured and no default is available"))?; + + if url_str.is_empty() { return Err(anyhow::Error::msg("Elasticsearch URL is empty")); } - let mut url = Url::parse(url)?; + let mut url = Url::parse(url_str)?; if container_mode { rewrite_localhost(&mut url)?; } + tracing::info!("Default Elasticsearch URL: {}", url); + if creds.is_some() { + tracing::info!("Using configured authentication credentials"); + } else { + tracing::info!("No default authentication configured (can be provided dynamically via headers)"); + } + let pool = elasticsearch::http::transport::SingleNodeConnectionPool::new(url.clone()); let mut transport = elasticsearch::http::transport::TransportBuilder::new(pool); if let Some(creds) = creds { @@ -210,7 +292,7 @@ impl ElasticsearchMcp { let transport = transport.build()?; let es_client = Elasticsearch::new(transport); - Ok(base_tools::EsBaseTools::new(es_client)) + Ok(base_tools::EsBaseTools::new(es_client, config.ssl_skip_verify)) } }