Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
4892ef5
testing changes
madhav165 Oct 29, 2025
8412188
Add JS for file validation
madhav165 Nov 3, 2025
1200852
JS cleanup
madhav165 Nov 3, 2025
0d68908
linting fixes
madhav165 Nov 3, 2025
2be6f5a
Working till adding gateway
madhav165 Nov 3, 2025
79e0c9c
Use ca cert for tool calls
madhav165 Nov 3, 2025
c95f3ea
Fix health checks
madhav165 Nov 3, 2025
c3eef69
Use ca_cert in update_gateway
madhav165 Nov 3, 2025
f8cbc90
Flake8 fixes
madhav165 Nov 3, 2025
4973ad8
Linting fixes
madhav165 Nov 3, 2025
fa1b2dd
Update doctest
madhav165 Nov 3, 2025
1fa0b35
Add Ed25519 signing code
madhav165 Nov 4, 2025
ea6baf6
Add validator for public key
madhav165 Nov 4, 2025
aa87aa1
Add cert validation
madhav165 Nov 4, 2025
035c5ac
Allow multiple uploads
madhav165 Nov 4, 2025
17f6510
linting fixes
madhav165 Nov 4, 2025
18eb3f8
bandit fix
madhav165 Nov 4, 2025
475e99d
Fix some tests
madhav165 Nov 4, 2025
4c9ad0d
Fix test
madhav165 Nov 4, 2025
7fe0e86
Fix tests
madhav165 Nov 5, 2025
96dbaef
Linting fixes
madhav165 Nov 5, 2025
dc87607
Fix fstring
madhav165 Nov 5, 2025
7862668
eslint fixes
madhav165 Nov 5, 2025
9549ca0
lint-web fixes
madhav165 Nov 5, 2025
ca6aa3b
Add alembic migration
madhav165 Nov 5, 2025
ce71623
Make signing certs optional
madhav165 Nov 5, 2025
55e6b85
Update README
madhav165 Nov 5, 2025
6cbf51b
Minor change to README
madhav165 Nov 5, 2025
bf295f6
Update sso_provider field validator
madhav165 Nov 5, 2025
e6086ac
Update charts
madhav165 Nov 5, 2025
9541f56
flake8 fix
madhav165 Nov 5, 2025
8922b55
flake8 fixes
madhav165 Nov 5, 2025
e18163d
Use Containerfile.lite in docker compose
madhav165 Nov 5, 2025
0f50431
checking compose upgrade for pg 18
madhav165 Nov 5, 2025
48e3202
Include pg_hba.conf step in upgrade
madhav165 Nov 5, 2025
a753f45
lint fix
madhav165 Nov 5, 2025
b5dfeb4
Mention about Postgres upgrade in Changelog
madhav165 Nov 5, 2025
9aea196
Minor fix to commented alembic upgrade
madhav165 Nov 6, 2025
41327c6
Add documentation on self signed certs
madhav165 Nov 7, 2025
1daf7c2
fix: resolve rebase conflicts and update plugin API calls
crivetimihai Nov 8, 2025
7d439f4
Rebase
crivetimihai Nov 8, 2025
a5aa466
Coverage
crivetimihai Nov 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -1179,3 +1179,11 @@ PAGINATION_INCLUDE_LINKS=true

# Enable event logging within spans
# OBSERVABILITY_EVENTS_ENABLED=true

#####################################
# Ed25519 Key Support
#####################################
# Ed25519 private key for signing
ENABLE_ED25519_SIGNING=false
PREV_ED2519_PRIVATE_KEY=""
ED25519_PRIVATE_KEY=private_key
6 changes: 3 additions & 3 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# ===============================================================
#
# - runs the full test-suite across three Python versions
# - measures branch + line coverage (fails < 40 %)
# - measures branch + line coverage (fails < 60% main, < 40% doctest)
# - uploads the XML/HTML coverage reports as build artifacts
# - (optionally) generates / commits an SVG badge - kept disabled
# - posts a concise per-file coverage table to the job summary
Expand Down Expand Up @@ -67,7 +67,7 @@ jobs:
python-version: ${{ matrix.python }}

# -----------------------------------------------------------
# 3️⃣ Run the tests with coverage (fail under 65% coverage)
# 3️⃣ Run the tests with coverage (fail under 60% coverage)
# -----------------------------------------------------------
- name: πŸ§ͺ Run pytest
run: |
Expand All @@ -78,7 +78,7 @@ jobs:
--cov-report=html \
--cov-report=term \
--cov-branch \
--cov-fail-under=65
--cov-fail-under=60

# -----------------------------------------------------------
# 4️⃣ Run doctests (fail under 40% coverage)
Expand Down
117 changes: 114 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@

---

## [0.9.0] - 2025-11-04 [WIP] - REST Passthrough, Multi-Tenancy Fixes & Platform Enhancements
## [0.9.0] - 2025-11-05 - REST Passthrough, Ed25519 Certificate Signing, Multi-Tenancy Fixes & Platform Enhancements

### Overview

This release delivers **REST API Passthrough Capabilities**, **API & UI Pagination**, **Multi-Tenancy Bug Fixes**, and **Platform Enhancements** with **60+ issues resolved** and **50+ PRs merged**, bringing significant improvements across security, observability, and developer experience:
This release delivers **Ed25519 Certificate Signing**, **REST API Passthrough Capabilities**, **API & UI Pagination**, **Multi-Tenancy Bug Fixes**, and **Platform Enhancements** with **60+ issues resolved** and **50+ PRs merged**, bringing significant improvements across security, observability, and developer experience:

- **πŸ“„ REST API & UI Pagination** - Comprehensive pagination support for all admin endpoints with HTMX-based UI and performance testing up to 10K records
- **πŸ”Œ REST Passthrough API Fields** - Comprehensive REST tool configuration with query/header mapping, timeouts, and plugin chains
Expand All @@ -21,6 +21,92 @@ This release delivers **REST API Passthrough Capabilities**, **API & UI Paginati
- **πŸ¦€ Rust Plugin Framework** - Optional Rust-accelerated plugins with 5-100x performance improvements
- **πŸ’» Admin UI** - Quality of life improvements for admins when managing MCP servers

### ⚠️ BREAKING CHANGES

#### **πŸ—„οΈ PostgreSQL 17 β†’ 18 Upgrade Required**

**Docker Compose users must run the upgrade utility before starting the stack.**

The default PostgreSQL image has been upgraded from version 17 to 18. This is a **major version upgrade** that requires a one-time data migration using `pg_upgrade`.

**Migration Steps:**

1. **Stop your existing stack:**
```bash
docker compose down
```

2. **Run the automated upgrade utility:**
```bash
make compose-upgrade-pg18
```

This will:
- Prompt for confirmation (⚠️ **backup recommended**)
- Run `pg_upgrade` to migrate data from Postgres 17 β†’ 18
- Automatically copy `pg_hba.conf` to preserve network access settings
- Create a new `pgdata18` volume with upgraded data

3. **Start the upgraded stack:**
```bash
make compose-up
```

4. **(Optional) Run maintenance commands** to update statistics:
```bash
docker compose exec postgres /usr/lib/postgresql/18/bin/vacuumdb --all --analyze-in-stages --missing-stats-only -U postgres
docker compose exec postgres /usr/lib/postgresql/18/bin/vacuumdb --all --analyze-only -U postgres
```

5. **Verify the upgrade:**
```bash
docker compose exec postgres psql -U postgres -c 'SELECT version();'
# Should show: PostgreSQL 18.x
```

6. **(Optional) Clean up old volume** after confirming everything works:
```bash
docker volume rm mcp-context-forge_pgdata
```

**Manual Upgrade (without Make):**

If you prefer not to use the Makefile:

```bash
# Stop stack
docker compose down

# Run upgrade
docker compose -f docker-compose.yml -f compose.upgrade.yml run --rm pg-upgrade

# Copy pg_hba.conf
docker compose -f docker-compose.yml -f compose.upgrade.yml run --rm pg-upgrade \
sh -c "cp /var/lib/postgresql/OLD/pg_hba.conf /var/lib/postgresql/18/docker/pg_hba.conf"

# Start upgraded stack
docker compose up -d
```

**Why This Change:**

- Postgres 18 introduces a new directory structure (`/var/lib/postgresql/18/docker`) for better compatibility with `pg_ctlcluster`
- Enables future upgrades using `pg_upgrade --link` without mount point boundary issues
- Aligns with official PostgreSQL Docker image best practices (see [postgres#1259](https://github.com/docker-library/postgres/pull/1259))

**What Changed:**

- `docker-compose.yml`: Updated from `postgres:17` β†’ `postgres:18`
- Volume mount: Changed from `pgdata:/var/lib/postgresql/data` β†’ `pgdata18:/var/lib/postgresql`
- Added `compose.upgrade.yml` for automated upgrade process
- Added `make compose-upgrade-pg18` target for one-command upgrades

**Troubleshooting:**

- **Error: "data checksums mismatch"** - Fixed automatically in upgrade script (disables checksums to match old cluster)
- **Error: "no pg_hba.conf entry"** - Fixed automatically by copying old `pg_hba.conf` during upgrade
- **Error: "Invalid cross-device link"** - Upgrade uses copy mode (not `--link`) to work across different Docker volumes

### Added

#### **πŸ“„ REST API and UI Pagination** (#1224, #1277)
Expand Down Expand Up @@ -101,7 +187,32 @@ This release delivers **REST API Passthrough Capabilities**, **API & UI Paginati
* **Keycloak Integration** (#1217, #1216, #1109) - Full Keycloak support with application/x-www-form-urlencoded
* **OAuth Timeout Configuration** (#1201) - Configurable `OAUTH_DEFAULT_TIMEOUT` for OAuth providers

#### **πŸ”Œ Plugin Framework Enhancements** (#1196, #1198, #1137, #1240, #1289)
#### **οΏ½ Ed25519 Certificate Signing** - Enhanced certificate validation and integrity verification
* **Digital Certificate Signing** - Sign and verify certificates using Ed25519 cryptographic signatures
- Ensures certificate authenticity and prevents tampering
- Built on proven Ed25519 algorithm (RFC 8032) for high security and performance
- Zero-dependency Python implementation using `cryptography` library
* **Key Generation Utility** - Built-in key generation tool at `mcpgateway/utils/generate_keys.py`
- Generates secure Ed25519 private keys in base64 format
- Simple command-line interface for development and production use
* **Key Rotation Support** - Graceful key rotation with zero downtime
- Configure both current (`ED25519_PRIVATE_KEY`) and previous (`PREV_ED25519_PRIVATE_KEY`) keys
- Automatic fallback to previous key for verification during rotation period
- Supports rolling updates in distributed deployments
* **Environment Variable Configuration** - Three new environment variables for certificate signing
- `ENABLE_ED25519_SIGNING` - Enable/disable signing (default: "false")
- `ED25519_PRIVATE_KEY` - Current signing key (base64-encoded)
- `PREV_ED25519_PRIVATE_KEY` - Previous key for rotation support (base64-encoded)
* **Kubernetes & Helm Support** - Full integration with Helm chart deployment
- Secret management via `values.yaml` configuration
- JSON Schema validation in `values.schema.json`
- External Secrets Operator integration examples
* **Production Ready** - Comprehensive documentation and security best practices
- Complete documentation in main README.md
- Helm chart documentation with Kubernetes examples
- Security guidelines for key storage and rotation

#### **οΏ½πŸ”Œ Plugin Framework Enhancements** (#1196, #1198, #1137, #1240, #1289)
* **πŸ¦€ Rust Plugin Framework** (#1289, #1249) - Optional Rust-accelerated plugins with automatic Python fallback
- Complete PyO3-based framework for building high-performance plugins
- **PII Filter (Rust)**: 5-100x faster than Python implementation with identical functionality
Expand Down
15 changes: 15 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2780,6 +2780,21 @@ compose-validate:
$(COMPOSE) config --quiet
@echo "βœ… Compose file is valid"

compose-upgrade-pg18: compose-validate
@echo "⚠️ This will upgrade Postgres 17 -> 18"
@echo "⚠️ Make sure you have a backup!"
@read -p "Continue? [y/N] " confirm && [ "$$confirm" = "y" ] || exit 1
@echo "πŸ”„ Running Postgres upgrade..."
$(COMPOSE) -f $(COMPOSE_FILE) -f compose.upgrade.yml run --rm pg-upgrade
@echo "πŸ”§ Copying pg_hba.conf from old cluster..."
@$(COMPOSE) -f $(COMPOSE_FILE) -f compose.upgrade.yml run --rm pg-upgrade sh -c \
"cp /var/lib/postgresql/OLD/pg_hba.conf /var/lib/postgresql/18/docker/pg_hba.conf && \
echo 'βœ… pg_hba.conf copied successfully'"
@echo "βœ… Upgrade complete!"
@echo "πŸ“ Next steps:"
@echo " 1. Update docker-compose.yml to use postgres:18"
@echo " 2. Run: make compose-up"

compose-up: compose-validate
@echo "πŸš€ Using $(COMPOSE_CMD); starting stack..."
IMAGE_LOCAL=$(call get_image_name) $(COMPOSE) up -d
Expand Down
70 changes: 70 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1506,6 +1506,76 @@ ContextForge implements **OAuth 2.0 Dynamic Client Registration (RFC 7591)** and
> Documentation endpoints (`/docs`, `/redoc`, `/openapi.json`) are always protected by authentication.
> By default, they require Bearer token authentication. Setting `DOCS_ALLOW_BASIC_AUTH=true` enables HTTP Basic Authentication as an additional method using the same credentials as `BASIC_AUTH_USER` and `BASIC_AUTH_PASSWORD`.

### Ed25519 Certificate Signing

MCP Gateway supports **Ed25519 digital signatures** for certificate validation and integrity verification. This cryptographic signing mechanism ensures that CA certificates used by the gateway are authentic and haven't been tampered with.

| Setting | Description | Default | Options |
| --------------------------- | ------------------------------------------------ | ------- | ------- |
| `ENABLE_ED25519_SIGNING` | Enable Ed25519 signing for certificates | `false` | bool |
| `ED25519_PRIVATE_KEY` | Ed25519 private key for signing (PEM format) | (none) | string |
| `PREV_ED25519_PRIVATE_KEY` | Previous Ed25519 private key for key rotation | (none) | string |

**How It Works:**

1. **Certificate Signing** - When `ENABLE_ED25519_SIGNING=true`, the gateway signs the CA certificate of each MCP server/gateway using the Ed25519 private key.

2. **Certificate Validation** - Before using a CA certificate for subsequent calls, the gateway validates its signature to ensure authenticity and integrity.

3. **Disabled Mode** - When `ENABLE_ED25519_SIGNING=false`, certificates are neither signed nor validated (default behavior).

**Key Generation:**

```bash
# Generate a new Ed25519 key pair
python mcpgateway/utils/generate_keys.py

# Output will show:
# - Private key (set this to ED25519_PRIVATE_KEY)
```

**Key Rotation:**

To rotate keys without invalidating existing signed certificates:

1. Move the current `ED25519_PRIVATE_KEY` value to `PREV_ED25519_PRIVATE_KEY`
2. Generate a new key pair using the command above
3. Set the new private key to `ED25519_PRIVATE_KEY`
4. The gateway will automatically re-sign valid certificates at the point of key change

**Example Configuration:**

```bash
# Enable Ed25519 signing
ENABLE_ED25519_SIGNING=true

# Current signing key (PEM format)
ED25519_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEIJ5pW... (your key here)
-----END PRIVATE KEY-----"

# Previous key for rotation (optional)
PREV_ED25519_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEIOld... (old key here)
-----END PRIVATE KEY-----"
```

> πŸ” **Security Best Practices:**
> - Store private keys securely (use secrets management tools like Vault, AWS Secrets Manager, etc.)
> - Rotate keys periodically (recommended: every 90-180 days)
> - Never commit private keys to version control
> - Use environment variables or encrypted config files
>
> πŸ”‘ **Public Key Derivation:**
> - Public keys are automatically derived from private keys
> - No need to configure public keys separately
> - Both `ED25519_PUBLIC_KEY` and `PREV_ED25519_PUBLIC_KEY` are computed at startup
>
> ⚑ **Performance:**
> - Ed25519 signing is extremely fast (~64 microseconds per signature)
> - Minimal impact on gateway performance
> - Recommended for production deployments requiring certificate integrity

### Response Compression

MCP Gateway includes automatic response compression middleware that reduces bandwidth usage by 30-70% for text-based responses (JSON, HTML, CSS, JS). Compression is negotiated automatically based on client `Accept-Encoding` headers with algorithm priority: **Brotli** (best compression) > **Zstd** (fastest) > **GZip** (universal fallback).
Expand Down
15 changes: 15 additions & 0 deletions charts/mcp-stack/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)

---

## [0.9.0] - 2025-11-05

### Added
* **Ed25519 Certificate Signing** - Digital certificate signing and verification using Ed25519 cryptographic signatures:
- `ENABLE_ED25519_SIGNING` - Enable/disable Ed25519 certificate signing (default: "false")
- `ED25519_PRIVATE_KEY` - Current Ed25519 private key for signing (base64-encoded)
- `PREV_ED25519_PRIVATE_KEY` - Previous Ed25519 private key for graceful key rotation (base64-encoded)
- Built-in key generation utility at `mcpgateway/utils/generate_keys.py`
- Zero-downtime key rotation support with automatic fallback verification
- Complete documentation with Kubernetes examples and External Secrets integration
- Ensures certificate authenticity and prevents tampering using RFC 8032 algorithm

### Changed
* **Chart version** - Bumped to 0.9.0 for Ed25519 certificate signing support

## [0.6.1] - 2025-09-01

### Added
Expand Down
Loading
Loading