Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dapr state store clickhouse #3675

Open
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

middt
Copy link

@middt middt commented Feb 23, 2025

Description

This PR adds a new state store component for ClickHouse, a column-oriented database management system. The ClickHouse state store component provides the following features:

  • Basic CRUD operations for state management
  • Support for TTL (Time-To-Live)
  • ETag support for optimistic concurrency
  • Bulk operations support
  • Username/password authentication
  • Uses ReplacingMergeTree engine for efficient updates

Key implementation details:

  • Uses the official ClickHouse Go driver
  • Implements the full state.Store interface
  • Handles connection management and cleanup
  • Includes comprehensive integration tests
  • Follows Dapr's component development guidelines

Implementation Details

The component includes:

  1. State store implementation (clickhouse.go)

    • Full CRUD operations
    • Bulk operations
    • TTL support
    • ETag support
    • Authentication support
  2. Tests (clickhouse_test.go)

    • Integration tests for CRUD operations
    • Metadata validation tests
    • Authentication tests
    • Cleanup handling
  3. Configuration options:

    apiVersion: dapr.io/v1alpha1
    kind: Component
    metadata:
      name: statestore
      namespace: default
    spec:
      type: state.clickhouse
      version: v1
      metadata:
      - name: clickhouseURL
        value: "tcp://localhost:9000"
      - name: databaseName
        value: "dapr_state"
      - name: tableName
        value: "state_store"
      - name: username
        value: "default"
      - name: password
        value: ""

Checklist

  • Code compiles correctly
  • Created/updated tests
  • Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

Testing Done

  1. Unit tests for metadata validation
  2. Integration tests for:
    • Basic CRUD operations
    • TTL functionality
    • ETag support
    • Bulk operations
    • Authentication
    • Error handling

Tests were run against ClickHouse v23.8 using the official Go driver.

Additional Notes

  • The component uses ReplacingMergeTree engine for better update performance
  • Includes proper cleanup in tests to avoid test pollution
  • Follows ClickHouse best practices for table design
  • Implements proper connection management and cleanup

@middt middt requested review from a team as code owners February 23, 2025 04:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant