Skip to content

feat: add Google Gemini provider#177

Open
AasheeshLikePanner wants to merge 1 commit intolemony-ai:mainfrom
AasheeshLikePanner:feature/gemini-provider
Open

feat: add Google Gemini provider#177
AasheeshLikePanner wants to merge 1 commit intolemony-ai:mainfrom
AasheeshLikePanner:feature/gemini-provider

Conversation

@AasheeshLikePanner
Copy link
Copy Markdown
Contributor

🎯 Description

  • Add GeminiProvider with support for all Gemini models (1.5, 2.0, 2.5, 3.x)
  • Support for completion, streaming, and tool calling
  • Updated pricing for latest models (March 2026):
    • Gemini 3.1 Pro Preview: $2.00/$12.00 per 1M tokens
    • Gemini 3.1 Flash Preview: $0.50/$3.00 per 1M tokens
    • Gemini 3.1 Flash-Lite Preview: $0.25/$1.50 per 1M tokens
    • Gemini 2.5 Pro: $1.25/$10.00 per 1M tokens
    • Gemini 2.5 Flash: $0.30/$2.50 per 1M tokens
    • Gemini 2.5 Flash-Lite: $0.10/$0.40 per 1M tokens
  • Automatic retry logic with exponential backoff
  • HTTP config support for enterprise SSL/proxy
  • Fix agent module export in cascadeflow/init.py

Tested with real API key - working correctly.

🔗 Related Issues

  • Related to #

🔄 Type of Change

  • ✨ New feature (non-breaking change which adds functionality)
  • 🚀 New provider integration
  • 🤖 Tool calling / function support

🧪 Testing

Test cases added

  • Manual testing

How to test

from cascadeflow.providers import PROVIDER_REGISTRY
from cascadeflow.providers.gemini import GeminiProvider
import os
os.environ['GEMINI_API_KEY'] = 'your-key'

provider = GeminiProvider()
response = await provider.complete(
    prompt="What is Python?",
    model="gemini-2.5-flash"
)
print(response.content)

📋 Checklist

Code Quality

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • My changes generate no new warnings
  • I have added type hints where appropriate
  • I have run `black` to format my code
  • I have run `ruff` and fixed all linting issues
  • I have run `mypy` for type checking

Testing

  • New and existing unit tests pass locally with my changes
  • Tested with real Gemini API key

Breaking Changes

  • This PR includes NO breaking changes

By submitting this PR, I confirm that:

  • I have read and followed the CONTRIBUTING.md guidelines
  • My contribution is my own original work or properly attributed
  • I agree to license my contribution under the project's MIT license
  • I have tested my changes thoroughly

- Add GeminiProvider with support for all Gemini models (1.5, 2.0, 2.5, 3.x)
- Support for completion, streaming, and tool calling
- Updated pricing for latest models (March 2026):
  - Gemini 3.1 Pro: .00/.00 per 1M tokens
  - Gemini 3.1 Flash: /bin/zsh.50/.00 per 1M tokens
  - Gemini 3.1 Flash-Lite: /bin/zsh.25/.50 per 1M tokens
  - Gemini 2.5 Pro: .25/.00
  - Gemini 2.5 Flash: /bin/zsh.30/.50
  - Gemini 2.5 Flash-Lite: /bin/zsh.10//bin/zsh.40
- Automatic retry logic with exponential backoff
- HTTP config support for enterprise SSL/proxy
- Fix agent module export in __init__.py

Tested with real API key - working correctly.
Copy link
Copy Markdown
Collaborator

@saschabuehrle saschabuehrle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. I reviewed and validated against current main. This PR is not merge-ready yet.

Required fixes before merge:

  1. Tool calling path is broken at runtime
  • _check_tool_support() returns True, but _complete_with_tools_impl() is not implemented.
  • With tools=..., BaseProvider routes to _complete_with_tools_impl() and raises NotImplementedError.
  1. Gemini request body shape is incorrect
  • systemInstruction and tools are currently placed inside generationConfig.
  • They must be top-level request fields for Gemini generateContent requests.
  1. Streaming path likely mismatched
  • Current code parses SSE data: lines but does not configure SSE mode consistently for Gemini streaming.
  • Ensure endpoint/query mode and parser match real Gemini stream format.
  1. Incorrect latency metric
  • latency_ms is currently set from promptTokenCount (token count, not latency).
  • Use measured elapsed time for latency.
  1. Add provider tests
  • Need targeted tests for: complete(), stream(), tool-calling path, and usage/cost parsing.

Also, this PR includes unrelated cascadeflow.agent export changes. Please keep provider work focused (or split into separate PRs).

Please address these and re-request review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants