Skip to content

Conversation

fbricon
Copy link
Contributor

@fbricon fbricon commented Sep 1, 2025

This PR enables users to toggle thinking for Ollama models that support it. In this recording, you can see qwen3 reasoning is not enabled by default, but setting reasoning : true in the defaultCompletionOptions allows it:

  - name: qwen3:8b
    provider: ollama
    model: qwen3:8b
    defaultCompletionOptions:
      reasoning: true
ollama-models-toggle-thinking.mp4
  • shows reasoning toggle if model supports reasoning or config explicitly forces it
  • reasoning session state properly resets when changing models
  • reasoning status passed as think:true to ollama
  • updated docs to mention support of reasoning configuration for ollama models

Hardcoding reasoning capabilities on ollama is tricky: e.g. qwen3:8b has reasoning capabilities but think=true causes an Ollama error if the version pulled into ollama is too old. Pulling a newer version will fix it.

Also worth noting some models, like gpt-oss:20b will just ignore the think:false flag passed to ollama. It supports different levels of thinking, but needs to be set in the system prompt. See https://huggingface.co/openai/gpt-oss-120b#reasoning-levels


Summary by cubic

Adds a reasoning/thinking toggle for supported Ollama models. Auto-detects support, shows the toggle in the UI, and sends think to Ollama; docs and schema updated.

  • New Features
    • Detects reasoning support (Anthropic, deepseek‑r, or when completionOptions.reasoning is true).
    • Shows the Reasoning toggle only when supported; resets state on model change.
    • Streams reasoning options: sets reasoning and budget for non‑Ollama; sends think to Ollama and skips budget.
    • Updates docs and VS Code schema to note reasoning support for some Ollama models.

@fbricon fbricon requested a review from a team as a code owner September 1, 2025 13:25
@fbricon fbricon requested review from Patrick-Erichsen and removed request for a team September 1, 2025 13:25
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Sep 1, 2025
@fbricon
Copy link
Contributor Author

fbricon commented Sep 1, 2025

I need to fix those gui tests

@fbricon fbricon force-pushed the ollama-support-reasoning-flag branch from c153d7b to ad392a7 Compare September 2, 2025 14:17
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Sep 2, 2025
Copy link
Collaborator

@Patrick-Erichsen Patrick-Erichsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few questions/nitpicks, thanks for the contribution @fbricon ! Appreciate the detailed description and screen recording.

Comment on lines +141 to +143
if ("anthropic" === model.underlyingProviderName) {
return true;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe older models such as 3.5 Sonnet don't support reasoning?

Although I'm a little confused, this file is just for Ollama autodetect, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just moved that existing check from gui/src/components/mainInput/InputToolbar.tsx

And no, this file seems to handle different models from different providers

@github-project-automation github-project-automation bot moved this from Todo to In Progress in Issues and PRs Sep 3, 2025
@fbricon fbricon force-pushed the ollama-support-reasoning-flag branch from ad392a7 to 14e84c9 Compare September 3, 2025 08:29
Copy link
Collaborator

@Patrick-Erichsen Patrick-Erichsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making those updates! Lmk if this is good to merge and I'll do so.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Sep 3, 2025
@fbricon
Copy link
Contributor Author

fbricon commented Sep 3, 2025

I can't test the Anthropic models so if you can do that and I didn't break anything, then go ahead and merge it

@RomneyDa
Copy link
Collaborator

@fbricon have verified this works with e.g. claude 4.1 opus local. Appreciate the contribution!

@RomneyDa RomneyDa merged commit bf9ba15 into continuedev:main Sep 11, 2025
55 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Issues and PRs Sep 11, 2025
@github-actions github-actions bot locked and limited conversation to collaborators Sep 11, 2025
@sestinj
Copy link
Contributor

sestinj commented Sep 11, 2025

🎉 This PR is included in version 1.13.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm This PR has been approved by a maintainer released size:L This PR changes 100-499 lines, ignoring generated files.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants