Skip to content

metric(qwen3): report prefix-cache query/hit - #669

Open
Kokoro2336 wants to merge 6 commits into
openinfer-project:mainfrom
Kokoro2336:feat/issue-603
Open

metric(qwen3): report prefix-cache query/hit#669
Kokoro2336 wants to merge 6 commits into
openinfer-project:mainfrom
Kokoro2336:feat/issue-603

Conversation

@Kokoro2336

@Kokoro2336 Kokoro2336 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Description

Resolves #603

Report prefix-cache query/hit

Type of Change

  • New feature (non-breaking change which adds functionality)

Checklist

  • My code follows the style guidelines of this project (see docs/conventions/coding-style.md).

  • I have performed a self-review of my own code.

  • I have formatted my commits according to Commitizen conventions.

  • I have run the local test suite and all tests pass (see CLAUDE.md).

@Kokoro2336

Copy link
Copy Markdown
Contributor Author
ba1316c6ee82ac87d726104276f97fc9

I've made a simple test on the feature, sending the same prompt twice and filtering out queries/hits.

@Kokoro2336
Kokoro2336 marked this pull request as ready for review July 17, 2026 06:29
@Kokoro2336
Kokoro2336 marked this pull request as draft July 17, 2026 13:31
@Kokoro2336
Kokoro2336 marked this pull request as ready for review July 17, 2026 15:12

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6bfd50749b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread openinfer-qwen3/src/executor.rs Outdated
@xiaguan xiaguan self-assigned this Jul 18, 2026
@xiaguan

xiaguan commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

This PR makes openinfer-kv-offload optional without forwarding the feature from openinfer-server, so the default Qwen3 server rejects --kv-offload while recommending a nonexistent qwen3-kv-offload feature.

@Kokoro2336

Copy link
Copy Markdown
Contributor Author

@xiaguan Fixed

Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

metrics: report real prefix-cache query/hit counters from the qwen3 scheduler

2 participants