Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions vendor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,7 @@ these are upstream fixes a sync will *gain*.)

| Patch | Files | Status upstream |
|---|---|---|
| Grok usage-row accessor naming: `ParsedUsage::token_breakdown(&self)` avoids Rust 1.96.1's `clippy::wrong_self_convention` rejection of the former borrowed `into_*` name. This is a private-method rename only; parsed output, cache schema, and runtime behavior are unchanged. | `src/sessions/grok.rs` | TokenBar-local maintenance for the local `turn_completed.usage` path |
| PR #2 (perf, partially retired by M26-A): `LocalParseOptions.modified_after` mtime pruning and `latest_source_mtime_ms()` change probing remain active. M26-A removes `HASH_MEMO` because path/size/mtime memoization can bypass bounded sample validation, and removes monolith-only `STORE_MEMO` because shard dirty tracking owns persistence | `src/message_cache.rs`, `src/lib.rs` | Remaining mtime/pruning seam not yet forwarded; retired memos must not be re-applied |
| PR #3 (perf): streaming per-file aggregation replaces materialize-then-aggregate for the graph/model/monthly/hourly reports — `StreamingAggregator` + `SessionizeAccumulator` folded by `scan_messages_streaming` in one cache-aware pass (no full-history `Vec`). Each client lane owns its dedup set (follow-up `0752e35`: prevents cross-client `dedup_key` collisions). | `src/aggregator.rs`, `src/lib.rs`, `src/sessionize.rs`, `tests/streaming_snapshot.rs` | not yet forwarded to junhoyeo/tokscale |
| #6 (fix): the **agents report** now folds over `scan_messages_streaming` too — new `get_agents_report` (mirrors `get_model_report`, `resolve_report_clients` + a single streaming pass into `AgentAccumulator`), so it shares the one deduped/per-client-gated/priced stream as every other report (resolves the issue #6 divergence: agents no longer over-counts copilot/codebuff/kimi/cursor/warp/… duplicate `dedup_key`s, and scans the same client set). `parse_local_unified_messages` survives as public API only (footgun-documented, no in-repo callers). `crates/tb_core_ffi/src/agents_report.rs` is now a thin mapper like `model_report.rs` (no longer byte-identical to the archived Tauri original — accepted). | `src/lib.rs`, `crates/tb_core_ffi/src/agents_report.rs` | not yet forwarded to junhoyeo/tokscale |
Expand Down
30 changes: 13 additions & 17 deletions vendor/tokscale-core/SYNC.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,14 @@
| Field | Value |
|---|---|
| Source repo | [Nanako0129/TokenBar](https://github.com/Nanako0129/TokenBar) |
| Copied commit | [`55ca05d3a0a5bf0f02ed20f46bb3e73e65a07218`](https://github.com/Nanako0129/TokenBar/commit/55ca05d3a0a5bf0f02ed20f46bb3e73e65a07218) |
| Copied commit | [`1d874cdf1c6db97225d4703a25214be75bc33e7b`](https://github.com/Nanako0129/TokenBar/commit/1d874cdf1c6db97225d4703a25214be75bc33e7b) |
| Copied on | 2026-07-28 |

`55ca05d3a0a5bf0f02ed20f46bb3e73e65a07218` is the final rebase-merged Native
PR #111 commit on `main`. It preserves the post-merge M19-B checkpoint and adds
the issue-107 source-generation-aware filter-parity diagnostic: one Rust-owned
probe uses a fresh graph and brackets hourly and Agents nil/full reports with
opaque local-source tokens. Source movement is classified as `sourceChanged`
instead of a filter mismatch; independently refreshed cost remains diagnostic
but cannot decide parity. A hermetic vendor fixture covers exact client gates,
synthetic traffic, duplicate Codebuff roots, unattributed `Main`, cold/warm
cache, and inherited scanner-root isolation.
`1d874cdf1c6db97225d4703a25214be75bc33e7b` is the final rebase-merged Native
PR #112 commit on `main`. It preserves the issue-107 filter-parity diagnostic
and renames the private Grok `ParsedUsage` accessor to `token_breakdown(&self)`
so Rust 1.96.1 strict Clippy accepts its borrowed receiver. Parsing, cache
schema, C ABI, and runtime output are unchanged.

The active cache is format 2 at `source-message-cache-v2`; format-1 shards are
stale and rebuild cold under format 2. The legacy schema-32 monolith
Expand All @@ -32,21 +28,21 @@ cross-check copy are byte-identical to the same Native fixture.

The macOS-side downstream gate passed against this exact sync:

- `crates/` is byte-identical to Native `55ca05d3`; `vendor/` is
- `crates/` is byte-identical to Native `1d874cdf`; `vendor/` is
byte-identical after excluding Native-only `vendor/AGENTS.md` and this
Windows-only `SYNC.md`; the C header and both provider-v3 fixture copies also
match byte-for-byte.
- `Cargo.lock` is unchanged. The two new Rust fixture/source files pass a
focused `rustfmt --check`; the exact shared tree was not reformatted.
- Focused parity tests pass (`tb_core_ffi` 10/10 and vendor fixture 1/1).
- `Cargo.lock` is unchanged, `git diff --check` passes, and the exact shared
tree was not reformatted.
- Rust 1.96.1 strict Clippy passes for `tokscale-core`; focused Grok tests pass
49/49.
- `scripts/check.sh` passes with locked dependencies: workspace check, all Rust
tests (including 319 FFI and 1,290 vendor unit tests), release build, locked
.NET restore, solution build with zero warnings/errors, all 287 Core tests,
and the 11-entry P/Invoke smoke.
- The live smoke observed source movement while local sessions were changing,
so both parity reports correctly returned `sourceChanged` and skipped
comparison. Credential/network-backed agent usage was intentionally disabled
with `TB_SMOKE_SKIP_NETWORK=1`; pricing was cache-only.
comparison. The agent-usage probe also completed successfully.

This is local macOS-side evidence. Hosted Windows x64 and ARM64 cross-package
checks remain CI-owned, and this record does not claim real ARM64 runtime
Expand All @@ -60,7 +56,7 @@ then update its commit, date, and verification fields before committing:

```bash
: "${TOKENBAR_NATIVE:?set TOKENBAR_NATIVE to a clean Native checkout}"
source_commit=55ca05d3a0a5bf0f02ed20f46bb3e73e65a07218
source_commit=1d874cdf1c6db97225d4703a25214be75bc33e7b
stage="$(mktemp -d)"
sync_record="$(mktemp)"
trap 'rm -rf "$stage"; rm -f "$sync_record"' EXIT
Expand Down
4 changes: 2 additions & 2 deletions vendor/tokscale-core/src/sessions/grok.rs
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ impl ParsedUsage {

/// Grok's `inputTokens` includes cache reads. TokenBar totals sum every
/// bucket, so net uncached input into `input` and put cache in `cache_read`.
fn into_token_breakdown(&self) -> TokenBreakdown {
fn token_breakdown(&self) -> TokenBreakdown {
let cache_read = self.cached_read_tokens.max(0);
let input = self.input_tokens.max(0).saturating_sub(cache_read);
TokenBreakdown {
Expand Down Expand Up @@ -524,7 +524,7 @@ fn emit_usage_messages(
.filter(|m| !m.trim().is_empty())
.unwrap_or_else(|| fallback_model.to_string());

let tokens = row.into_token_breakdown();
let tokens = row.token_breakdown();
if tokens.input == 0
&& tokens.output == 0
&& tokens.cache_read == 0
Expand Down
Loading