feat(adapters): HRM-Text dual-timescale recurrent adapter - #60
Merged
Conversation
First recurrent architecture in the adapter registry. HRM-Text (HrmTextForCausalLM, Sapient HRM-Text) is dual-timescale: two separately- parameterised H/L block stacks (slow z_H / fast z_L) that share a block design but own distinct weights; per forward the L-stack executes 6x and the H-stack 2x. The safetensors stores fused attn.gqkv_proj (gated QKV) and mlp.gate_up_proj — the converter reads these directly (128 fused ternary tensors for the 1B, 64 H + 64 L; not the 256 split state_dict). MagicNorm is parameterless, so the only protected tensors are the embeddings, the untied LM head, and the 1-D recurrent init z_L_init (named explicitly). classify_weight reuses the standard substring+ dimensionality policy; a stack_of() helper tags H/L/shared for per-stack reporting of the asymmetric reuse leverage. Pure addition: one new adapter file + one registry entry (registry canonical-set test updated to match). No existing adapter or converter logic touched. Verified against sapientinc/HRM-Text-1B: dry-run + full convert both 128 ternary / 3 fp16 / 83.0% ternary / 3.07x vs bf16 / ~63s, matching the runtime-registered prototype byte-for-byte (probe HRM_PROBE_20260613T011321Z). Footprint only — no throughput (recurrence-forward runtime integration scoped, not run) and no coherence claim (Lane B held under program QAT hold). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First recurrent architecture in the adapter registry — the "compresses recurrent architectures" capability (probe
HRM_PROBE_20260613T011321Z, 2026-06-13).What
src/terncore/adapters/hrm_text.py(HrmTextForCausalLM,model_type=hrm_text).adapters/__init__.py; registry canonical-set test updated to match.tests/test_hrm_text_adapter.py(22 tests).Architecture notes
H_moduleslowz_H/L_modulefastz_L) share a block design but own distinct weights. Per forward the L-stack executes 6× (2 H-cycles × 3 L-steps), the H-stack 2× — equal footprint, asymmetric reuse leverage.stack_of()tags H/L/shared for per-stack reporting.attn.gqkv_proj(gated QKV) andmlp.gate_up_proj; the converter reads them directly → 128 fused ternary tensors (64 H + 64 L), not the 256 of the split state_dict. No H/L collision.z_L_init(named explicitly).Verification (
sapientinc/HRM-Text-1B)Dry-run and full CLI convert both: 128 ternary / 3 fp16 / 83.0% ternary / 3.07× vs bf16 / ~63 s, byte-for-byte identical to the runtime-registered prototype. 96 adapter/registry/validation tests pass locally.
Scope
Pure addition — no existing adapter or converter logic touched (only the registry list + its canonical-set test). Footprint only: no throughput (recurrence-forward runtime integration is scoped, not done) and no coherence claim (Lane B held under the program QAT hold).
🤖 Generated with Claude Code