Skip to content

feat(embed): update default embedding profile#291

Merged
shan-nvidia merged 36 commits into
mainfrom
oholworthy/embed-finetune-recipe-2607
Jul 15, 2026
Merged

feat(embed): update default embedding profile#291
shan-nvidia merged 36 commits into
mainfrom
oholworthy/embed-finetune-recipe-2607

Conversation

@oliverholworthy

@oliverholworthy oliverholworthy commented Jul 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Update the existing embed recipe's default profile for the new embedding model family, including isolated artifacts and checkpoint compatibility checks.
  • Keep the existing legacy profile available for training and deployment workflows.
  • Add direct consolidated-checkpoint deployment with Retriever NIM 2.1 and vLLM; vLLM evaluation uses /v2/embed with input_type.
  • Protect SDG API keys from saved configuration, CLI provenance, and remote Slurm job output.
  • Refresh profile-aware documentation and reproducible stage lockfiles.

Why

The previous default profile was coupled to its export-based serving contract. This update makes the new default use the native checkpoint path through NIM or vLLM, while retaining the prior profile for existing users.

Validation

  • uv run pytest tests/recipes/embed -q --basetemp=/tmp/nemotron-final-public-review — 278 passed, 4 skipped
  • Shared executor and SDG regression coverage — 106 passed
  • Root and changed stage uv lock --check validations passed
  • Live Retriever NIM 2.1 smoke: readiness plus query and passage embeddings, both 2,048 dimensions
  • Final diff hygiene and credential scan completed; no literal credentials found

Review note

Stage 2 currently pins the source archive commit a9f4423819c513fd08083324fe1f738746ac6e54 from Automodel PR #3021. That PR remains draft/open, so this PR stays draft pending the dependency's merge or an explicit preview-release decision.

@oliverholworthy oliverholworthy self-assigned this Jul 14, 2026
@oliverholworthy oliverholworthy added the enhancement New feature or request label Jul 14, 2026
@oliverholworthy oliverholworthy changed the title feat(embed): add embedding fine-tuning recipe feat(embed): update default embedding profile Jul 14, 2026
@oliverholworthy
feat(embed): add Nemotron 3 and Llama profiles
b101c5c 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
feat(embed): deploy PyTorch checkpoints with Retriever NIM
3f75481 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): align checkpoint and NIM evaluation
d865a2c 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
test(embed): cover dual-profile runtime contracts
8cd5c60 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
docs(embed): document Nemotron 3 and Llama workflows
7ee9b1b 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
Fix embed prep container dependencies
05dd91a 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
Serialize multi-GPU recipe environment setup
fb85e12 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): standardize prompt prefix separators
5effb52 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): enable secure W&B training telemetry
2da7b27 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
Fix Nemotron 3 embed publication blockers
08a2808 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
chore(skills): migrate retrieval eval dataset format
e63813b 
Restore EVAL.md from upstream/main and adopt the agentskills.io wrapper without changing existing case semantics.

Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
feat(embed): add vLLM deployment backend
caec8d5 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): validate empty NIM image fallback
3243de8 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): pin Automodel checkpoint exporter
a2d324e 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): align deploy fingerprint with checkpoint
d72d2a1 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): use vllm document input type
94f8b0e 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): support NIM 2.1 checkpoint deployment
5677d74 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
chore(skills): restore retrieval recipe metadata
7116653 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): protect SDG API keys
e907dc4 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
docs(embed): focus README on default workflow
d58a710 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
docs(embed): polish public recipe documentation
18688d2 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): update hosted SDG embedding model
b946b72 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
docs(embed): clarify recipe workflow and deployment
9dd5d4f 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): make endpoint drift gate fail closed
8e76773 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): correct deployment guidance
d68d09d 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
chore(embed): update Automodel container to 26.06
e94ec2c 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): use backend-neutral container name
db6397e 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
docs(embed): align public runtime guidance
7a5a142 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
docs(skills): sync embedding recipe guidance
aa83449 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
oliverholworthy force-pushed the oholworthy/embed-finetune-recipe-2607 branch from 29bb9bb to aa83449 Compare July 15, 2026 11:06
@oliverholworthy
fix(embed): let NIM select the runtime pipeline
15ef649 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
fix(embed): inherit NIM runtime defaults
ada3d9e 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
feat(embed): default to Retriever NIM 2.2
af6c696 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy
oliverholworthy marked this pull request as ready for review July 15, 2026 17:18
@oliverholworthy

oliverholworthy commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

/nvskills-ci

@oliverholworthy
chore(skills): bump retrieval recipes to 0.2.0
a0a8997 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy

oliverholworthy commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

/nvskills-ci

@oliverholworthy
fix(skills): avoid deployment guidance scan false positive
15c68bc 
Signed-off-by: Oliver Holworthy <1216955+oliverholworthy@users.noreply.github.com>
@oliverholworthy

oliverholworthy commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

/nvskills-ci

svc-nvskills-signing and others added 2 commits July 15, 2026 21:04
@svc-nvskills-signing
Attach NVSkills validation signatures
c06d7a8 
Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>
@shan-nvidia
Merge branch 'main' into oholworthy/embed-finetune-recipe-2607
e0c8215
shan-nvidia
shan-nvidia approved these changes Jul 15, 2026
View reviewed changes

@shan-nvidia shan-nvidia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@shan-nvidia
shan-nvidia merged commit 2a68716 into main Jul 15, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shan-nvidia shan-nvidia shan-nvidia approved these changes

Assignees

@oliverholworthy oliverholworthy

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@oliverholworthy @shan-nvidia @svc-nvskills-signing