refactor(v1): replace custom template system with apply_chat_template

[frozenleaves]

Remove the hand-written v1 template/rendering stack (core/utils/rendering.py, plugins/model_plugins/rendering.py and the qwen3/qwen3_nothink template modules) and introduce a new core/rendering package that drives tokenization through the model's own apply_chat_template:

• rendering.py -- Renderer orchestration + render_messages + process_samples
• format.py -- v1<->HF message conversion + token subsequence search
• label.py -- assistant-region labeling + structural verification
• escape.py -- special-token escaping (prompt-injection hardening)
• markers.py -- per-model assistant role markers (explicit whitelist)

ModelArguments gains custom_chat_template (replacing the removed template field); ModelEngine syncs/injects the chat template via _sync_chat_template. Import paths for the moved Renderer are updated across base_sampler, base_trainer, model_engine, inference_engine, cli_sampler and core/utils/batching. Example configs and tests drop the obsolete template: key.

Also drop the unused regex-based parse_message: the CLI sampler now appends the assistant turn as raw text to the conversation history.

[gemini-code-assist]

Code Review

This pull request refactors the rendering system in LlamaFactory, replacing the previous custom rendering plugins with a unified, secure Renderer class that handles message formatting, special-token escaping, and assistant-region labeling. Feedback on the changes highlights several critical improvements for robustness and correctness. Key recommendations include explicitly generating and resetting position_ids for pairwise/DPO sequences to prevent positional embedding offsets, adding defensive checks and try-except blocks to handle malformed JSON in tool calls and tool definitions, and implementing safety checks to avoid potential AttributeError crashes when accessing tokenizer attributes or when get_tokenizer returns None.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

If get_tokenizer(self.processor) returns None (e.g., if the processor does not have an associated tokenizer), accessing tokenizer.chat_template will raise an AttributeError. Adding a safety check ensures robust execution.

Suggested change

	def _sync_chat_template(self) -> None:
	"""Sync chat_template and inject custom_chat_template."""
	tokenizer = get_tokenizer(self.processor)
	if not is_tokenizer(self.processor) and not getattr(self.processor, "chat_template", None):
	if getattr(tokenizer, "chat_template", None):
	self.processor.chat_template = tokenizer.chat_template
	if self.args.custom_chat_template:
	if not is_tokenizer(self.processor):
	self.processor.chat_template = self.args.custom_chat_template
	else:
	tokenizer.chat_template = self.args.custom_chat_template
	def _sync_chat_template(self) -> None:
	"""Sync chat_template and inject custom_chat_template."""
	tokenizer = get_tokenizer(self.processor)
	if not is_tokenizer(self.processor) and not getattr(self.processor, "chat_template", None):
	if tokenizer is not None and getattr(tokenizer, "chat_template", None):
	self.processor.chat_template = tokenizer.chat_template
	if self.args.custom_chat_template:
	if not is_tokenizer(self.processor):
	self.processor.chat_template = self.args.custom_chat_template
	elif tokenizer is not None:
	tokenizer.chat_template = self.args.custom_chat_template