Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills/azure-foundry-finetuning
1 change: 1 addition & 0 deletions .claude/skills/azure-foundry-finetuning
31 changes: 31 additions & 0 deletions .github/skills/azure-foundry-finetuning/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
name: azure-foundry-finetuning-py
description: Submit fine-tuning jobs on Azure AI Foundry.
compatibility: Requires uv python package
license: MIT
metadata:
author: Microsoft
version: "1.0.0"
package: azure-ai-projects
---

# Azure AI Foundry Fine-Tuning Py

## When to use this

This skill helps users submit supervised fine-tuning (SFT) jobs on Azure AI Foundry. Confirm with the user which arguments are used for the fine-tuning job before submission.

## Prerequisite enforcement

Before running any script in this skill, you must verify `uv` is available:

`command -v uv >/dev/null 2>&1 || pip install uv`

If `uv` is still unavailable after installation, stop and report the prerequisite failure. Do not run this skill's scripts without `uv`.

## Available scripts

Use --help flag to see the usage of each script, e.g. `uv run scripts/submit_sft.py --help`.

- **`scripts/submit_sft.py`** — Submits SFT fine-tuning jobs to Azure AI Foundry. After successfully submitting the job, ask user if they want to monitor the job, and if yes, call `monitor_ft_job.py` with the returned job ID.
- **`scripts/monitor_ft_job.py`** — Monitors a fine-tuning job by job ID. Stream the output of monitoring to the user until the job is completed, and report the final status of the job.
10 changes: 10 additions & 0 deletions .github/skills/azure-foundry-finetuning/scripts/common.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import argparse
import sys


class HelpOnErrorParser(argparse.ArgumentParser):
"""ArgumentParser that prints full help when parsing fails."""

def error(self, message):
self.print_help(sys.stderr)
self.exit(2, f"\nerror: {message}\n")
71 changes: 71 additions & 0 deletions .github/skills/azure-foundry-finetuning/scripts/monitor_ft_job.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/usr/bin/env python3
# /// script
# dependencies = [
# "azure-identity",
# "azure-ai-projects",
# ]
# ///
import argparse
import sys
import time
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from common import HelpOnErrorParser


TERMINAL_STATUSES = {"succeeded", "failed", "cancelled"}


def monitor_ft_job(args):
credential = DefaultAzureCredential()
project_client = AIProjectClient(endpoint=args.project_endpoint, credential=credential)
openai_client = project_client.get_openai_client()

print(f"Monitoring fine-tuning job: {args.job_id}")
while True:
job = openai_client.fine_tuning.jobs.retrieve(args.job_id)
status = (getattr(job, "status", "") or "").lower()
print(f"Job status: {status}")

if status in TERMINAL_STATUSES:
print(f"Job reached terminal status: {status}")
if status == "succeeded":
return
raise RuntimeError(f"Fine-tuning job ended with terminal status: {status}")

time.sleep(args.poll_interval)


def build_parser():
parser = HelpOnErrorParser(
description="Monitor an SFT fine-tuning job until completion",
epilog=(
"Example:\n"
" ./monitor_ft_job.py --project-endpoint https://<resource>.services.ai.azure.com/api/projects/<project> --job-id ftjob_123"
),
formatter_class=argparse.RawTextHelpFormatter,
)
parser.add_argument(
"--project-endpoint",
required=True,
help="Azure AI Project endpoint URL",
)
parser.add_argument("--job-id", required=True, help="Fine-tuning job ID")
parser.add_argument(
"--poll-interval",
type=int,
default=10,
help="Polling interval in seconds (default: 10)",
)
return parser


if __name__ == "__main__":
parser = build_parser()

# If no arguments are provided, show help instead of failing with usage only.
if len(sys.argv) == 1:
parser.print_help()
parser.exit(0)

monitor_ft_job(parser.parse_args())
127 changes: 127 additions & 0 deletions .github/skills/azure-foundry-finetuning/scripts/submit_sft.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
#!/usr/bin/env python3
# /// script
# dependencies = [
# "azure-identity",
# "azure-ai-projects",
# ]
# ///
import argparse
import sys
import time
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from common import HelpOnErrorParser


def upload_and_wait_for_processing(openai_client, path, label, timeout_seconds=600, poll_interval_seconds=5):
"""Upload a file and wait until it is processed before proceeding."""
with open(path, "rb") as file_handle:
uploaded = openai_client.files.create(file=file_handle, purpose="fine-tune")

file_id = getattr(uploaded, "id", None)
if not file_id:
raise RuntimeError(f"{label} upload did not return a file id.")

print(f"Uploaded {label} file: {file_id}. Waiting for processing...")

deadline = time.time() + timeout_seconds
while time.time() < deadline:
current = openai_client.files.retrieve(file_id)
status = (getattr(current, "status", "") or "").lower()

if status == "processed":
print(f"{label} file processed successfully: {file_id}")
return file_id

if status in {"failed", "error", "cancelled"}:
raise RuntimeError(
f"{label} file processing failed for {file_id} with status '{status}'."
)

time.sleep(poll_interval_seconds)

raise TimeoutError(
f"Timed out after {timeout_seconds}s waiting for {label} file {file_id} to process."
)


def submit_sft(args):
credential = DefaultAzureCredential()

project_endpoint = args.project_endpoint
project_client = AIProjectClient(endpoint=project_endpoint, credential=credential)

openai_client = project_client.get_openai_client()

train_file_id = upload_and_wait_for_processing(
openai_client,
args.training_file,
"training",
)
val_file_id = upload_and_wait_for_processing(
openai_client,
args.validation_file,
"validation",
)

hyperparameters = {}
if args.epochs is not None:
hyperparameters["n_epochs"] = args.epochs
if args.batch_size is not None:
hyperparameters["batch_size"] = args.batch_size
if args.learning_rate is not None:
hyperparameters["learning_rate_multiplier"] = args.learning_rate

supervised = {}
if hyperparameters:
supervised["hyperparameters"] = hyperparameters

job_kwargs = dict(
model=args.model,
training_file=train_file_id,
validation_file=val_file_id,
method={
"type": "supervised",
"supervised": supervised,
},
suffix=args.suffix,
)

job = openai_client.fine_tuning.jobs.create(**job_kwargs)

print("Submitted finetuning job:", job.id)


def build_parser():
parser = HelpOnErrorParser(
description="Submit SFT fine-tuning job",
epilog=(
"Example:\n"
" ./submit_sft.py --project-endpoint https://<resource>.services.ai.azure.com/api/projects/<project> --training-file train.jsonl --validation-file valid.jsonl --model gpt-4o-mini-2024-07-18"
),
formatter_class=argparse.RawTextHelpFormatter,
)
parser.add_argument(
"--project-endpoint",
required=True,
help="Azure AI Project endpoint URL",
)
parser.add_argument("--training-file", required=True, help="Path to training JSONL file")
parser.add_argument("--validation-file", required=True, help="Path to validation JSONL file")
parser.add_argument("--model", required=True, help="Base model name")
parser.add_argument("--suffix", default="sft-finetuned", help="Model name suffix")
parser.add_argument("--epochs", type=int, default=None, help="Number of epochs")
parser.add_argument("--batch-size", type=int, default=None, help="Batch size")
parser.add_argument("--learning-rate", type=float, default=None, help="Learning rate multiplier")
return parser


if __name__ == "__main__":
parser = build_parser()

# If no arguments are provided, show help instead of submitting with defaults.
if len(sys.argv) == 1:
parser.print_help()
parser.exit(0)

submit_sft(parser.parse_args())
66 changes: 66 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ This repository contains **12 end-to-end demos** and **sample datasets** for fin
- [Quick Start](#-quick-start)
- [Demos](#-demos)
- [Sample Datasets](#-sample-datasets)
- [AI Agent Skills](#-ai-agent-skills)
- [Prerequisites](#-prerequisites)
- [Contributing](#contributing)

Expand Down Expand Up @@ -64,6 +65,71 @@ Ready-to-use datasets for testing fine-tuning techniques in the **[Sample_Datase

> ⚠️ **Note**: These datasets are for **learning and experimentation only**—not for production use. Training jobs may incur costs on your Azure subscription.

---

## 🤖 AI Agent Skills

This repo includes the same fine-tuning skill content for multiple coding agents.

- **GitHub Copilot** skills: **[.github/skills](.github/skills/)**
- **Claude Code** skills: **[.claude/skills](.claude/skills/)**
- **Codex-compatible agents** skills: **[.agents/skills](.agents/skills/)**

The goal is to keep skill guidance **agent-agnostic** so you can use the same workflows (for example, submitting and monitoring fine-tuning jobs) regardless of the assistant.

### How to use with GitHub Copilot + VS Code

1. Open this repository in **VS Code** with the **GitHub Copilot Chat** extension enabled.
2. Ask a task in Copilot Chat from the repo context (for example: "help me submit an SFT job with my dataset").
3. Copilot will match relevant skills from **[.github/skills](.github/skills/)** when your request aligns with a skill description.
4. If available in your version, invoke a skill slash command directly in chat.

### How to use with Copilot CLI

Use `copilot` from the repository root so it can discover workspace skills in **[.github/skills](.github/skills/)**.

1. Verify the CLI is installed:

```bash
copilot --version
```

2. Start a chat session from this repo root:

```bash
cd /path/to/root_of_repo
copilot
```

3. Ask for the same type of task you would ask in chat, for example:

```text
Help me submit an SFT job with my dataset.
```

4. If your CLI version supports slash commands, invoke the skill directly:

```text
/azure-foundry-finetuning
```

5. Follow prompts for Azure project details, model, training file, and validation file, then run the suggested scripts.

### How to use with Claude

1. Open this repository in your Claude coding environment from the repo root.
2. Ask for a fine-tuning workflow task in natural language.
3. Claude can use skills from **[.claude/skills](.claude/skills/)** when your prompt matches the skill scope.
4. Follow the generated steps/scripts and provide required Azure inputs when prompted.

### How to use with Codex

1. Open this repository in a Codex-compatible coding agent environment from the repo root.
2. Ask for the task you want to perform (for example, setting up SFT submission or monitoring a job).
3. The agent can apply skills from **[.agents/skills](.agents/skills/)** when the request matches a skill.
4. Run the suggested commands and scripts with your Azure project configuration.


---

## ✅ Prerequisites
Expand Down