Fine-Tuning

Overview

The Fine-Tuning API submits a training job to a provider that supports custom training (OpenAI, Together AI, Fireworks, Mistral, Cohere) and exposes the resulting model on the standard /v1/chat/completions endpoint. AllRoutes handles dataset upload, job orchestration, and inference routing -- you don't have to learn each provider's bespoke API.

Fine-tuning works best when you have:

• 50-1000+ examples of the input/output behavior you want
• A consistent system prompt across examples
• Output formats (JSON schemas, tone, brand voice) that are hard to express in prompts

If you can solve the task with prompt engineering and few-shot examples, do that first -- fine-tuning is more expensive and less flexible.

Step 1: Upload Training Data

Training data is a JSONL file, one example per line, in OpenAI chat format:

{"messages": [{"role": "system", "content": "You are a JSON-only assistant."}, {"role": "user", "content": "Cat or dog?"}, {"role": "assistant", "content": "{\"answer\":\"cat\"}"}]}
{"messages": [{"role": "system", "content": "You are a JSON-only assistant."}, {"role": "user", "content": "Apple or pear?"}, {"role": "assistant", "content": "{\"answer\":\"apple\"}"}]}

Upload it via the Files API:

curl https://api.allroutes.ai/v1/files \
  -H "Authorization: Bearer allroutes_sk_..." \
  -F purpose=fine-tune \
  -F file=@training.jsonl

The response includes a file_id (e.g., file_abc123) that the next step needs.

Step 2: Create a Fine-Tuning Job

POST https://api.allroutes.ai/v1/fine_tuning/jobs

Request Body

Field	Type	Required	Description
model	string	Yes	Base model to fine-tune (e.g., gpt-4o-mini-2024-07-18, meta-llama/Llama-3.1-8B-Instruct)
training_file	string	Yes	file_id from Step 1
validation_file	string	No	Optional validation split for early stopping
hyperparameters	object	No	{"n_epochs": 3, "learning_rate_multiplier": 1.0, "batch_size": "auto"}
suffix	string	No	Custom suffix for the resulting model name
seed	integer	No	Reproducible training

Example

curl https://api.allroutes.ai/v1/fine_tuning/jobs \
  -H "Authorization: Bearer allroutes_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini-2024-07-18",
    "training_file": "file_abc123",
    "hyperparameters": {"n_epochs": 3},
    "suffix": "support-bot"
  }'

Response

{
  "id": "ftjob_xyz789",
  "object": "fine_tuning.job",
  "model": "gpt-4o-mini-2024-07-18",
  "training_file": "file_abc123",
  "status": "queued",
  "hyperparameters": {"n_epochs": 3, "batch_size": 4, "learning_rate_multiplier": 1.0},
  "fine_tuned_model": null,
  "created_at": 1714000000,
  "estimated_finish": 1714003600
}

Step 3: Monitor the Job

# Get current status
curl https://api.allroutes.ai/v1/fine_tuning/jobs/ftjob_xyz789 \
  -H "Authorization: Bearer allroutes_sk_..."

# Stream events (SSE)
curl https://api.allroutes.ai/v1/fine_tuning/jobs/ftjob_xyz789/events \
  -H "Authorization: Bearer allroutes_sk_..." \
  -H "Accept: text/event-stream"

Job States

Status	Meaning
queued	Waiting for provider capacity
validating_files	Schema validation in progress
running	Training in progress
succeeded	Done; fine_tuned_model field is populated
failed	Training failed; check error field
cancelled	Cancelled via the API or dashboard

Step 4: Use the Fine-Tuned Model

When the job succeeds, the fine_tuned_model field becomes a regular model identifier you can pass to /v1/chat/completions:

response = client.chat.completions.create(
    model="ft:gpt-4o-mini-2024-07-18:org:support-bot:abc123",
    messages=[{"role": "user", "content": "Where is my order?"}],
)

Fine-tuned models behave like any other model -- streaming, tool calls, plugins, and caching all work.

List, Cancel, Delete

# List jobs
curl https://api.allroutes.ai/v1/fine_tuning/jobs \
  -H "Authorization: Bearer allroutes_sk_..."

# Cancel a running job
curl -X POST https://api.allroutes.ai/v1/fine_tuning/jobs/ftjob_xyz789/cancel \
  -H "Authorization: Bearer allroutes_sk_..."

# Delete the resulting model
curl -X DELETE https://api.allroutes.ai/v1/models/ft:gpt-4o-mini-2024-07-18:org:support-bot:abc123 \
  -H "Authorization: Bearer allroutes_sk_..."

Supported Base Models

Provider	Base Models
OpenAI	gpt-4o-mini-2024-07-18, gpt-4o-2024-08-06, gpt-3.5-turbo-1106
Together AI	Llama 3.1 8B/70B, Mistral 7B, Mixtral 8x7B, Qwen 2 7B
Fireworks	Llama 3.1 8B/70B, Mixtral 8x7B, custom Hugging Face models
Mistral	mistral-small, open-mistral-7b
Cohere	command-r, command-r-plus

Pricing

Fine-tuning is billed by the provider in two parts:

1. Training tokens -- one-time cost based on dataset size × n_epochs
2. Inference tokens -- per-token cost on the resulting model (typically 2-3x the base model rate)

AllRoutes adds the standard platform fee on training and inference, or 0% with BYOK. See Models for the live pricing of each base model.

Best Practices

• Start small -- 100 high-quality examples beat 10,000 noisy ones
• Hold out a validation set -- set validation_file for early stopping and overfit detection
• Iterate the system prompt -- a fine-tuned model still uses the system prompt at inference; keep it identical between training and runtime
• Pin the base model version -- always include the date suffix (e.g., 2024-07-18) so future base-model rotations don't invalidate your tunes
• Use a suffix -- makes the resulting model identifier readable in dashboards and logs

Troubleshooting

Symptom	Likely Cause	Fix
failed: invalid_format	JSONL has malformed line(s)	Re-validate with jq -c < file.jsonl
failed: token_limit_exceeded	Examples exceed the base model's context	Truncate or split long examples
Slow training	Large dataset on busy provider	Try a different provider via the provider field
Model performs worse than base	Overfit; too many epochs	Lower n_epochs to 2 or add validation file

See Also

• Files API -- upload training and validation data
• Chat Completions -- use the resulting model
• Models API -- list base models that support fine-tuning
• BYOK -- 0% commission on training and inference costs