Migration kit
Migrate agents from GPT & Claude to open-weight models.
Token spend is a scaling bottleneck. Transition workloads from proprietary flagships to targeted open-weight models with confidence. Evaluate on the batch tier, shadow on live traffic, and deploy via our OpenAI-compatible gateway.
$ dw batches run batches/candidates.jsonl --watch
[INFO] Validating batch schemas... OK
[INFO] Enqueuing jobs against Qwen3.6-35B, Kimi-K3
» Batch complete. Run `dw batches analytics` to score.
The migration architecture
Replacing a proprietary flagship requires hard data. The Doubleword migration kit enforces a three-step pipeline to guarantee that your open-weight candidate matches production criteria before you flip the switch.
Baseline the flagship
Extract real task traces from your current deployment to create a golden set. Capture the input, the exact JSON output, the latency, and the cost.
If you use Arize Phoenix for tracing, export tasks directly from the logs. The migration workbook includes a 150-ticket support-triage golden set as a pre-labelled, automated scoring example.
query = """
SELECT
input.value as prompt,
output.value as baseline_response,
attributes.llm.latency as latency_ms
FROM spans
WHERE attributes.llm.model_name = 'gpt-5.6'
LIMIT 150
"""
golden_set = client.query(query)
golden_set.to_jsonl("golden_set.jsonl")
Offline evaluation
Run the golden set against candidate models on the Doubleword batch tier. Stamp each model onto its own copy of the batch file using the CLI.
Doubleword enforces schema validity via response_format. Candidates are scored on:
- —Category and escalation accuracy vs. golden labels.
- —Semantic similarity to the baseline, judged via a lightweight LLM-as-a-judge.
# 1. Prepare batch files for candidates
$ dw files prepare golden_set.jsonl \
--model moonshotai/kimi-k3 -o kimi.jsonl
$ dw files prepare golden_set.jsonl \
--model qwen/qwen-3.6-35b -o qwen.jsonl
# 2. Execute on the batch tier (costs cents)
$ dw batches run kimi.jsonl --output-id .kimi-id
$ dw batches run qwen.jsonl --output-id .qwen-id
# 3. Analyze against baseline
$ dw batches analytics --from-file .qwen-id
Evaluation results
150-ticket support triage
| Model | Category acc. | Escalation acc. | Valid JSON | Cost (150 runs) | Avg TTFB |
|---|---|---|---|---|---|
| Claude Fable 5Incumbent | 89.5% | 99.0% | 150/150 | $1.200 | 2.1s |
| Kimi K3 | 89.3% | 94.0% | 149/150 | $0.330 | 4.7s |
| Qwen3.6-35BWinner | 87.3% | 98.0% | 150/150 | $0.005 | 1.7s |
Analysis: Qwen3.6-35B trails the incumbent by only 2.2% on category accuracy, but operates at a fraction of the cost with lower latency. Kimi K3 is over-indexed for this specific extraction task, though it performs competitively on complex reasoning evals. Total evaluation cost for all candidates: ~$0.35.
Shadow the pick
Validate the winner on live traffic before full deployment. Serve the user the incumbent response, but asynchronously fan out the prompt to the open-weight candidate on the Doubleword async tier.
The shadow call runs out-of-band. The user experiences zero latency penalty, while you continuously log and compare the candidate's output against the production baseline.
Synchronous
Claude Fable 5
Returned to user
Async tier
Qwen3.6-35B
Logged for eval
Make the call
If it matches accuracy, meets your latency budget, and costs far less — switch.
Doubleword endpoints are OpenAI-compatible, turning migration into a single-line configuration change.
Enable prompt caching
Keep the harness. A repeated prefix — like a large system prompt — makes cached reads roughly 90% cheaper on supported models.
Right-size by step
Send heavy reasoning workloads to a flagship, and route simple JSON extractions to smaller open models — all on a single API key.