You’re probably overpaying for AI by 2x. OpenAI, Anthropic, and Google all offer 50% off their standard API pricing if you can wait 24 hours for results. Most people don’t know this exists. The ones who do save thousands.
Here’s the complete guide to batch APIs — when to use them, how to set them up, and a calculator to see your actual savings.
The Problem
Everyone uses synchronous APIs. You send a request, wait a few seconds, get a response. Simple. But if you’re processing thousands of documents, classifying support tickets, generating embeddings, or running nightly reports — you’re paying full price for no reason.
Batch APIs exist for exactly this. Same models, same quality, half the cost. The trade-off: results take up to 24 hours instead of seconds. For background jobs, that’s usually fine.
The Solution
What Are Batch APIs and When Should You Use Them?
The Trade-off: 50% Off for 24-Hour Turnaround
| Factor | Synchronous API | Batch API |
|——–|—————–|———–|
| Cost | Full price | 50% discount |
| Latency | Seconds | Up to 24 hours |
| Rate limits | Standard | Higher (10x+) |
| Complexity | Simple request/response | File upload, poll, download |
Perfect Use Cases (Use Batch):
- Daily/weekly report generation
- Bulk document classification or tagging
- Embedding generation for RAG/vector databases
- Processing large datasets overnight
- Non-urgent data enrichment
- Scheduled content analysis
Avoid Batch When:
- Real-time user interactions (chat, search, UX)
- Live decision-making
- Anything needing sub-minute response
- Low volume (under ~100 requests/day — overhead not worth it)
Provider-by-Provider: Pricing, Setup, and Limits
OpenAI Batch API — 50% Off GPT-4o, GPT-4o-mini, More
Models Supported: GPT-4o, GPT-4o-mini, GPT-4-turbo, GPT-3.5-turbo
Discount: 50% off standard pricing
Turnaround: Up to 24 hours
Rate Limits: Much higher than sync
Pricing Example (GPT-4o-mini):
- Standard: $0.15/1M input, $0.60/1M output
- Batch: $0.075/1M input, $0.30/1M output
- 10M tokens = save ~$5.25
Setup: JSONL file → Upload → Create batch → Poll status → Download results
Anthropic Message Batches — 50% Off Claude 3.5 Sonnet, Haiku
Models Supported: Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3 Opus
Discount: 50% off standard pricing
Turnaround: Up to 24 hours
Endpoint: /v1/messages/batches
Pricing Example (Claude 3.5 Sonnet):
- Standard: $3/1M input, $15/1M output
- Batch: $1.50/1M input, $7.50/1M output
- 10M tokens = save ~$105
Google Gemini Batch API — 50% Off Flash and Pro
Models Supported: Gemini 1.5 Flash, Gemini 1.5 Pro
Discount: 50% off standard pricing
Note: Gemini 3.6/3.7 Flash intro pricing $0.75/$3.75 until Dec 31, then $1.50/$7.50. Batch = 50% of whatever the current rate is.
Pricing Example (Gemini 1.5 Flash approx):
- Standard: ~$0.075/1M input, $0.30/1M output
- Batch: ~$0.0375/1M input, $0.15/1M output
- 10M tokens = save ~$2.25
xAI Grok — No Batch Discount
Confirmed: Grok 4.6 has no batch pricing. Full price only. Factor this into provider decisions.
Step-by-Step: Set Up Your First Batch Job (OpenAI Example)
1. Prepare Your JSONL File
Each line = one request. Required fields: custom_id, method, url, body.
{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Classify this ticket: ..."}], "max_tokens": 100}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Classify this ticket: ..."}], "max_tokens": 100}}Save as batch_requests.jsonl. Up to 50,000 requests per batch.
2. Upload and Create the Batch
import openai
client = openai.OpenAI()
# Upload file
batch_file = client.files.create(
file=open("batch_requests.jsonl", "rb"),
purpose="batch"
)
# Create batch
batch = client.batches.create(
input_file_id=batch_file.id,
endpoint="/v1/chat/completions",
completion_window="24h"
)
print(f"Batch ID: {batch.id}")
print(f"Status: {batch.status}")
3. Poll for Completion
import time
while True:
batch = client.batches.retrieve(batch.id)
print(f"Status: {batch.status}")
if batch.status in ["completed", "failed", "expired", "cancelled"]:
break
time.sleep(60) # Check every minute
4. Download and Parse Results
if batch.status == "completed":
result_file = client.files.content(batch.output_file_id)
# Save results
with open("batch_results.jsonl", "wb") as f:
f.write(result_file.read())
# Parse each line
for line in open("batch_results.jsonl"):
result = json.loads(line)
custom_id = result["custom_id"]
response = result["response"]["body"]
# Handle your resultsCost Calculator: How Much Could You Save?
| Monthly Volume | Provider | Standard Cost | Batch Cost | Monthly Savings |
|—————-|———-|—————|————|—————–|
| 1M tokens | OpenAI GPT-4o-mini | $3.75 | $1.88 | $1.87 |
| 10M tokens | OpenAI GPT-4o-mini | $37.50 | $18.75 | $18.75 |
| 1M tokens | Anthropic Sonnet 3.5 | $90 | $45 | $45 |
| 10M tokens | Anthropic Sonnet 3.5 | $900 | $450 | $450 |
| 1M tokens | Gemini 1.5 Flash | ~$1.88 | ~$0.94 | ~$0.94 |
Quick formula: Monthly tokens / 1,000,000 × (standard_price_per_M × 0.5) = monthly savings
Pro Tips: Hybrid Workflows, Monitoring, and Common Mistakes
Hybrid Workflow: Route urgent requests to sync API, batch the rest. Example: User-facing chat = sync. Nightly report generation = batch. Same model, half the bill.
Monitoring: Set up alerts for failed batches. Check error_file_id if batch fails. Common errors: malformed JSONL, rate limit on file upload, token limits exceeded.
Common Mistakes:
- Using batch for real-time features (defeats the purpose)
- Not validating JSONL before upload (wastes 24h on failures)
- Forgetting to download results before they expire (30 days)
- Sending tiny batches (under 100 requests — overhead not worth it)
Idempotency: Use custom_id to match requests to results. Design your system to handle re-processing if needed.
Takeaway
Audit your AI spend this week. Find every workload that doesn’t need seconds-level response. Move those to batch APIs. You’ll cut those costs in half immediately. The setup takes an afternoon. The savings compound forever.
