# LLM Latency Benchmark Report

**Generated**: 2026-03-09 12:37:53

## Summary

| Model | TTFT (median) | Total (median) | Tokens/sec | Success |
|-------|---------------|----------------|------------|---------|
| xai/grok-4-1-fast-non-reasoning | 479ms | 4024ms | 87.1 | 100% |
| openai/gpt-5-nano | 539ms | 3170ms | 111.7 | 100% |
| gemini/gemini-3-flash-preview | 820ms | 2564ms | 178.4 | 100% |
| openai/gpt-5-mini | 1055ms | 5847ms | 66.8 | 100% |
| anthropic/claude-haiku-4-5-20251001 | 1221ms | 4619ms | 94.1 | 100% |
| gemini/gemini-3.1-flash-lite-preview | 1463ms | 2124ms | 383.8 | 100% |

## Detailed Results

### xai/grok-4-1-fast-non-reasoning

**TTFT (Time to First Token)**
- Min: 358ms
- Max: 2163ms
- Mean: 1000ms
- Median: 479ms
- Stdev: 1009ms

**Total Response Time**
- Min: 3803ms
- Max: 5062ms
- Mean: 4296ms
- Median: 4024ms
- Stdev: 672ms

**Individual Runs**

- Run 1: TTFT=479ms, Total=4024ms, Tokens=300, 84.6 tok/s
- Run 2: TTFT=358ms, Total=3803ms, Tokens=300, 87.1 tok/s
- Run 3: TTFT=2163ms, Total=5062ms, Tokens=300, 103.5 tok/s

### openai/gpt-5-nano

**TTFT (Time to First Token)**
- Min: 368ms
- Max: 543ms
- Mean: 483ms
- Median: 539ms
- Stdev: 100ms

**Total Response Time**
- Min: 2972ms
- Max: 3230ms
- Mean: 3124ms
- Median: 3170ms
- Stdev: 135ms

**Individual Runs**

- Run 1: TTFT=543ms, Total=3230ms, Tokens=300, 111.7 tok/s
- Run 2: TTFT=539ms, Total=2972ms, Tokens=300, 123.3 tok/s
- Run 3: TTFT=368ms, Total=3170ms, Tokens=300, 107.0 tok/s

### gemini/gemini-3-flash-preview

**TTFT (Time to First Token)**
- Min: 788ms
- Max: 1000ms
- Mean: 869ms
- Median: 820ms
- Stdev: 115ms

**Total Response Time**
- Min: 2479ms
- Max: 2618ms
- Mean: 2554ms
- Median: 2564ms
- Stdev: 70ms

**Individual Runs**

- Run 1: TTFT=1000ms, Total=2564ms, Tokens=296, 189.3 tok/s
- Run 2: TTFT=788ms, Total=2618ms, Tokens=296, 161.7 tok/s
- Run 3: TTFT=820ms, Total=2479ms, Tokens=296, 178.4 tok/s

### openai/gpt-5-mini

**TTFT (Time to First Token)**
- Min: 916ms
- Max: 1985ms
- Mean: 1319ms
- Median: 1055ms
- Stdev: 582ms

**Total Response Time**
- Min: 4721ms
- Max: 6474ms
- Mean: 5681ms
- Median: 5847ms
- Stdev: 888ms

**Individual Runs**

- Run 1: TTFT=1055ms, Total=5847ms, Tokens=300, 62.6 tok/s
- Run 2: TTFT=916ms, Total=4721ms, Tokens=300, 78.8 tok/s
- Run 3: TTFT=1985ms, Total=6474ms, Tokens=300, 66.8 tok/s

### anthropic/claude-haiku-4-5-20251001

**TTFT (Time to First Token)**
- Min: 945ms
- Max: 1571ms
- Mean: 1246ms
- Median: 1221ms
- Stdev: 313ms

**Total Response Time**
- Min: 4303ms
- Max: 4758ms
- Mean: 4560ms
- Median: 4619ms
- Stdev: 233ms

**Individual Runs**

- Run 1: TTFT=1571ms, Total=4758ms, Tokens=300, 94.1 tok/s
- Run 2: TTFT=945ms, Total=4619ms, Tokens=300, 81.7 tok/s
- Run 3: TTFT=1221ms, Total=4303ms, Tokens=300, 97.3 tok/s

### gemini/gemini-3.1-flash-lite-preview

**TTFT (Time to First Token)**
- Min: 1327ms
- Max: 1542ms
- Mean: 1444ms
- Median: 1463ms
- Stdev: 109ms

**Total Response Time**
- Min: 2121ms
- Max: 2314ms
- Mean: 2186ms
- Median: 2124ms
- Stdev: 110ms

**Individual Runs**

- Run 1: TTFT=1542ms, Total=2314ms, Tokens=296, 383.8 tok/s
- Run 2: TTFT=1327ms, Total=2124ms, Tokens=296, 371.7 tok/s
- Run 3: TTFT=1463ms, Total=2121ms, Tokens=296, 449.7 tok/s

## Configuration

- **Prompt**: "Explain how a CPU cache works in 3 paragraphs."
- **Max Tokens**: 300
- **Timeout**: 60s