# LLM Latency Benchmark Report

**Generated**: 2026-01-27 05:01:05

## Summary

| Model | TTFT (median) | Total (median) | Success Rate |
|-------|---------------|----------------|--------------|
| xai/grok-4-1-fast-non-reasoning | 633ms | 868ms | 100% |
| gemini/gemini-3-flash-preview | 843ms | 866ms | 100% |
| anthropic/claude-haiku-4-5-20251001 | 926ms | 1386ms | 100% |
| openai/gpt-5-mini | N/A | N/A | 0% |

## Detailed Results

### xai/grok-4-1-fast-non-reasoning

**TTFT (Time to First Token)**
- Min: 373ms
- Max: 893ms
- Mean: 633ms
- Median: 633ms
- Stdev: 368ms

**Total Response Time**
- Min: 624ms
- Max: 1112ms
- Mean: 868ms
- Median: 868ms
- Stdev: 345ms

**Individual Runs**

- Run 1: TTFT=893ms, Total=1112ms, Tokens=20
- Run 2: TTFT=373ms, Total=624ms, Tokens=20

### gemini/gemini-3-flash-preview

**TTFT (Time to First Token)**
- Min: 840ms
- Max: 845ms
- Mean: 843ms
- Median: 843ms
- Stdev: 4ms

**Total Response Time**
- Min: 858ms
- Max: 874ms
- Mean: 866ms
- Median: 866ms
- Stdev: 11ms

**Individual Runs**

- Run 1: TTFT=840ms, Total=858ms, Tokens=2
- Run 2: TTFT=845ms, Total=874ms, Tokens=2

### anthropic/claude-haiku-4-5-20251001

**TTFT (Time to First Token)**
- Min: 516ms
- Max: 1336ms
- Mean: 926ms
- Median: 926ms
- Stdev: 579ms

**Total Response Time**
- Min: 983ms
- Max: 1788ms
- Mean: 1386ms
- Median: 1386ms
- Stdev: 569ms

**Individual Runs**

- Run 1: TTFT=1336ms, Total=1788ms, Tokens=18
- Run 2: TTFT=516ms, Total=983ms, Tokens=19

### openai/gpt-5-mini

**All runs failed**

**Individual Runs**

- Run 1: FAILED - litellm.UnsupportedParamsError: gpt-5 models (including gpt-5-codex) don't support temperature=0.7. Only temperature=1 is supported. For gpt-5.1, temperature is supported when reasoning_effort='none' 
- Run 2: FAILED - litellm.UnsupportedParamsError: gpt-5 models (including gpt-5-codex) don't support temperature=0.7. Only temperature=1 is supported. For gpt-5.1, temperature is supported when reasoning_effort='none' 

## Configuration

- **Prompt**: "Write a haiku about programming."
- **Max Tokens**: 100
- **Timeout**: 60s