Understand what a document does, not just what it says — rhetorical role mapping, claim quality rating, style evaluation
You read a 3,000-word essay. You can tell it's "pretty good." But can you say why? Can you point to the three strongest claims? Can you tell which evidence actually supports a claim vs. which is decorative? Can you spot where the author sneaks in an undefended assertion between two well-evidenced ones?
Most document tools tell you what a document says: keywords, topics, summaries. Draft tells you what the document does — what rhetorical moves the author makes, how strong the claims are, and where the writing falls short.
Three analysis layers, each independent, each building on the role map as foundation:
Role Map
What does each chunk do?
Claim Rating
How novel & insightful?
Style
How well written?
Review
Multi-lens, multi-model
Every sentence in a document serves a rhetorical function. An LLM reads the full text and classifies each contiguous chunk into one of 12 roles:
| Role | What it does | Example |
|---|---|---|
| claim | Assertion the author wants you to believe | "Nurse scheduling is the binding bottleneck for hospital AI" |
| evidence | Facts, data, statistics supporting a claim | "Revenue grew 15% YoY, driven by a 23% price increase" |
| example | Concrete illustration or case study | "Consider Norway's universal healthcare system..." |
| explanation | Making a concept or mechanism clear | "This works because the incentive structure aligns..." |
| concession | Acknowledging a counterpoint or limitation | "Critics rightly note that correlation is not causation" |
| appeal | Call to action, emotional persuasion | "We cannot afford to wait another decade" |
| context | Background, setup, prior work | "Since the 2008 financial crisis, regulators have..." |
(Plus: definition, description, analogy, qualification, transition — 12 roles total.)
The output is a two-panel interactive view. Left: a sidebar of color-coded cards, one per segment. Right: the full document text with role-colored highlights. Click either side and the other scrolls to match.
The claim about Taiwan's 92% share gets — high novelty (most people don't know the exact number) and high insight (it reframes "supply chain risk" as "single-island dependency"). The TSMC market share evidence gets — well-known fact, moderate insight.
After role extraction, the system can rate every claim, evidence segment, and example on two dimensions:
| Dimension | Scale | What it captures |
|---|---|---|
| Novelty | 1–5 | Is this a fresh observation or restated conventional wisdom? |
| Insight | 1–5 | Does it reveal something non-obvious — a mechanism, implication, or reframing? |
All claims are rated in a single batch prompt so the model can calibrate relative to each other within the document. A claim that's insightful compared to the document's other claims will score higher than the same claim rated in isolation.
Claims scoring 4+ on either dimension get a ★ star in the sidebar. High-rated claims are the diamonds — the observations worth quoting, building on, or challenging.
Evaluates prose quality against ~215 curated principles extracted from canonical style guides (Strunk & White, Zinsser, Pinker, etc.). The key architectural insight: evaluating 1–3 principles per LLM call dramatically outperforms dumping all criteria at once.
Orient
Identify doc type & relevant principles
Select
Pick 15–25 most applicable
Batch Eval
Score 2–3 principles per call
Improve
Rewrite addressing violations
Output: a per-violation report (principle, severity, specific quote, suggested fix) plus an improved version of the text with a changes summary.
The most comprehensive analysis: runs 2–4 evaluation lenses (structure, logic, language, readability), each with multiple models, then synthesizes findings into a unified report. Built on the vario multi-model infrastructure.
draft/ +-- core/ # Pure logic, no UI deps | +-- roles.py # 12-role taxonomy + LLM extraction | +-- mapper.py # Self-contained HTML renderer (sidebar + highlights) | +-- rating.py # Batch novelty/insight scoring +-- style/ # Prose quality evaluation | +-- guides/ # 215 curated principles (YAML) + full text cache | +-- orient.py # Document triage | +-- evaluate.py # Few-principles-at-a-time scoring engine | +-- improve.py # Rewrite engine +-- ui/ | +-- app.py # Gradio server on :7980 (draft.localhost) +-- tests/ +-- tasks.py # inv draft.server, inv draft.map
| Decision | Why |
|---|---|
| Core produces self-contained HTML strings | No UI framework dependency — works from CLI, API, tests, notebooks. The Gradio layer is a thin shell. |
| 12-role taxonomy (not 5, not 50) | Covers the full range of rhetorical moves without requiring expert training to understand. Every educated reader knows what a "claim" and "evidence" are. |
| Batch claim rating (all claims in one call) | The model calibrates relative to the document — "novel compared to what the author already said." Per-claim calls lose this context. |
| Markdown output for ratings | Tam et al.: JSON output kills novelty/insight scores by 17–18%. Parse with regex, not schema. |
| Style eval: 2–3 principles per call | CheckEval/DeCE research: small batches massively outperform monolithic "evaluate everything at once" prompts. |
| Document loading in lib/ingest | PDF, DOCX, HTML, URL, paste — shared across all rivus projects, not duplicated in draft. |
| Component | Source | What Draft Reuses |
|---|---|---|
| LLM calls | lib/llm | call_llm with model aliases, prompt caching |
| Document loading | lib/ingest | PDF/DOCX/HTML/URL parsing with smart escalation |
| Multi-model review | vario.review | Parallel lens evaluation across models |
| UI components | lib/gradio | DocInput widget, emoji favicon, footer |
Generated 2026-03-03 · Live at draft.localhost · Design doc: claim-quality-rating