Vario

By thinking 2–3× as much about a problem — generating alternatives, evaluating from more angles, building up background — you can almost always improve on a first-pass answer. Vario makes that effort computational, not human.

Ask an LLM a hard question and it generates one answer, immediately, with no preparation. It doesn’t build a rubric for what “good” means. It doesn’t look for precedent. It doesn’t consider alternative framings. It doesn’t check its own work.

These aren’t exotic techniques — they’re what any thoughtful person does on a question that matters. The gap between a quick take and a considered one is real and valuable. Vario exists to close that gap automatically.

Not 19 Strategies — A Language for Thinking

Vario is not “19 strategies that compete.” It’s a composable language for expressing how to think about a problem — and a learning system that discovers which thinking moves work for which situations.

The 19 current strategies (majority_vote, debate, tree_search, etc.) are specific instantiations, but the real value is the primitives they’re built from and the learning that accumulates across runs. A strategy is just a composition of moves. The moves are the atoms:

A strategy like debate is really: Generate × 3 → Critique (mutual) → Synthesize (strongest arguments). tree_search is: Generate → Verify → Decompose (expand promising branches) → Synthesize. The current YAML strategies hard-wire these compositions — the goal is to make the composition itself learnable.

Two Levels of Learning

Vario learns at two levels, each feeding the other:

The move library (SQLite) currently tracks strategy × situation win/loss. The future is tracking move × context × outcome, so the system can compose novel strategies from primitives rather than selecting from a fixed menu.

Why Not Just Prompt the LLM?

Fair question. A Claude Code session with tool calling already does “generate hypothesis, run code, see result, iterate.” So what do recipes add?

The honest answer: for one-shot interactive tasks, just prompt the LLM. Recipes earn their keep when you need parallelism, multiple models, cost control, or — most importantly — when you want to learn what works across many runs.

The Execute Block — Real-World Grounding

The killer feature isn’t “better prompting” — it’s the execute block. It bridges the LLM pipeline to real-world evaluation:

• Trading strategies: LLM generates config variants → execute block runs walk-forward backtest → real Spearman correlations come back
• Code: LLM generates implementations → execute block compiles and runs tests → actual pass/fail
• Visual/UX: LLM proposes layout changes → execute block takes screenshots → real pixel comparisons
• CEO assessment: LLM generates scoring rubrics → execute block correlates against historical stock returns

Without the execute block, everything is LLM-judging-LLM — turtles all the way down. With it, the feedback loop is grounded in reality. The LLM generates hypotheses; the real world evaluates them.

The Compounding Advantage

Parallelism and multi-model are one-time efficiency gains. Learning what works is cumulative.

With structured recipes + tracked results, you can:

• A/B test recipes — run debate vs best_of_n vs self_refine on the same problem set, measure which wins
• Build a track record — “debate wins 70% on creative tasks, best_of_n wins 85% on analytical tasks”
• Evolve recipes — the route block gets better at picking recipes because it has data
• Discover patterns — “adding a score step before synthesis improves scores by 12% on average”
• Tune params — “n=5 is better than n=3 for best_of_n, but n=3 is enough for debate”

Raw prompting and tool calling are stateless — every session starts from zero. Recipes are stateful across runs — each run contributes to the knowledge of what works. That’s the real moat.

Extensible by Design

One way or another, you need to capture how to do something like score — what rubric, which model, how harsh, whether to give feedback or just a number. A prompt buries these choices inside a wall of text. A recipe makes them explicit, tunable, and composable.

This matters because the architecture is open to new processing types. Today we have 8 block types (produce, score, verify, revise, reduce, execute, enrich, loop). Adding a new one is:

1. Write an async Python function with the standard (candidates, params, context) signature
2. Register it in BLOCK_REGISTRY
3. Use it in any recipe YAML immediately

No framework changes, no schema migration, no executor rewrite. The same extensibility applies to how existing blocks work — a score block’s behavior is fully determined by its params (rubric, model, feedback mode), all visible in the YAML, all tunable per-recipe.

This is the architectural bet: blocks are cheap to add, recipes are cheap to compose, and the learning system discovers which compositions work. The vocabulary of thinking moves grows over time without requiring redesign.

The Steer Block — Adaptive Orchestration

Fixed pipelines always run the same steps regardless of how the problem is going. best_of_n generates 5 candidates even if the first one scored 98. self_refine runs 3 rounds even if the answer stopped improving after 1. The pipeline doesn’t watch what’s happening.

The steer block replaces a fixed stage list with a decision loop. Given an action space of available blocks, it observes the current state — candidates, scores, budget remaining, what’s been tried — and picks what to do next:

With steer, a recipe collapses to: steer(action_space=[produce, score, revise, execute, reduce]). That’s it. Produce is just another action — steer can request more candidates at any point, with different params (wider search, different model, different lens). The steer block is the recipe — everything else is its action space.

The Research Says This Works

Recent work validates every piece of vario’s approach — and reveals the gap vario fills:

Full survey with 30+ papers: meta_llm_strategies.html

Polya’s Framework, Computationally

Vario implements a problem-solving structure humans have used since Polya wrote it down in 1945 — and that LLMs skip entirely. The four phases map directly to vario’s stage types:

Polya considered step 1 the most important and the most neglected. Eighty years later, LLMs have the same blind spot. The expand stage exists to close it: think about what to think about, before generating any answers.

Architecture

Concrete Example: Investment Thesis Evaluation

When a user says “evaluate this investment thesis,” a sophisticated vario session composes moves from its library:

1. Gather: “What are the key claims? Let me extract them first.”
2. Frame: “Each claim needs different analysis — market size needs data, competitive moat needs adversarial thinking, team quality needs precedent matching.”
3. Generate: Run market-size through search-augmented models; moat through adversarial lens; team through precedent-matching.
4. Verify: Cross-check market numbers against multiple sources.
5. Critique: “What’s the strongest bear case? What assumption would kill the thesis?”
6. Hedge: “What are we most uncertain about? Where would more information change the conclusion?”
7. Synthesize: Merge into structured assessment with confidence levels per claim.

This is not a single “strategy” — it’s a plan composed from moves, where the plan itself was chosen based on problem analysis.

Compute Tiers

Not every question needs a 10-stage pipeline. The research is clear: most quality gain comes early. Vario lets you dial compute to the problem.

Key finding: self-consistency (sample N, majority vote) is the single highest-ROI LLM strategy — +18% on GSM8K at 5× cost. (Wang et al. 2023)

Composable Building Blocks

Everything in Vario composes at three levels:

Example: rubric-first evaluation

Example: iterative optimization with real-world feedback

Example: composition — recipes reference other recipes

Where It Targets

Vario is most valuable where a first draft is easy but a good answer requires judgment:

Theoretical Grounding

Vario doesn’t invent a new theory. It implements established ones computationally.

The Arc

Key Components

Blocks — Atomic operations: produce, score, execute, revise, reduce, verify, enrich, loop. Each is an async Python function that can call LLMs, tools, APIs, or spawn sessions.

Recipes — YAML compositions of blocks. Named, parameterized with $variables, composable (a recipe can reference other recipes via type:). Built-in library + user-defined.

Executor — Pipeline engine with loop/convergence detection, budget tracking, soft landing on budget exhaustion, and structured traces for every stage.

Execute block — Bridges to real-world evaluation: backtests, code execution, screenshots, external APIs. The grounding that makes feedback loops real.

Move Library — SQLite-backed record of which recipes work for which problem types. Adaptive priority tables evolve with experience.

vario CLI — Recipe runner. Takes NL or YAML, runs through the executor, returns results. Also usable as a tool from Claude Code sessions.

Try It

The fastest way to see the difference: take a question you care about and run it two ways.

• Named recipe: vario best_of_n "your question" — produce 5, score, pick best
• From NL: vario parse "debate with 3 strong models" — generates runnable recipe YAML
• With real-world eval: vario recipe.yaml "optimize" --handler my.evaluator — execute block bridges to your code
• List available: vario list — show all built-in recipes

Good test questions: anything where the first answer feels “fine” but you suspect there’s a better one. Evaluations, estimates, explanations, decisions with trade-offs. Or anything with a measurable objective — that’s where the execute block shines.

Open Vario Studio → vario.localhost

Full README (single source of truth)  ·  Related work survey (30+ papers)  ·  Benchmark data