An LLM council is a group of large language models that each answer a question independently, then peer-rate one another's responses to arrive at a consensus. The pattern was popularized by Andrej Karpathy's open-source 'llm-council' repository, a concept-proof in which several models answer and then rank each other before a final synthesis. Argumentree.AI extends the council pattern with structured argument trees and anonymized mutual rating—anonymized because research on using an LLM as a judge documents a self-enhancement bias in which models favor their own outputs. A hosted LLM council lets you query multiple models without supplying your own API keys. The honest trade-off is cost and latency: a council makes multiple model calls, so it is slower and more expensive than a single query, in exchange for cross-model validation and visible disagreement.
An LLM council has several language models answer independently, peer-rate each other, and reach a consensus. It turns "one confident answer" into a deliberation you can inspect.
An LLM council is multiple models answering independently, then rating each other to form consensus. The pattern comes from Karpathy's open-source llm-council. Argumentree.AI hosts it—no API keys needed—and rates anonymously to curb self-flattery.
The council pattern is simple to state: multiple LLMs answer the same question independently, then evaluate one another's answers, and a consensus is drawn from the peer ratings. No single model gets the last word. Because the models don't see each other's responses before answering, their independent reasoning is preserved—and the places where they diverge become visible instead of being hidden behind one confident reply.
Every model answers the question on its own, without seeing the others
Each model evaluates the other models' answers
Ratings are aggregated to reveal where the council agrees and disagrees
Andrej Karpathy published an open-source "llm-council" repository demonstrating exactly this flow—several models answer, rank each other, and a final response is synthesized. It's a public proof that models can meaningfully evaluate one another. Argumentree.AI builds on the same idea.
Argumentree.AI takes the council pattern further in two ways. First, instead of comparing whole answers, it structures each model's reasoning into an argument tree—so individual claims can be rated and inspected, not just the final paragraph. Second, it uses anonymized mutual rating: when a model rates arguments, it doesn't know which model authored them.
Research on using an LLM as a judge documents a self-enhancement bias—models tend to rate their own outputs more favorably. Hiding authorship before rating removes the shortcut, so a model can't simply reward itself. That's the design rationale for anonymized cross-rating.
Running a council yourself usually means signing up with each model provider and wiring in separate API keys. Argumentree.AI offers a hosted LLM council: you ask once, and all available models participate—no keys to manage, no per-provider setup. You get the council's consensus and its disagreements in one place.
A council is not free. By definition it makes multiple model calls, so it uses more compute and takes longer to return than a single query. That's the deliberate trade: you spend more to gain cross-model validation and a visible map of where the models disagree. For high-stakes research that's a good deal; for a trivial lookup, one model is plenty.
Convene a council of several models with a single question
Inspect each model's reasoning point by point, not as a blob
Cross-rating hides authorship to curb self-enhancement bias
No per-provider API keys—consensus and dissent out of the box
An LLM council is a group of large language models that each answer a question independently, then peer-rate one another's responses to produce a consensus view. Instead of relying on a single model, the council makes agreement and disagreement between models explicit—so you can see where the models converge and where they don't.
Andrej Karpathy released an open-source 'llm-council' repository demonstrating the pattern: several models answer a query, then rank each other's answers before a final synthesis. It's a concept-proof of collective reasoning across models. Argumentree.AI extends the idea with structured argument trees and anonymized mutual rating.
Research on using an LLM as a judge has documented a self-enhancement bias—models tend to favor their own outputs. Anonymizing which model produced which argument before rating reduces that bias, so a model can't simply reward itself. This is the design rationale behind Argumentree.AI's anonymized cross-rating.
Not with a hosted LLM council. Open-source implementations typically require you to supply API keys for each provider. Argumentree.AI offers a hosted multi-LLM council so you can query several models and see their consensus without wiring up individual API keys yourself.
Honestly, running a council costs more than a single query—it makes multiple model calls, so it uses more compute and takes longer to return. That trade-off buys you cross-model validation and a visible map of disagreement. For high-stakes questions the added cost and latency are usually worth it; for trivial lookups a single model is fine.
Several models, one question, consensus you can inspect—no API keys required.
Start Free