The Mission
The mission of Legal-10 is to create a definitive, open-source standard for the evaluation of agentic legal competency. As artificial intelligence moves from simple document retrieval to complex, self-directed research, the legal profession requires a governance layer that ensures reasoning is not just fluent, but fundamentally sound.
Research Execution
We define legal competency through the lens of Research Execution. This is an integrated skill set aligned with established professional standards, including:
- I. The ABA MacCrate Report: Emphasizing problem-solving, legal analysis, and research performance.
- II. AALL Research Competencies: Focused on bibliographic control, source selection, and citation integrity.
- III. Shultz-Zedeck Factors: Measuring problem-solving, practical judgement, and diligence.
The Cascade Penalty
"A legal error doesn't just lower a score; it propagates. A wrong citation at step 3 makes the oral argument at step 10 substantively worthless, regardless of its rhetorical beauty."
Legal-10 implements a multiplicative scoring model. In our Agentic evaluation, models lose significant points not just for the error itself, but for every subsequent step that relies on that error. This mirrors the high stakes of real-world litigation.
The 10-Step Chain
| Step | Domain | The Gate |
|---|---|---|
| S1-S3 | Discovery & Intake | Fact Patterns & issue Spotting |
| S4-S6 | Research & Analysis | Jurisdictional Filtering |
| S7-S8 | Synthesis & Cite-Gate | Citation Integrity (Binary) |
| S9-S10 | Advocacy | Strategic Coherence |
Core Values
Transparency
Every prompt, every trace, and every grading rubric is open for public audit. No black boxes.
Observability
We provide full Langfuse traces for model runs, enabling engineers to see exactly where the chain broke.
Independence
Legal-10 is model-agnostic. We serve the law, not the model providers.
Rigorous Truth
Grounded in the Supreme Court Database (SCDB) and verified by actual legal practitioners.