Tagged “evals”

1. Link · Oct 7, 2026 · mcp evals open-source

   intake-eval-harness: an evaluation harness for tool-using MCP servers

   A small evaluation harness for tool-using MCP servers: run a fixed suite against a server, score the final answer, and enforce deterministic assertions over the tool trace. Tool-using systems fail in ways a unit test does not catch (the model picks the wrong tool, skips a required step, or answers without calling anything), so a fixed suite turns those failures into a repeatable number instead of a surprise in production. Extracted from intake-triage-mcp.

   permalink

   Edited with AI assistance; the ideas and words are mine.

2. Post · Jul 2, 2026 · legal-tech mcp intake deterministic evals

   Anatomy of a legal intake automation

   A concrete teardown of a legal intake workflow, its deterministic tools, human gates, and evaluation strategy.

All tags