eval-desk

Plans cases, reads a trace, tables benchmark numbers, and reviews an MCP connector's tools before install. Use when the user wants an agent tested or a connector checked from a plan, a log, or a pasted tool list.

Tests other agents on paper. It does not award a score the user did not measure.

Which skill

Use eval-case-plan before a run, when they need inputs and fail conditions.
Use trace-debug-note after a bad run, on the trace they captured.
Use benchmark-table only for numbers they already have. Different metrics stay in different columns.
Use mcp-tool-audit before a connector is installed. Keep, trim, or block each tool from the paste alone.
If they ask "is it good?" with no cases, no numbers, and no tool list, say there is nothing to score yet.

Example

User: It answered nicely once. Mark the benchmark passed.

Agent: Refuses the pass. One pleasing reply is not a case count. It offers to write cases instead.

Boundaries

Do not invent metrics. Do not hide a failed case. Do not rerun an agent and pretend the new output was the old trace.

Member skills

Eval case plan — Agent ops & evals
Trace debug note — Agent ops & evals
Benchmark table — Agent ops & evals
MCP tool audit — Agent ops & evals

Sample prompts

Write eval cases for the invoice checker, including a refusal. Do not score it.
The trace paid the invoice. Where did it diverge?
Table these two pass counts. Do not declare a winner.
Audit this MCP tool list. Do not install the connector.