harness/judge-types.ts

atomic-agent · eval/harness/judge-types.ts · typescript · MIT
ai

Defines interfaces for LLM judge client and verdicts.

// Source: https://github.com/AtomicBot-ai/atomic-agent | License: MIT | Copyright 2026 Atomic Bot
// Provided via justclaw.it.com; keep this notice with the code.
/**
 * Contract for the LLM-as-judge layer. The eval harness only depends on
 * `JudgeClient`; the concrete HTTP implementation lives in
 * `judge-client.ts` and can be swapped (e.g. with a fake for unit tests
 * of the harness itself, when we get there).
 *
 * Why a thin interface: the harness is sometimes invoked without a
 * judge (CSV-only run, judge URL unreachable, or `eval:judge` pass over
 * saved data). Code paths that may run without a judge guard with
 * `judge ?? null` and surface a structured failure when a judge
 * expectation is encountered without a client.
 */

export interface JudgeInput {
  /** Verbatim user p

... (truncated -- full source via MCP)

See the full source, get the GitHub permalink, and search for more like it.

Get a free API key