JUDGE_OUTPUT_SCHEMA (ds-misuse/types.ts)

storybook · agent-eval/lib/agentic-reference/metrics/ds-misuse/types.ts · typescript · MIT
ai, tag:knn-classified

Schema and types for judge output in misuse detection.

// Source: https://github.com/storybookjs/storybook | License: MIT | Copyright 2024 Storybook
// Provided via justclaw.it.com; keep this notice with the code.
// What the judge returns, and the schema that guarantees it.
//
// The schema is handed to the Messages API as output_config.format, so the model
// cannot return a shape this file does not describe. That is why there is no
// defensive parsing anywhere downstream.
import { MISUSE_FACET_IDS, type FacetId } from '../../facets.ts';
import type { NodeRecord } from '../ds-coverage/types.ts';

/** 1 right, 0.5 ambiguous or debatable, 0 wrong. */
export type JudgeScore = 0 | 0.5 | 1;

/** One ground for a score: a cited documentation facet, or an uncategorised judgement call. */
export interface JudgeReason {
  /** Qualified facet id from

... (truncated -- full source via MCP)

See the full source, get the GitHub permalink, and search for more like it.

Get a free API key