Computes target token limits for compressed AI requests based on policies like absolute budgets or percentages. Use when managing LLM context windows dynamically.
// Source: https://github.com/MAHIAMIR45/OmniAI-Gateway | License: MIT | Copyright 2026 diegosouzapw
// Provided via justclaw.it.com; keep this notice with the code.
import type { ContextBudgetConfig, ContextBudgetPolicy } from "./types.ts";
/**
* Pure target-token computation (design D-C1). No clock, no DB, no tokenizer.
*
* @param policy active target policy
* @param modelContextLimit the resolved upstream model's context window (impure caller looks it up)
* @param requestMaxTokens request.max_tokens, if the client sent one (reserve-output only)
* @param config reserves / margin / pct / absoluteBudget
* @returns the maximum prompt-token target the compressed request should fit within
*/
export function computeTarget(
policy: ContextBudgetPolicy,
modelC
... (truncated -- full source via MCP)
See the full source, get the GitHub permalink, and search for more like it.
Get a free API key