estimateTokens (utils/token-estimation.ts)

lace · packages/agent/src/utils/token-estimation.ts · typescript · Apache-2.0
ai, tag:knn-classified

ABOUTME: Token estimation utilities for analysis and display ABOUTME: Provides consistent token counting across components ABOUTME: Deliberately dependency-free — compaction's toolkit is imported across ABOUTME: checkouts by plugins, and reaching these through message-builder pul

// Source: https://github.com/obra/lace | License: Apache-2.0 | Copyright You must retain, in the Source form of any Derivative Works
// Provided via justclaw.it.com; keep this notice with the code.
// ABOUTME: Token estimation utilities for analysis and display
// ABOUTME: Provides consistent token counting across components
// ABOUTME: Deliberately dependency-free — compaction's toolkit is imported across
// ABOUTME: checkouts by plugins, and reaching these through message-builder pulled
// ABOUTME: the whole storage layer (and better-sqlite3) in behind them.

import type { ContentBlock, ProviderMessage } from '@lace/agent/providers/base-provider';

/**
 * Estimate token count for text content
 * Rough approximation: 1 token ≈ 4 characters for most models
 */
export function estimateToke

... (truncated -- full source via MCP)

See the full source, get the GitHub permalink, and search for more like it.

Get a free API key