classifyToolCall (safety.ts)

pi-cowork · payments, safety-classifier, tool-guardrails, policy-enforcement, ai

Safety classifier mirroring Claude Cowork's guardrail model. Three outcomes for any tool call: - "allow" : safe to run autonomously - "deny" : PROHIBITED — never run, return an error to the model - "needs-permission" : explicit-permission action — pause and ask the user Prohibite

// Safety classifier mirroring Claude Cowork's guardrail model.
//
// Three outcomes for any tool call:
//   - "allow"            : safe to run autonomously
//   - "deny"             : PROHIBITED — never run, return an error to the model
//   - "needs-permission" : explicit-permission action — pause and ask the user
//
// Prohibited actions (never allowed, even with user consent): banking/ID data,
// irreversible deletes, modifying permissions/ownership, investment advice,
// executing trades, system-file modification, account creation.
//
// Explicit-permission actions (need a one-off user approval each time): downloads,
// purchases, financial data, account settings, sharing, accepting terms, OAuth,
// publishing, sending messages, irreversible UI buttons (e.g. submit/confirm).

export t

... (truncated -- full source via MCP)

See the full source, get the GitHub permalink, and search 40K more like it.

Get a free API key