Assistant Reasoning Relay (chatmodel/AssistantThinkingRelay.java)

mateclaw · mateclaw-server/src/main/java/vip/mate/llm/chatmodel/AssistantThinkingRelay.java · java · Apache-2.0
ai, lang:java, tag:reactor, tag:reasoning, tag:concurrency

Relays per-request assistant reasoning_content from producer to consumer using a request-embedded token, avoiding ThreadLocal loss across Reactor scheduler hops. Reuse when threading reasoning metadata through reactive LLM pipelines.

// Source: https://github.com/mateaix/mateclaw | License: Apache-2.0 | Copyright You must retain, in the Source form of any Derivative Works
// Provided via justclaw.it.com; keep this notice with the code.
package vip.mate.llm.chatmodel;

import java.util.List;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;

/**
 * Relays per-request assistant {@code reasoning_content} from the producer
 * ({@code NodeStreamingChatHelper}, which sees {@code AssistantMessage.metadata})
 * to the consumer ({@link OpenAiRequestRewriter#patchReasoningContent}, which
 * rebuilds the outbound {@code ChatCompletionRequest}).
 *
 * <p>Why not {@link ThreadLocal}: {@code OpenAiChatModel.stream()} hops to
 * {@code boundedElastic} via {@code subscribeOn}, so a {@code ThreadLocal} on the
 * cal

... (truncated -- full source via MCP)

See the full source, get the GitHub permalink, and search for more like it.

Get a free API key