Model Settings

Model settings control how the model generates responses. You set them on the agent at construction time.

Setting on an agent

Pass a modelSettings object when creating an agent:

import { Agent } from "@usestratus/sdk/core";

const agent = new Agent({
  name: "assistant",
  model,
  modelSettings: {
    temperature: 0.7,
    maxTokens: 1000,
  },
});

Settings are sent to the model on every call the agent makes. To change settings between runs, clone the agent with new values:

const creativeAgent = agent.clone({
  modelSettings: { temperature: 1.2, topP: 0.95 },
});

ModelSettings reference

Reasoning models

For reasoning models (o1, o3, etc.), use reasoningEffort and maxCompletionTokens instead of temperature and maxTokens.

reasoningEffort controls how much internal reasoning the model does before responding. Higher effort produces more thorough answers but uses more tokens and takes longer.

import { Agent } from "@usestratus/sdk/core";

const agent = new Agent({
  name: "analyst",
  model,
  modelSettings: {
    reasoningEffort: "high", 
    maxCompletionTokens: 16384, 
  },
});

Valid values for reasoningEffort:

Prompt caching

Azure automatically caches prompt prefixes for requests over 1,024 tokens. Use promptCacheKey to improve cache hit rates when many requests share long common prefixes.

const agent = new Agent({
  name: "assistant",
  model,
  modelSettings: {
    promptCacheKey: "support-agent-v2", 
  },
});

Cache hits appear as cacheReadTokens in UsageInfo and are billed at a discount. No opt-in is needed for basic caching — promptCacheKey is only for improving hit rates across requests with shared prefixes.

Tool choice

The toolChoice setting controls whether and how the model calls tools. Set it inside modelSettings.

const agent = new Agent({
  name: "assistant",
  model,
  tools: [getWeather],
  modelSettings: {
    toolChoice: "auto", 
  },
});

const agent = new Agent({
  name: "assistant",
  model,
  tools: [getWeather, searchDocs],
  modelSettings: {
    toolChoice: "required", 
  },
});

const agent = new Agent({
  name: "assistant",
  model,
  tools: [getWeather],
  modelSettings: {
    toolChoice: "none", 
  },
});

const agent = new Agent({
  name: "assistant",
  model,
  tools: [getWeather, searchDocs],
  modelSettings: {
    toolChoice: { 
      type: "function", 
      function: { name: "get_weather" }, 
    }, 
  },
});

Tool use behavior

toolUseBehavior is separate from modelSettings. It is set directly on the agent and controls what happens after a tool executes -- not what the model generates.

const agent = new Agent({
  name: "assistant",
  model,
  tools: [getWeather],
  toolUseBehavior: "run_llm_again", 
});

const agent = new Agent({
  name: "data-fetcher",
  model,
  tools: [fetchData],
  toolUseBehavior: "stop_on_first_tool", 
});

const result = await run(agent, "Get the latest sales data");
// result.output is the return value of fetchData

const agent = new Agent({
  name: "researcher",
  model,
  tools: [searchDocs, summarize, finalAnswer],
  toolUseBehavior: { 
    stopAtToolNames: ["final_answer"], 
  }, 
});

const agent = new Agent({
  name: "researcher",
  model,
  tools: [searchDocs, summarize, finalAnswer],
  toolUseBehavior: (toolResults) => { 
    return toolResults.some((r) => r.toolName === "final_answer"); 
  }, 
});

toolUseBehavior: async (toolResults) => {
  const shouldStop = await checkCompletion(toolResults);
  return shouldStop;
},

Response format

Structured output is configured via outputType on the agent, not through modelSettings directly. When you set outputType to a Zod schema, Stratus sends the appropriate response_format to Azure automatically.

const agent = new Agent({
  name: "extractor",
  model,
  outputType: z.object({
    name: z.string(),
    age: z.number(),
  }),
});

See Structured Output for details.

Predicted output

Predicted output speeds up completions when you know roughly what the model will return (e.g. code edits). The model diffs against your prediction instead of generating from scratch.

const agent = new Agent({
  name: "editor",
  model,
  modelSettings: {
    prediction: { 
      type: "content", 
      content: existingCode, 
    }, 
  },
});

Audio output

For gpt-4o-audio deployments, you can request audio output alongside text.

const agent = new Agent({
  name: "narrator",
  model,
  modelSettings: {
    modalities: ["text", "audio"], 
    audio: { voice: "alloy", format: "mp3" }, 
  },
});

Available voices: "alloy", "echo", "fable", "onyx", "nova", "shimmer".

Available formats: "wav", "mp3", "flac", "opus", "pcm16".

Data sources

Azure On Your Data lets you ground model responses in your own data via Azure Search, Cosmos DB, and other Azure data sources. The model queries the data source and includes relevant results in its context.

const agent = new Agent({
  name: "rag-agent",
  model,
  modelSettings: {
    dataSources: [{ 
      type: "azure_search", 
      parameters: { 
        endpoint: "https://search.example.com", 
        index_name: "knowledge-base", 
        authentication: { type: "api_key", key: process.env.SEARCH_KEY! }, 
      }, 
    }], 
  },
});

Encrypted reasoning

When using reasoning models (o3, o4-mini) in stateless mode (store: false), you need to preserve reasoning context across conversation turns. Set include to receive encrypted reasoning items that can be passed back in the next request.

const model = new AzureResponsesModel({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT!,
  apiKey: process.env.AZURE_OPENAI_API_KEY!,
  deployment: "o4-mini",
  store: false,
});

// First turn: request encrypted reasoning
const response = await model.getResponse({
  messages: [{ role: "user", content: "Solve this step by step: 15 * 23 + 7" }],
  modelSettings: {
    reasoningEffort: "medium",
    include: ["reasoning.encrypted_content"], 
  },
});

// Extract reasoning items from output
const reasoningItems = response.outputItems?.filter(
  (item) => item.type === "reasoning"
) ?? [];

// Second turn: pass reasoning back to preserve context
const followUp = await model.getResponse({
  messages: [{ role: "user", content: "Now divide that result by 4" }],
  rawInputItems: reasoningItems, 
  modelSettings: {
    reasoningEffort: "medium",
    include: ["reasoning.encrypted_content"],
  },
});

The encrypted reasoning items are opaque — they can't be read or modified, only passed back to the API. This allows the model to maintain its reasoning chain across turns without server-side storage.

Context compaction

For long-running sessions on the Responses API, server-side context compaction shrinks the context window while preserving essential information.

const agent = new Agent({
  name: "long-session",
  model: responsesModel,
  modelSettings: {
    contextManagement: [{ 
      type: "compaction", 
      compact_threshold: 200000, 
    }], 
  },
});

When the output token count crosses compact_threshold, the API automatically compacts the context and emits a compaction item in outputItems. On subsequent turns the compaction item carries forward essential context using fewer tokens.

You can pass compaction items back in follow-up requests via rawInputItems:

const result = await model.getResponse({
  messages: [{ role: "user", content: "Continue the conversation" }],
  modelSettings: {
    contextManagement: [{ type: "compaction", compact_threshold: 200000 }],
  },
});

// If compaction occurred, pass the item back in the next request
const compactionItems = result.outputItems?.filter(
  (item) => item.type === "compaction"
) ?? [];

const followUp = await model.getResponse({
  messages: [{ role: "user", content: "What were we talking about?" }],
  rawInputItems: compactionItems, 
});

For explicit compaction (outside of automatic context_management), use the compact() method on AzureResponsesModel.

Next steps

• Tools -- define functions the model can call
• Agents -- agent configuration reference
• Streaming -- stream responses in real time
• Hooks -- intercept tool calls and handoffs before they execute