Using Reasoning vs Standard Models

AI Assistant API supports two model tiers, which you can specify using the reasoning.effort field in the request:

  • Standard, or fast, model: Use for straightforward questions, including message counts, lookups, and message/thread summaries.
    To use the standard model: Specify minimal in the reasoning.effort field.
  • Reasoning, or thinking, model: Use for complex analysis, including risk assessment, regulatory evaluation, or multi-step reasoning.
    To use the reasoning model: Specify low, medium, or high in the reasoning.effort field, where high will take longer to generate an answer.

If no value is provided in the reasoning.effort field, the standard model is used by default.

The reasoning model spends additional tokens on internal “thinking” before producing its answer, which is reflected in the output_tokens_details.reasoning_tokens field in the response.

Identifying reasoning items in the response

When you use the reasoning model, the response output array contains standard items along with reasoning items, where each reasoning item includes:

  • type: "reasoning"
  • content: An array of reasoning_text parts showing analytical process of AI Assistant.
  • summary: An array, which will be empty in this initial version of the AI Assistant API.

The answer always appears in a message item, i.e. type: "message", with content in the output_text field, providing transparency into how AI Assistant produces its answer.

Sending a reasoning model request

To send a request for a complex task where AI Assistant needs to evaluate multiple factors, use effort: "high":

curl -N -X POST "https://<api-host>/v1/responses" \
  -H "Authorization: Bearer <YOUR_ACCESS_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "stream": true,
    "conversation": "conv_a1b2c3d4",
    "input": "Analyze the compliance risks in the Andrea Lacey messages from the Lautner Wealth Merger case. Consider FINRA Rule 3110, SEC Rule 17a-4, and MiFID II Article 16. Flag any messages that suggest pre-arranged trading, off-channel communications, or failure to supervise.",
    "reasoning": {
      "effort": "high"
    },
    "tools": [
      { "type": "archive_search" }
    ]
  }'

Retrieving the reasoning items

After the stream completes, retrieve the response to see the full output including the reasoning items:

curl -s -X GET "https://<api-host>/v1/responses/resp_m1n2o3p4" \
  -H "Authorization: Bearer <YOUR_ACCESS_TOKEN>"

Response:

{
  "id": "resp_m1n2o3p4",
  "object": "response",
  "status": "completed",
  "created_at": 1725475400,
  "model": "gpt-4o",
  "conversation": { "id": "conv_a1b2c3d4" },
  "output": [
    {
      "id": "rs_001",
      "type": "reasoning",
      "summary": [],
      "content": [
        { "type": "reasoning_text", "text": "Evaluating FINRA Rule 3110: reviewing whether supervisory review was documented for messages concerning the merger timeline..." }
      ],
      "status": "completed"
    },
    {
      "id": "msg_001",
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "I identified the following potential compliance risks in Andrea Lacey's messages:\n\n1. FINRA Rule 3110 (Supervision): 3 messages dated Jan 22, Feb 3, and Feb 14 reference approval of the merger timeline without evidence of supervisory review.",
          "annotations": [
            {
              "type": "url_citation",
              "url": "grn://archive/arc_7f3a/msg/0042ab",
              "title": "RE: Lautner merger timeline",
              "start_index": 0,
              "end_index": 85
            },
            {
              "type": "url_citation",
              "url": "grn://archive/arc_7f3a/msg/0058cd",
              "title": "FW: Quick question about the merger",
              "start_index": 86,
              "end_index": 180
            }
          ]
        }
      ],
      "status": "completed"
    }
  ],
  "reasoning": { "effort": "high", "summary": null },
  "usage": {
    "input_tokens": 1850,
    "input_tokens_details": { "cached_tokens": 0 },
    "output_tokens": 420,
    "output_tokens_details": { "reasoning_tokens": 280 },
    "total_tokens": 2270
  }
}

In this example response, the output array contains two items:

  • Reasoning item, e.g. rs_001, type: "reasoning" – The reasoning_text content shows when AI Assistant evaluated regulatory framework against the messages. This provides transparency into the analysis but is not the final answer.
  • Message item, e.g. msg_001, type: "message" – Contains the structured findings with output_text content and url_citation annotations pointing to the specific source messages. This is the answer that would be presented to a reviewer if they were to ask a question and receive the answer in Global Relay Portal.

Determining the appropriate reasoning effort

Follow these guidelines to decide which reasoning effort level to use:

  • minimal or omitted: Use for straightforward questions, such as “How many messages did Andrea Lacey send me in the last 5 days?” or “List the dates of messages about the merger.”
  • low: Use for light analysis, such as “What are the main topics discussed in the Lautner Merger thread?”
  • medium: Use for moderate analysis, such as “Are there any messages that reference off-channel communications in the Lautner Merger thread?”
  • high: Use for deep analysis, such as “Evaluate the messages in the Lautner Merger Investigation case against FINRA, SEC, and MiFID II requirements and specific violations.”