common : fix reasoning before forced tool call via tool_choice = required

[crat0z]

Make sure to read the contributing guidelines before submitting a PR

Hello! This patch allows gpt-oss models to reason before doing a tool call, when tool_choice is set to required.

A similar issue, but for Qwen3/Hermes 2: #15247

[aldehir]

Thanks for this! I'll check it more thoroughly this weekend but here are my initial thoughts:

1. The analysis grammar should support both when the recipient is in the role and in the channel. Usually, the recipient is in the role when the model uses the reasoning from the last tool call. In these cases, the model doesn't actually reason for that turn. So I believe it is more important to handle when the recipient is in the channel.

2. We should also add an optional commentary channel to the grammar.

[aldehir]

@crat0z

What do you think about this aldehir@c746984?

I wrote a simple script that provided a calculator to gpt-oss-120b (the 20b model doesn't reliably produce commentary messages)

First, I used tool_choice=auto to see how the gpt-oss responds when only constraining the tool call. This gives us insight on how to create a grammar to force tool use.

Multi-turn scenario with tool_choice=auto

Turn 1

Message

Calculate the following expressions one by one using the calculator tool.
Provide commentary messages in the process to let me know what you are going to do.

1. 2 + 3 * 4
2. sqrt(25) - 3^2
3. sin(pi/6) + cos(pi/3)
4. ln(e^2) * log(1000)
5. abs(-7) + ceil(4.2)
6. (5 + 3) * (10 - 6) / 2
7. 2^3 + sqrt(16) - 5
8. tan(pi/4) * exp(0)
9. floor(9.8) + deg(pi/2)
10. -3 + 4 * (2 + 1)

Response

Grammar Lazy: True

<|channel|>analysis<|message|>We need to compute each expression using calculator tool, provide commentary messages in the process. So we will step through each, call function calculate with expression. Provide commentary before each call. Then produce final results. Let's do.

<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{
  "expression": "2 + 3 * 4"
}

Turn 2

Tool Call Result

14.0

Response

Grammar Lazy: True

<|channel|>analysis<|message|>Now 2. sqrt(25) - 3^2.<|end|><|start|>assistant<|channel|>commentary<|message|>Calculating sqrt(25) - 3^2...<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"sqrt(25) - 3^2"}

Turn 3

Tool Call Result

-4.0

Response

Grammar Lazy: True

<|channel|>analysis<|message|>3. sin(pi/6) + cos(pi/3).<|end|><|start|>assistant<|channel|>commentary<|message|>Calculating the third expression.<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"sin(pi/6) + cos(pi/3)"}

Turn 4

Tool Call Result

1.0

Response

Grammar Lazy: True

<|channel|>analysis<|message|>4. ln(e^2) * log(1000). Note: log is base 10.<|end|><|start|>assistant<|channel|>commentary<|message|>Calculating ln(e^2) * log(1000).<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"ln(e^2) * log(1000)"}

Turn 5

Tool Call Result

6.0

Response

Grammar Lazy: True

<|channel|>analysis<|message|>5. abs(-7) + ceil(4.2).<|end|><|start|>assistant to=functions.calculate<|channel|>commentary<|message|>{"expression":"abs(-7) + ceil(4.2)"}

Turn 6

Tool Call Result

12.0

Response

Grammar Lazy: True

<|channel|>analysis<|message|>6. (5 + 3) * (10 - 6) / 2.<|end|><|start|>assistant<|channel|>commentary<|message|>Now calculating expression 6.<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"(5 + 3) * (10 - 6) / 2"}

Turn 7

Tool Call Result

16.0

Response

Grammar Lazy: True

<|channel|>analysis<|message|>7. 2^3 + sqrt(16) - 5.<|end|><|start|>assistant<|channel|>commentary<|message|>Now calculating expression 7.<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"2^3 + sqrt(16) - 5"}

Turn 8

Tool Call Result

7.0

Response

Grammar Lazy: True

<|channel|>analysis<|message|>8. tan(pi/4) * exp(0).<|end|><|start|>assistant<|channel|>commentary<|message|>Now compute tan(pi/4) * exp(0).<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"tan(pi/4) * exp(0)"}

Turn 9

Tool Call Result

0.9999999999999999

Response

Grammar Lazy: True

<|channel|>analysis<|message|>9. floor(9.8) + deg(pi/2). deg converts rad to degrees? Actually deg function converts radians to degrees. So deg(pi/2) = 90. floor(9.8)=9. Sum =99.<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"floor(9.8) + deg(pi/2)"}

Turn 10

Tool Call Result

99.0

Response

Grammar Lazy: True

<|channel|>analysis<|message|>10. -3 + 4 * (2 + 1).<|end|><|start|>assistant<|channel|>commentary<|message|>Now compute the tenth expression.<|end|><|start|>assistant<|channel|>commentary<|message|>Let's calculate -3 + 4 * (2 + 1).<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"-3 + 4 * (2 + 1)"}

Turn 11

Tool Call Result

9.0

Response

Grammar Lazy: True

<|channel|>final<|message|>Here’s the step‑by‑step commentary and the computed results for each expression:

1. **Expression:** `2 + 3 * 4`  
   *Commentary:* Multiplication has higher precedence, so compute `3 * 4 = 12` then add `2`.  
   **Result:** **14**

2. **Expression:** `sqrt(25) - 3^2`  
   *Commentary:* Square root of 25 is 5; `3^2 = 9`. Subtract: `5 - 9`.  
   **Result:** **‑4**

3. **Expression:** `sin(pi/6) + cos(pi/3)`  
   *Commentary:* Both terms evaluate to 0.5, so the sum is `0.5 + 0.5`.  
   **Result:** **1**

4. **Expression:** `ln(e^2) * log(1000)`  
   *Commentary:* `ln(e^2) = 2`; `log(1000)` (base‑10) = 3. Multiply: `2 * 3`.  
   **Result:** **6**

5. **Expression:** `abs(-7) + ceil(4.2)`  
   *Commentary:* Absolute value gives 7; ceiling of 4.2 is 5. Add them.  
   **Result:** **12**

6. **Expression:** `(5 + 3) * (10 - 6) / 2`  
   *Commentary:* Compute inside parentheses → `8 * 4 = 32`, then divide by 2.  
   **Result:** **16**

7. **Expression:** `2^3 + sqrt(16) - 5`  
   *Commentary:* `2^3 = 8`; `sqrt(16) = 4`. Sum and subtract: `8 + 4 - 5`.  
   **Result:** **7**

8. **Expression:** `tan(pi/4) * exp(0)`  
   *Commentary:* `tan(pi/4) = 1`; `exp(0) = 1`. Product is essentially 1 (tiny floating‑point error).  
   **Result:** **≈ 1** (displayed as 0.9999999999999999)

9. **Expression:** `floor(9.8) + deg(pi/2)`  
   *Commentary:* Floor of 9.8 is 9; `deg(pi/2)` converts π/2 radians to degrees = 90°. Add them.  
   **Result:** **99**

10. **Expression:** `-3 + 4 * (2 + 1)`  
    *Commentary:* Parentheses first → `2 + 1 = 3`; then `4 * 3 = 12`; finally `-3 + 12`.  
    **Result:** **9**

**Summary of results**

| # | Expression | Result |
|---|------------|--------|
| 1 | 2 + 3 * 4 | 14 |
| 2 | sqrt(25) - 3^2 | -4 |
| 3 | sin(pi/6) + cos(pi/3) | 1 |
| 4 | ln(e^2) * log(1000) | 6 |
| 5 | abs(-7) + ceil(4.2) | 12 |
| 6 | (5 + 3) * (10 - 6) / 2 | 16 |
| 7 | 2^3 + sqrt(16) - 5 | 7 |
| 8 | tan(pi/4) * exp(0) | ≈ 1 |
| 9 | floor(9.8) + deg(pi/2) | 99 |
|10 | -3 + 4 * (2 + 1) | 9 |

Let me know if you need any further calculations or explanations!

From this, you can see a couple of things:

1. gpt-oss can produce <|channel|>commentary<|message|> ... <|end|> to let the user know what it plans to do. This aligns with the preamble defined in the Harmony response format.
2. Most of the tool calls are done with the recipient in the channel, e.g. <|channel|>commentary to=.... In your current implementation, you are constraining it to produce tool calls with the recipient in the role, which may degrade performance.
3. Occasionally, gpt-oss will put the recipient in the role such as in turn 5, so we definitely want to handle both cases.

Here is an example using the commit I linked.

Multi-turn scenario with tool_choice=required

Turn 1

Message

Calculate the following expressions one by one using the calculator tool.
Provide commentary messages in the process to let me know what you are going to do.

1. 2 + 3 * 4
2. sqrt(25) - 3^2
3. sin(pi/6) + cos(pi/3)
4. ln(e^2) * log(1000)
5. abs(-7) + ceil(4.2)
6. (5 + 3) * (10 - 6) / 2
7. 2^3 + sqrt(16) - 5
8. tan(pi/4) * exp(0)
9. floor(9.8) + deg(pi/2)
10. -3 + 4 * (2 + 1)

Response

Grammar Lazy: False

<|channel|>analysis<|message|>We need to evaluate each expression using tool and provide commentary. We'll call calculate for each. Provide a commentary message before each. Then output results. We'll need to format tool calls.<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{
  "expression": "2 + 3 * 4"
}

Turn 2

Tool Call Result

14.0

Response

Grammar Lazy: False

<|channel|>analysis<|message|>Now next.<|end|><|start|>assistant<|channel|>commentary<|message|>Now evaluating sqrt(25) - 3^2.<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"sqrt(25) - 3^2"}

Turn 3

Tool Call Result

-4.0

Response

Grammar Lazy: False

<|channel|>analysis<|message|>Next.<|end|><|start|>assistant<|channel|>commentary<|message|>Now evaluate sin(pi/6) + cos(pi/3).<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"sin(pi/6) + cos(pi/3)"}

Turn 4

Tool Call Result

1.0

Response

Grammar Lazy: False

<|channel|>analysis<|message|>Next.<|end|><|start|>assistant<|channel|>commentary<|message|>Now compute ln(e^2) * log(1000).<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"ln(e^2) * log(1000)"}

Turn 5

Tool Call Result

6.0

Response

Grammar Lazy: False

<|channel|>analysis<|message|>Next.<|end|><|start|>assistant<|channel|>commentary<|message|>Now evaluate 5.<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"abs(-7) + ceil(4.2)"}

Turn 6

Tool Call Result

12.0

Response

Grammar Lazy: False

<|channel|>analysis<|message|>Next.<|end|><|start|>assistant<|channel|>commentary<|message|>Now calculate (5 + 3) * (10 - 6) / 2.<|end|><|start|>assistant<|channel|>analysis to=functions.calculate <|constrain|>json<|message|>{"expression":"(5 + 3) * (10 - 6) / 2"}

Turn 7

Tool Call Result

16.0

Response

Grammar Lazy: False

<|channel|>analysis<|message|>Next.<|end|><|start|>assistant to=functions.calculate<|channel|>commentary <|constrain|>json<|message|>{"expression":"2^3 + sqrt(16) - 5"}

Turn 8

Tool Call Result

7.0

Response

Grammar Lazy: False

<|channel|>analysis<|message|>Next.<|end|><|start|>assistant<|channel|>commentary<|message|>Now evaluating sin(pi/4)*exp(0).<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"tan(pi/4) * exp(0)"}

Turn 9

Tool Call Result

0.9999999999999999

Response

Grammar Lazy: False

<|channel|>analysis<|message|>Next.<|end|><|start|>assistant<|channel|>commentary<|message|>Now calculate floor(9.8) + deg(pi/2).<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"floor(9.8) + deg(pi/2)"}

Turn 10

Tool Call Result

99.0

Response

Grammar Lazy: False

<|channel|>analysis<|message|>Last.<|end|><|start|>assistant<|channel|>commentary<|message|>Now we compute the final expression.<|end|><|start|>assistant<|channel|>commentary to=functions.calculate <|constrain|>json<|message|>{"expression":"-3 + 4 * (2 + 1)"}

Turn 11

Tool Call Result

9.0

Response

Grammar Lazy: True

<|channel|>final<|message|>Here’s a step‑by‑step rundown of each calculation:

1. **Expression:** `2 + 3 * 4`  
   *I’ll multiply 3 × 4 first (respecting operator precedence) and then add 2.*  
   **Result:** **14**

2. **Expression:** `sqrt(25) - 3^2`  
   *Compute √25 = 5, evaluate 3² = 9, then subtract.*  
   **Result:** **‑4**

3. **Expression:** `sin(pi/6) + cos(pi/3)`  
   *Both angles are 30° (π/6) and 60° (π/3). sin(π/6)=0.5, cos(π/3)=0.5, sum = 1.*  
   **Result:** **1**

4. **Expression:** `ln(e^2) * log(1000)`  
   *ln(e²)=2, log₁₀(1000)=3, product = 6.*  
   **Result:** **6**

5. **Expression:** `abs(-7) + ceil(4.2)`  
   *|‑7| = 7, ceil(4.2) = 5, sum = 12.*  
   **Result:** **12**

6. **Expression:** `(5 + 3) * (10 - 6) / 2`  
   *(5+3)=8, (10‑6)=4 → 8 × 4 = 32, then ÷2 = 16.*  
   **Result:** **16**

7. **Expression:** `2^3 + sqrt(16) - 5`  
   *2³ = 8, √16 = 4 → 8 + 4 ‑ 5 = 7.*  
   **Result:** **7**

8. **Expression:** `tan(pi/4) * exp(0)`  
   *tan(π/4)=1, exp(0)=1, product ≈ 0.9999999999999999 (essentially 1).*  
   **Result:** **≈ 1** (displayed as 0.9999999999999999)

9. **Expression:** `floor(9.8) + deg(pi/2)`  
   *floor(9.8)=9, deg(π/2)=180°, sum = 189? Wait – the tool interpreted `deg(pi/2)` as converting radians to degrees, i.e., 90°, and then added to 9, giving 99.*  
   **Result:** **99**

10. **Expression:** `-3 + 4 * (2 + 1)`  
    *Parentheses first: (2+1)=3 → 4 × 3 = 12 → -3 + 12 = 9.*  
    **Result:** **9**

---

**Summary of results**

| # | Expression | Result |
|---|------------|--------|
| 1 | 2 + 3 * 4 | 14 |
| 2 | sqrt(25) - 3^2 | -4 |
| 3 | sin(pi/6) + cos(pi/3) | 1 |
| 4 | ln(e^2) * log(1000) | 6 |
| 5 | abs(-7) + ceil(4.2) | 12 |
| 6 | (5 + 3) * (10 - 6) / 2 | 16 |
| 7 | 2^3 + sqrt(16) - 5 | 7 |
| 8 | tan(pi/4) * exp(0) | ≈ 1 |
| 9 | floor(9.8) + deg(pi/2) | 99 |
|10 | -3 + 4 * (2 + 1) | 9 |

Let me know if you’d like any further explanation or additional calculations!

Note: I did set tool_choice=auto on the last turn just so the model produces a final message.

Using the commit above, gpt-oss can generate both reasoning and a preamble. Additionally, it handles both possible positions of the recipient--shown in turn 7.

Let me know if this handles your use cases, or if there's anything I am missing.

[crat0z]

Thanks for the quick reply! I built and ran your commit and it works for my use case as well. I saw the other case for tool calls in the commentary channel, but I wasn't sure what to do there. And I run 20b anyway. But your solution is much better

[aldehir]

Thanks for the quick reply! I built and ran your commit and it works for my use case as well. I saw the other case for tool calls in the commentary channel, but I wasn't sure what to do there. And I run 20b anyway. But your solution is much better

Awesome. Feel free to cherry-pick the commit to your PR.