Skip to content

Commit 904348a

Browse files
feat(api): Support images and files for function call outputs in responses, BatchUsage
This release introduces a major version change to reflect a breaking modification in the `ResponseFunctionToolCallOutputItem` and `ResponseCustomToolCallOutput` schemas. Specifically, the `output` field, which previously accepted only a `string` value, has been expanded to support multiple structured types: ``` Before: output: string After: output: string | Array<ResponseInputText | ResponseInputImage | ResponseInputFile> ``` This change allows custom tool calls to return images, files, and rich text content in addition to plain strings, aligning `ResponseCustomToolCallOutput` with the broader `ResponseInput` type system. Because this alters the type and shape of the field, it may break existing callsites that only accept strings. BREAKING CHANGE: `ResponseFunctionToolCallOutputItem.output` and `ResponseCustomToolCallOutput.output` now return `string | Array<ResponseInputText | ResponseInputImage | ResponseInputFile>` instead of `string` only. This may break existing callsites that assume `output` is always a string.
1 parent e0b022d commit 904348a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+1388
-56
lines changed

.stats.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
configured_endpoints: 118
2-
openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-410219ea680089f02bb55163c673919703f946c3d6ad7ff5d6f607121d5287d5.yml
3-
openapi_spec_hash: 2b3eee95d3f6796c7a61dfddf694a59a
4-
config_hash: 666d6bb4b564f0d9d431124b5d1a0665
2+
openapi_spec_url: https://storage.googleapis.com/stainless-sdk-openapi-specs/openai%2Fopenai-49233088b5e73dbb96bf7af27be3d4547632e3db1c2b00f14184900613325bbc.yml
3+
openapi_spec_hash: b34f14b141d5019244112901c5c7c2d8
4+
config_hash: 94e9ba08201c3d1ca46e093e6a0138fa

lib/openai.rb

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@
9999
require_relative "openai/models/batch_list_params"
100100
require_relative "openai/models/batch_request_counts"
101101
require_relative "openai/models/batch_retrieve_params"
102+
require_relative "openai/models/batch_usage"
102103
require_relative "openai/models/beta/assistant"
103104
require_relative "openai/models/beta/assistant_create_params"
104105
require_relative "openai/models/beta/assistant_deleted"
@@ -537,6 +538,8 @@
537538
require_relative "openai/models/responses/response_format_text_json_schema_config"
538539
require_relative "openai/models/responses/response_function_call_arguments_delta_event"
539540
require_relative "openai/models/responses/response_function_call_arguments_done_event"
541+
require_relative "openai/models/responses/response_function_call_output_item"
542+
require_relative "openai/models/responses/response_function_call_output_item_list"
540543
require_relative "openai/models/responses/response_function_tool_call_item"
541544
require_relative "openai/models/responses/response_function_tool_call_output_item"
542545
require_relative "openai/models/responses/response_function_web_search"
@@ -550,9 +553,12 @@
550553
require_relative "openai/models/responses/response_input"
551554
require_relative "openai/models/responses/response_input_audio"
552555
require_relative "openai/models/responses/response_input_content"
556+
require_relative "openai/models/responses/response_input_file_content"
557+
require_relative "openai/models/responses/response_input_image_content"
553558
require_relative "openai/models/responses/response_input_item"
554559
require_relative "openai/models/responses/response_input_message_content_list"
555560
require_relative "openai/models/responses/response_input_message_item"
561+
require_relative "openai/models/responses/response_input_text_content"
556562
require_relative "openai/models/responses/response_item"
557563
require_relative "openai/models/responses/response_item_list"
558564
require_relative "openai/models/responses/response_mcp_call_arguments_delta_event"

lib/openai/models.rb

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,8 @@ module OpenAI
6363

6464
BatchRetrieveParams = OpenAI::Models::BatchRetrieveParams
6565

66+
BatchUsage = OpenAI::Models::BatchUsage
67+
6668
Beta = OpenAI::Models::Beta
6769

6870
Chat = OpenAI::Models::Chat

lib/openai/models/batch.rb

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,16 @@ class Batch < OpenAI::Internal::Type::BaseModel
115115
# @return [Hash{Symbol=>String}, nil]
116116
optional :metadata, OpenAI::Internal::Type::HashOf[String], nil?: true
117117

118+
# @!attribute model
119+
# Model ID used to process the batch, like `gpt-5-2025-08-07`. OpenAI offers a
120+
# wide range of models with different capabilities, performance characteristics,
121+
# and price points. Refer to the
122+
# [model guide](https://platform.openai.com/docs/models) to browse and compare
123+
# available models.
124+
#
125+
# @return [String, nil]
126+
optional :model, String
127+
118128
# @!attribute output_file_id
119129
# The ID of the file containing the outputs of successfully executed requests.
120130
#
@@ -127,7 +137,15 @@ class Batch < OpenAI::Internal::Type::BaseModel
127137
# @return [OpenAI::Models::BatchRequestCounts, nil]
128138
optional :request_counts, -> { OpenAI::BatchRequestCounts }
129139

130-
# @!method initialize(id:, completion_window:, created_at:, endpoint:, input_file_id:, status:, cancelled_at: nil, cancelling_at: nil, completed_at: nil, error_file_id: nil, errors: nil, expired_at: nil, expires_at: nil, failed_at: nil, finalizing_at: nil, in_progress_at: nil, metadata: nil, output_file_id: nil, request_counts: nil, object: :batch)
140+
# @!attribute usage
141+
# Represents token usage details including input tokens, output tokens, a
142+
# breakdown of output tokens, and the total tokens used. Only populated on batches
143+
# created after September 7, 2025.
144+
#
145+
# @return [OpenAI::Models::BatchUsage, nil]
146+
optional :usage, -> { OpenAI::BatchUsage }
147+
148+
# @!method initialize(id:, completion_window:, created_at:, endpoint:, input_file_id:, status:, cancelled_at: nil, cancelling_at: nil, completed_at: nil, error_file_id: nil, errors: nil, expired_at: nil, expires_at: nil, failed_at: nil, finalizing_at: nil, in_progress_at: nil, metadata: nil, model: nil, output_file_id: nil, request_counts: nil, usage: nil, object: :batch)
131149
# Some parameter documentations has been truncated, see {OpenAI::Models::Batch}
132150
# for more details.
133151
#
@@ -165,10 +183,14 @@ class Batch < OpenAI::Internal::Type::BaseModel
165183
#
166184
# @param metadata [Hash{Symbol=>String}, nil] Set of 16 key-value pairs that can be attached to an object. This can be
167185
#
186+
# @param model [String] Model ID used to process the batch, like `gpt-5-2025-08-07`. OpenAI
187+
#
168188
# @param output_file_id [String] The ID of the file containing the outputs of successfully executed requests.
169189
#
170190
# @param request_counts [OpenAI::Models::BatchRequestCounts] The request counts for different statuses within the batch.
171191
#
192+
# @param usage [OpenAI::Models::BatchUsage] Represents token usage details including input tokens, output tokens, a
193+
#
172194
# @param object [Symbol, :batch] The object type, which is always `batch`.
173195

174196
# The current status of the batch.

lib/openai/models/batch_usage.rb

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# frozen_string_literal: true
2+
3+
module OpenAI
4+
module Models
5+
class BatchUsage < OpenAI::Internal::Type::BaseModel
6+
# @!attribute input_tokens
7+
# The number of input tokens.
8+
#
9+
# @return [Integer]
10+
required :input_tokens, Integer
11+
12+
# @!attribute input_tokens_details
13+
# A detailed breakdown of the input tokens.
14+
#
15+
# @return [OpenAI::Models::BatchUsage::InputTokensDetails]
16+
required :input_tokens_details, -> { OpenAI::BatchUsage::InputTokensDetails }
17+
18+
# @!attribute output_tokens
19+
# The number of output tokens.
20+
#
21+
# @return [Integer]
22+
required :output_tokens, Integer
23+
24+
# @!attribute output_tokens_details
25+
# A detailed breakdown of the output tokens.
26+
#
27+
# @return [OpenAI::Models::BatchUsage::OutputTokensDetails]
28+
required :output_tokens_details, -> { OpenAI::BatchUsage::OutputTokensDetails }
29+
30+
# @!attribute total_tokens
31+
# The total number of tokens used.
32+
#
33+
# @return [Integer]
34+
required :total_tokens, Integer
35+
36+
# @!method initialize(input_tokens:, input_tokens_details:, output_tokens:, output_tokens_details:, total_tokens:)
37+
# Represents token usage details including input tokens, output tokens, a
38+
# breakdown of output tokens, and the total tokens used. Only populated on batches
39+
# created after September 7, 2025.
40+
#
41+
# @param input_tokens [Integer] The number of input tokens.
42+
#
43+
# @param input_tokens_details [OpenAI::Models::BatchUsage::InputTokensDetails] A detailed breakdown of the input tokens.
44+
#
45+
# @param output_tokens [Integer] The number of output tokens.
46+
#
47+
# @param output_tokens_details [OpenAI::Models::BatchUsage::OutputTokensDetails] A detailed breakdown of the output tokens.
48+
#
49+
# @param total_tokens [Integer] The total number of tokens used.
50+
51+
# @see OpenAI::Models::BatchUsage#input_tokens_details
52+
class InputTokensDetails < OpenAI::Internal::Type::BaseModel
53+
# @!attribute cached_tokens
54+
# The number of tokens that were retrieved from the cache.
55+
# [More on prompt caching](https://platform.openai.com/docs/guides/prompt-caching).
56+
#
57+
# @return [Integer]
58+
required :cached_tokens, Integer
59+
60+
# @!method initialize(cached_tokens:)
61+
# Some parameter documentations has been truncated, see
62+
# {OpenAI::Models::BatchUsage::InputTokensDetails} for more details.
63+
#
64+
# A detailed breakdown of the input tokens.
65+
#
66+
# @param cached_tokens [Integer] The number of tokens that were retrieved from the cache. [More on
67+
end
68+
69+
# @see OpenAI::Models::BatchUsage#output_tokens_details
70+
class OutputTokensDetails < OpenAI::Internal::Type::BaseModel
71+
# @!attribute reasoning_tokens
72+
# The number of reasoning tokens.
73+
#
74+
# @return [Integer]
75+
required :reasoning_tokens, Integer
76+
77+
# @!method initialize(reasoning_tokens:)
78+
# A detailed breakdown of the output tokens.
79+
#
80+
# @param reasoning_tokens [Integer] The number of reasoning tokens.
81+
end
82+
end
83+
end
84+
end

lib/openai/models/responses/response_custom_tool_call_output.rb

Lines changed: 47 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,11 @@ class ResponseCustomToolCallOutput < OpenAI::Internal::Type::BaseModel
1111
required :call_id, String
1212

1313
# @!attribute output
14-
# The output from the custom tool call generated by your code.
14+
# The output from the custom tool call generated by your code. Can be a string or
15+
# an list of output content.
1516
#
16-
# @return [String]
17-
required :output, String
17+
# @return [String, Array<OpenAI::Models::Responses::ResponseInputText, OpenAI::Models::Responses::ResponseInputImage, OpenAI::Models::Responses::ResponseInputFile>]
18+
required :output, union: -> { OpenAI::Responses::ResponseCustomToolCallOutput::Output }
1819

1920
# @!attribute type
2021
# The type of the custom tool call output. Always `custom_tool_call_output`.
@@ -36,11 +37,53 @@ class ResponseCustomToolCallOutput < OpenAI::Internal::Type::BaseModel
3637
#
3738
# @param call_id [String] The call ID, used to map this custom tool call output to a custom tool call.
3839
#
39-
# @param output [String] The output from the custom tool call generated by your code.
40+
# @param output [String, Array<OpenAI::Models::Responses::ResponseInputText, OpenAI::Models::Responses::ResponseInputImage, OpenAI::Models::Responses::ResponseInputFile>] The output from the custom tool call generated by your code.
4041
#
4142
# @param id [String] The unique ID of the custom tool call output in the OpenAI platform.
4243
#
4344
# @param type [Symbol, :custom_tool_call_output] The type of the custom tool call output. Always `custom_tool_call_output`.
45+
46+
# The output from the custom tool call generated by your code. Can be a string or
47+
# an list of output content.
48+
#
49+
# @see OpenAI::Models::Responses::ResponseCustomToolCallOutput#output
50+
module Output
51+
extend OpenAI::Internal::Type::Union
52+
53+
# A string of the output of the custom tool call.
54+
variant String
55+
56+
# Text, image, or file output of the custom tool call.
57+
variant -> { OpenAI::Models::Responses::ResponseCustomToolCallOutput::Output::OutputContentListArray }
58+
59+
# A text input to the model.
60+
module OutputContentList
61+
extend OpenAI::Internal::Type::Union
62+
63+
discriminator :type
64+
65+
# A text input to the model.
66+
variant :input_text, -> { OpenAI::Responses::ResponseInputText }
67+
68+
# An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision).
69+
variant :input_image, -> { OpenAI::Responses::ResponseInputImage }
70+
71+
# A file input to the model.
72+
variant :input_file, -> { OpenAI::Responses::ResponseInputFile }
73+
74+
# @!method self.variants
75+
# @return [Array(OpenAI::Models::Responses::ResponseInputText, OpenAI::Models::Responses::ResponseInputImage, OpenAI::Models::Responses::ResponseInputFile)]
76+
end
77+
78+
# @!method self.variants
79+
# @return [Array(String, Array<OpenAI::Models::Responses::ResponseInputText, OpenAI::Models::Responses::ResponseInputImage, OpenAI::Models::Responses::ResponseInputFile>)]
80+
81+
# @type [OpenAI::Internal::Type::Converter]
82+
OutputContentListArray =
83+
OpenAI::Internal::Type::ArrayOf[union: -> {
84+
OpenAI::Responses::ResponseCustomToolCallOutput::Output::OutputContentList
85+
}]
86+
end
4487
end
4588
end
4689
end

lib/openai/models/responses/response_function_call_arguments_done_event.rb

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,12 @@ class ResponseFunctionCallArgumentsDoneEvent < OpenAI::Internal::Type::BaseModel
1616
# @return [String]
1717
required :item_id, String
1818

19+
# @!attribute name
20+
# The name of the function that was called.
21+
#
22+
# @return [String]
23+
required :name, String
24+
1925
# @!attribute output_index
2026
# The index of the output item.
2127
#
@@ -33,13 +39,15 @@ class ResponseFunctionCallArgumentsDoneEvent < OpenAI::Internal::Type::BaseModel
3339
# @return [Symbol, :"response.function_call_arguments.done"]
3440
required :type, const: :"response.function_call_arguments.done"
3541

36-
# @!method initialize(arguments:, item_id:, output_index:, sequence_number:, type: :"response.function_call_arguments.done")
42+
# @!method initialize(arguments:, item_id:, name:, output_index:, sequence_number:, type: :"response.function_call_arguments.done")
3743
# Emitted when function-call arguments are finalized.
3844
#
3945
# @param arguments [String] The function-call arguments.
4046
#
4147
# @param item_id [String] The ID of the item.
4248
#
49+
# @param name [String] The name of the function that was called.
50+
#
4351
# @param output_index [Integer] The index of the output item.
4452
#
4553
# @param sequence_number [Integer] The sequence number of this event.
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# frozen_string_literal: true
2+
3+
module OpenAI
4+
module Models
5+
module Responses
6+
# A text input to the model.
7+
module ResponseFunctionCallOutputItem
8+
extend OpenAI::Internal::Type::Union
9+
10+
discriminator :type
11+
12+
# A text input to the model.
13+
variant :input_text, -> { OpenAI::Responses::ResponseInputTextContent }
14+
15+
# An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision)
16+
variant :input_image, -> { OpenAI::Responses::ResponseInputImageContent }
17+
18+
# A file input to the model.
19+
variant :input_file, -> { OpenAI::Responses::ResponseInputFileContent }
20+
21+
# @!method self.variants
22+
# @return [Array(OpenAI::Models::Responses::ResponseInputTextContent, OpenAI::Models::Responses::ResponseInputImageContent, OpenAI::Models::Responses::ResponseInputFileContent)]
23+
end
24+
end
25+
end
26+
end
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# frozen_string_literal: true
2+
3+
module OpenAI
4+
module Models
5+
module Responses
6+
# @type [OpenAI::Internal::Type::Converter]
7+
ResponseFunctionCallOutputItemList =
8+
OpenAI::Internal::Type::ArrayOf[union: -> { OpenAI::Responses::ResponseFunctionCallOutputItem }]
9+
end
10+
end
11+
end

lib/openai/models/responses/response_function_tool_call_output_item.rb

Lines changed: 47 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,11 @@ class ResponseFunctionToolCallOutputItem < OpenAI::Internal::Type::BaseModel
1717
required :call_id, String
1818

1919
# @!attribute output
20-
# A JSON string of the output of the function tool call.
20+
# The output from the function call generated by your code. Can be a string or an
21+
# list of output content.
2122
#
22-
# @return [String]
23-
required :output, String
23+
# @return [String, Array<OpenAI::Models::Responses::ResponseInputText, OpenAI::Models::Responses::ResponseInputImage, OpenAI::Models::Responses::ResponseInputFile>]
24+
required :output, union: -> { OpenAI::Responses::ResponseFunctionToolCallOutputItem::Output }
2425

2526
# @!attribute type
2627
# The type of the function tool call output. Always `function_call_output`.
@@ -44,12 +45,54 @@ class ResponseFunctionToolCallOutputItem < OpenAI::Internal::Type::BaseModel
4445
#
4546
# @param call_id [String] The unique ID of the function tool call generated by the model.
4647
#
47-
# @param output [String] A JSON string of the output of the function tool call.
48+
# @param output [String, Array<OpenAI::Models::Responses::ResponseInputText, OpenAI::Models::Responses::ResponseInputImage, OpenAI::Models::Responses::ResponseInputFile>] The output from the function call generated by your code.
4849
#
4950
# @param status [Symbol, OpenAI::Models::Responses::ResponseFunctionToolCallOutputItem::Status] The status of the item. One of `in_progress`, `completed`, or
5051
#
5152
# @param type [Symbol, :function_call_output] The type of the function tool call output. Always `function_call_output`.
5253

54+
# The output from the function call generated by your code. Can be a string or an
55+
# list of output content.
56+
#
57+
# @see OpenAI::Models::Responses::ResponseFunctionToolCallOutputItem#output
58+
module Output
59+
extend OpenAI::Internal::Type::Union
60+
61+
# A string of the output of the function call.
62+
variant String
63+
64+
# Text, image, or file output of the function call.
65+
variant -> { OpenAI::Models::Responses::ResponseFunctionToolCallOutputItem::Output::OutputContentListArray }
66+
67+
# A text input to the model.
68+
module OutputContentList
69+
extend OpenAI::Internal::Type::Union
70+
71+
discriminator :type
72+
73+
# A text input to the model.
74+
variant :input_text, -> { OpenAI::Responses::ResponseInputText }
75+
76+
# An image input to the model. Learn about [image inputs](https://platform.openai.com/docs/guides/vision).
77+
variant :input_image, -> { OpenAI::Responses::ResponseInputImage }
78+
79+
# A file input to the model.
80+
variant :input_file, -> { OpenAI::Responses::ResponseInputFile }
81+
82+
# @!method self.variants
83+
# @return [Array(OpenAI::Models::Responses::ResponseInputText, OpenAI::Models::Responses::ResponseInputImage, OpenAI::Models::Responses::ResponseInputFile)]
84+
end
85+
86+
# @!method self.variants
87+
# @return [Array(String, Array<OpenAI::Models::Responses::ResponseInputText, OpenAI::Models::Responses::ResponseInputImage, OpenAI::Models::Responses::ResponseInputFile>)]
88+
89+
# @type [OpenAI::Internal::Type::Converter]
90+
OutputContentListArray =
91+
OpenAI::Internal::Type::ArrayOf[union: -> {
92+
OpenAI::Responses::ResponseFunctionToolCallOutputItem::Output::OutputContentList
93+
}]
94+
end
95+
5396
# The status of the item. One of `in_progress`, `completed`, or `incomplete`.
5497
# Populated when items are returned via API.
5598
#

0 commit comments

Comments
 (0)