Skip to content

Conversation

adrianlizarraga
Copy link
Contributor

Description

  • Add ability to drop constant initializers for fused nodes specified in GetCapability.
  • Rework how an EP specifies nodes that should be fused into one node within GetCapability.
    • Instead of passing the set of nodes as arguments to GraphSupportInfo_AddNodesToFuse(), the EP creates an OrtNodeFusionOptions object to specify the nodes and other relevant options. This makes it easier to extend the API in the future since we can't add more parameters to an existing function, but we can add more functions that modify an options object.

Motivation and Context

Add more functionality missing from GetCapability() in the EP ABI.

@adrianlizarraga adrianlizarraga marked this pull request as ready for review June 23, 2025 21:44
@chilo-ms
Copy link
Contributor

chilo-ms commented Jun 24, 2025

Another request that EP might want is to create the fused node's id hashed from not only graph_viewer but also other info, ex: TRT version.
The case for TRT is to distinguish engines built from same OrtGraph with different version of TRT libraries. The workflow is TRT EP provides the id to ORT and gets the fused node's name and assign that name for the engine cache file, and next time when running the new inference, it can search for the cache with the name.

OrtNodeFusionOptions can address this issue by letting EP provide its own hashed id.

Copy link
Contributor

@skottmckay skottmckay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@adrianlizarraga adrianlizarraga merged commit 27cdb5c into main Jun 26, 2025
89 checks passed
@adrianlizarraga adrianlizarraga deleted the adrianl/ep-abi-get-capability-drop-const-initializers branch June 26, 2025 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants