Skip to content

[QNN-EP] Add Support for CumSum in QNN EP #24820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jun 5, 2025
Merged

[QNN-EP] Add Support for CumSum in QNN EP #24820

merged 3 commits into from
Jun 5, 2025

Conversation

Akupadhye
Copy link
Contributor

  • Registered CumSum op in QNN EP
  • Added unit test to verify accuracy and assignment of op to QNN EP

Description

Added support for CumSum in QNN EP

Motivation and Context

There is no support for CumSum op in QNN EP

@Akupadhye
Copy link
Contributor Author

@microsoft-github-policy-service agree company="Qualcomm"

@edgchen1 edgchen1 added the ep:QNN issues related to QNN exeution provider label May 20, 2025
@HectorSVC
Copy link
Contributor

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@HectorSVC
Copy link
Contributor

Please check the doc to make sure there are no steps missing: https://onnxruntime.ai/docs/execution-providers/QNN-ExecutionProvider.html#add-new-operator-support-in-qnn-ep
For example, make sure the DQ->CumSum->Q is treat as a single node unit, so it's running as quantized CumSum instead of FP32 in FP16 mode on HTP. Another recent example is: #24659

@Akupadhye
Copy link
Contributor Author

verified that DQ->CumSum->Q is now treated as a single node unit.

2025-05-23 11:30:00.4123831 [V:onnxruntime:qdq_model_logger, qnn_model_wrapper.cc:256 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] Qnn_OpConfig node name: node_token_7 package_name: qti.aisw QNN_op_type: CumulativeSum num_of_inputs: 1 num_of_outputs: 1 num_of_params: 3
node_inputs:
name=node id=0 version=1 type=QNN_TENSOR_TYPE_NATIVE dataFormat=0 dataType=QNN_DATATYPE_UFIXED_POINT_8 rank=2 dimensions=(3 2 ) memType=QNN_TENSORMEMTYPE_RAW quantizeParams: encodingDefinition=QNN_DEFINITION_DEFINED quantizationEncoding=QNN_QUANTIZATION_ENCODING_SCALE_OFFSET
scale=0.0282353 offset=0
node_outputs:
name=node_token_8 id=0 version=1 type=QNN_TENSOR_TYPE_NATIVE dataFormat=0 dataType=QNN_DATATYPE_UFIXED_POINT_8 rank=2 dimensions=(3 2 ) memType=QNN_TENSORMEMTYPE_RAW quantizeParams: encodingDefinition=QNN_DEFINITION_DEFINED
quantizationEncoding=QNN_QUANTIZATION_ENCODING_SCALE_OFFSET scale=0.0447059 offset=0
node_params:
type=QNN_PARAMTYPE_SCALAR name=axis value=0
type=QNN_PARAMTYPE_SCALAR name=exclusive value=0
type=QNN_PARAMTYPE_SCALAR name=reverse value=0

@HectorSVC
Copy link
Contributor

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 5 pipeline(s).

HectorSVC
HectorSVC previously approved these changes May 28, 2025
View reviewed changes
Copy link
Contributor

@HectorSVC HectorSVC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@HectorSVC
Copy link
Contributor

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 5 pipeline(s).

Akupadhye added 3 commits June 4, 2025 12:15
@Akupadhye @qc-tbhardwa
[QNN-EP] Add Support for CumSum in QNN EP
96fcc2d 
- Registered CumSum op in QNN EP
- Added unit test to verify accuracy and assignment of op to QNN EP

Description
- Added support for CumSum in QNN EP

Motivation and Context
- There is no support for CumSum op in QNN EP
@Akupadhye @qc-tbhardwa
[QNN-EP] Add CumSum QDQ node selector
d0b4278 
Description
- Added CumSum QDQ node selector

Motivation and Context
- DQ->CumSum->Q is not recognized as a single qdq node unit. The CumSum
  node would run in FP16 precision in QNN HTP. Hence added the CumSum
  node unit selector to recognize it as a quantized CumSum node.
@Akupadhye @qc-tbhardwa
[QNN-EP] Fix lint errors
26254d0
@qc-tbhardwa qc-tbhardwa force-pushed the dev/Akupadhye/cumsum_op_support branch from 72d30d5 to 26254d0 Compare June 4, 2025 06:48
@HectorSVC
Copy link
Contributor

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 5 pipeline(s).

HectorSVC
HectorSVC approved these changes Jun 5, 2025
View reviewed changes
Copy link
Contributor

@HectorSVC HectorSVC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@HectorSVC HectorSVC merged commit 97d8d90 into microsoft:main Jun 5, 2025
82 checks passed
javier-intel pushed a commit to intel/onnxruntime that referenced this pull request Jun 15, 2025
@Akupadhye @javier-intel
[QNN-EP] Add Support for CumSum in QNN EP (microsoft#24820)
e6f3c29 
- Registered CumSum op in QNN EP
- Added unit test to verify accuracy and assignment of op to QNN EP

### Description
Added support for CumSum in QNN EP



### Motivation and Context
There is no support for CumSum op in QNN EP
adrianlizarraga pushed a commit that referenced this pull request Aug 1, 2025
@Akupadhye @adrianlizarraga
[QNN-EP] Add Support for CumSum in QNN EP (#24820)
3942795 
- Registered CumSum op in QNN EP
- Added unit test to verify accuracy and assignment of op to QNN EP

### Description
Added support for CumSum in QNN EP



### Motivation and Context
There is no support for CumSum op in QNN EP
@adrianlizarraga adrianlizarraga mentioned this pull request Aug 1, 2025
Merged
adrianlizarraga pushed a commit that referenced this pull request Aug 5, 2025
@Akupadhye @adrianlizarraga
[QNN-EP] Add Support for CumSum in QNN EP (#24820)
cf0ac6d 
- Registered CumSum op in QNN EP
- Added unit test to verify accuracy and assignment of op to QNN EP

### Description
Added support for CumSum in QNN EP



### Motivation and Context
There is no support for CumSum op in QNN EP
adrianlizarraga added a commit that referenced this pull request Aug 11, 2025
@adrianlizarraga @zhaoxul-qti
@1duo @HectorSVC @chenweng-quic @qti-yuduo @Akupadhye @qti-jkilpatrick @jeffkilpatrick @jywu-msft @github-actions @quic-calvnguy @snnn @fs-eire
rel-1.22.2 cherry-pick 1 (#25633)
19f37d7 
### Description
- #24265
- #24616
- #24640
- #24707
- #24646
- #24750
- #24809
- #24895
- #24820
- #25002
- #25171
- #25283
- #24818
- #25351
- #25361
- #25388
- #25520
- #25158




### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

Co-authored-by: quic-zhaoxul <[email protected]>
Co-authored-by: Yuduo Wu <[email protected]>
Co-authored-by: Hector Li <[email protected]>
Co-authored-by: chenweng-quic <[email protected]>
Co-authored-by: qti-yuduo <[email protected]>
Co-authored-by: Akupadhye <[email protected]>
Co-authored-by: Jeff Kilpatrick <[email protected]>
Co-authored-by: Jeff Kilpatrick <[email protected]>
Co-authored-by: George Wu <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: quic-calvnguy <[email protected]>
Co-authored-by: Changming Sun <[email protected]>
Co-authored-by: Yulong Wang <[email protected]>
This was referenced Sep 29, 2025
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HectorSVC HectorSVC HectorSVC approved these changes

Assignees
No one assigned
Labels
ep:QNN issues related to QNN exeution provider
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Akupadhye @HectorSVC @edgchen1