Skip to content

ggml webgpu: add support for soft_max, optimize rms_norm #16357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 2, 2025
+565 −47
Merged

ggml webgpu: add support for soft_max, optimize rms_norm #16357

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
483615d
Add inplace softmax
reeselevine Sep 24, 2025
27b893a
Move rms_norm to split row approach
reeselevine Sep 24, 2025
f9bb89c
Update debug for supports_op
reeselevine Sep 24, 2025
5d8e678
clean up debug statements
reeselevine Sep 30, 2025
190b355
Update tests/test-backend-ops.cpp
reeselevine Oct 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions ggml/include/ggml.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1617,6 +1617,13 @@ extern "C" {
float scale,
float max_bias);

GGML_API struct ggml_tensor * ggml_soft_max_ext_inplace(
struct ggml_context * ctx,
struct ggml_tensor * a,
struct ggml_tensor * mask,
float scale,
float max_bias);

GGML_API void ggml_soft_max_add_sinks(
struct ggml_tensor * a,
struct ggml_tensor * sinks);
Expand Down
Loading
Loading