Skip to content

fix: modality length mismatch with image_data #7887

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 15, 2025
Merged

fix: modality length mismatch with image_data #7887

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dcbfd29
fix: modality length mismatch with image_data
Yangruipis Jul 9, 2025
14050c8
Merge branch 'main' into fix/yanrguipis/modalities_length_mismatch
ispobock Jul 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions python/sglang/srt/managers/io_struct.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,9 @@ def _normalize_image_data(self, num):
self.modalities.append("image")
elif len(self.image_data[i]) > 1:
self.modalities.append("multi-images")
else:
# Ensure len(self.modalities) == len(self.image_data)
self.modalities.append(None)
Comment on lines +300 to +302
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This else block appears to be unreachable because the condition len(self.image_data[i]) == 0 is already handled by the elif not has_valid_data(self.image_data[i]): check earlier. This check ensures len(self.modalities) remains consistent with len(self.image_data). Please verify the specific input that was causing a length mismatch.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so. @Yangruipis may you check again?

Copy link
Contributor Author

@Yangruipis Yangruipis Jul 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see any logic about elif not has_valid_data(self.image_data[i]) here, would be kind to point it to me? thanks!

In fact I just got the issue when i'm passing image_data like [[<PIL.Image>], [], [<PIL.Image>, <PIL.Image>]] instead of [[<PIL.Image>], None, [<PIL.Image>, <PIL.Image>]] , it raises IndexError here

sglang/python/sglang/srt/managers/io_struct.py

Line 421 in 4ed5780

modalities=self.modalities[i] if self.modalities else None,

Whatever, it might be more safe and robust to handle the else logic here, we can't assume that the condition above can cover all the cases

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that makes sense. Thanks for pointing it out.

# Expand parallel_sample_num
self.image_data = self.image_data * self.parallel_sample_num
self.modalities = self.modalities * self.parallel_sample_num
Expand Down