Skip to content

Conversation

TomeHirata
Copy link
Collaborator

@TomeHirata TomeHirata commented Oct 1, 2025

Issue
There is an issue that some text content chunks are lost when there are more than 10 chunks for a string field, and the native response, such as Anthropic Citation, is used.

Root Cause
This was caused by the assumption that at most one streaming listener returns a chunk for the same chunk. This assumption is not true when native response, such as citation API, is used because a streaming listener buffers 10 chunks for string output fields, and sometimes the string chunk and native response chunk are produced simultaneously.

Fix
This PR fixes the issue by allowing multiple listeners to return a chunk for the same response chunk.

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a bug where text chunks were lost in long streaming responses when using native response fields like Anthropic Citations. The issue occurred when a streaming listener's buffer contained more than 10 chunks, and both string chunks and native response chunks were produced simultaneously.

  • Modified the streaming logic to handle multiple listeners returning chunks simultaneously
  • Enhanced test coverage with more realistic scenarios involving 10+ chunks
  • Fixed the assumption that only one listener can return a chunk at a time

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
tests/streaming/test_streaming.py Enhanced test with 10+ chunks and added answer chunk verification
dspy/streaming/streaming_listener.py Added guard to avoid empty tokens in buffer
dspy/streaming/streamify.py Fixed logic to handle multiple simultaneous chunk returns

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@stanfordnlp stanfordnlp deleted a comment from Copilot AI Oct 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant