Skip to content

Conversation

cpsievert
Copy link
Collaborator

@cpsievert cpsievert commented Feb 24, 2025

Closes #1813

This PR adds support for the content of a message provided to ui.Chat() or ui.MarkdownStream() to include Shiny UI, and moreover, if that UI includes output or input bindings, they are bound to the server session.

This means, among other things, you can now display things like data grids or Jupyter Widgets as well as collect user input from things like a select input. Here are a couple basic examples:

Output example
from shiny import render
from shiny.express import ui


ui.page_opts(
    title="Hello output bindings in Chat",
    fillable=True,
    fillable_mobile=True,
)

with ui.hold() as df_ui:

    @render.data_frame
    def df():
        import pandas as pd

        dat = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
        return render.DataGrid(dat)


chat = ui.Chat(
    id="chat",
    messages=[
        ui.TagList(
            "**Hello! Here is a `render.DataGrid()`**",
            df_ui,
        )
    ],
)

chat.ui()


@chat.on_user_submit
async def _(user_input: str):
    await chat.append_message_stream(f"You said: {user_input}")
Input example
from chatlas import ChatOllama

from shiny import reactive
from shiny.express import input, ui

chat_model = ChatOllama(model="llama3.2")


ui.page_opts(
    title="Hello input bindings in Chat",
    fillable=True,
    fillable_mobile=True,
)


welcome = f"""
**Hello! How would you like me to respond?**

{ui.input_select("tone", "", choices=["Happy", "Sad"])}
"""

chat = ui.Chat(
    id="chat",
    messages=[welcome],
)
chat.ui()


chat_model = ChatOllama(model="llama3.2")


@reactive.effect
@reactive.event(input.tone)
def _():
    chat_model.system_prompt = f"""
    You are a terse and {input.tone()} assistant.
    """


@chat.on_user_submit
async def _(user_input: str):
    stream = await chat_model.stream_async(user_input)
    await chat.append_message_stream(stream)

NOTE

@cpsievert cpsievert changed the title Chat shiny bind feat(Chat, MarkdownStream): Support Shiny UI inside of message content Feb 24, 2025
@cpsievert cpsievert marked this pull request as ready for review February 25, 2025 21:13
@cpsievert cpsievert requested a review from gadenbuie February 25, 2025 21:13
Copy link
Collaborator

@gadenbuie gadenbuie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The examples in the PR description show inputs/outputs in initial messages, but it's very hard to see how this feature would be used mid-stream or in the regular course of action with the chat component.

The test_chat_shiny_output test gets a little bit closer by showing that you can pass UI to chat.append_message() or chat.append_message_stream(). That's appropriate for a test where we're proving that the feature works, but it's still not approximating how someone using the chat component would interact with these features.

I wouldn't say the above is a blocker for this PR -- it's something that can be addressed with the documentation you're working on -- but I do think it limits my ability to review the PR.

@cpsievert

This comment was marked as outdated.

@cpsievert

This comment was marked as outdated.

@gadenbuie

This comment was marked as outdated.

@cpsievert

This comment was marked as outdated.

@gadenbuie

This comment was marked as outdated.

@cpsievert

This comment was marked as outdated.

@jeffbryner

This comment was marked as off-topic.

@cpsievert
Copy link
Collaborator Author

@gadenbuie, just curious, were you wanting more time to review before merging this one?

content
Content to display when the UI element is first rendered.
A string of content to display before any streaming occurs. When
`content_type` isn't `"text"`, it may also be UI element(s) such as input
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`content_type` isn't `"text"`, it may also be UI element(s) such as input
`content_type` is Markdown or HTML, it may also be UI element(s) such as input

It's a bit easier to parse this sentence if we say when UI elements are allowed.

I think we probably should update the content_type="semi-markdown" description that currently say "with HTML tags escaped" to clarify that Shiny UI tags aren't escaped. (If I understand correctly and that's true.)

Copy link
Collaborator Author

@cpsievert cpsievert Mar 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to clarify that Shiny UI tags aren't escaped. (If I understand correctly and that's true.)

Shiny UI tags get escaped as well:

from shiny import reactive
from shiny.express import ui

md = ui.MarkdownStream("stream")
md.ui(content_type="semi-markdown")

@reactive.effect
async def _():
    await md.stream([ui.div("Hello"), ui.tags.b("world")])

Copy link
Collaborator

@gadenbuie gadenbuie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! I had just a few small comments, all minor

@cpsievert cpsievert merged commit 9b61375 into main Mar 12, 2025
41 checks passed
@cpsievert cpsievert deleted the chat-shiny-bind branch March 12, 2025 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ui.Chat: Adding Shiny inputs via chat.append_message()
3 participants