Skip to content

langsmith: clarifications to messages format #655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

langsmith: clarifications to messages format #655

wants to merge 5 commits into from

Conversation

tanushree-sharma
Copy link
Contributor

@tanushree-sharma tanushree-sharma commented Sep 26, 2025 •
edited
Loading

Goals

  • be explicit about the message formats we render well
  • show how to use process_inputs and process_outputs to convert messages before they are logged

To-do:

  • Figure out formatting for reasoning, citations, streaming, list of available tools for the model to call

Preview deployment: https://langchain-5e9cc07a-preview-llmcal-1759474889-0b7785c.mintlify.app/langsmith/log-llm-trace

@github-actions github-actions bot added the langsmith For docs changes to LangSmith label Sep 26, 2025
@github-actions GitHub Actions
Copy link

Preview ID generated: preview-llmcal-1758848807-b95b6b2

@github-actions GitHub Actions
Copy link

Preview ID generated: preview-llmcal-1759194510-092ba06

@tanushree-sharma tanushree-sharma changed the title clarifications to llm call formats langsmith: clarifications to messages format Sep 30, 2025
@tanushree-sharma tanushree-sharma marked this pull request as ready for review September 30, 2025 01:14
tanushree-sharma
tanushree-sharma commented Sep 30, 2025
View reviewed changes
@github-actions GitHub Actions
Copy link

Preview ID generated: preview-llmcal-1759195340-f7db8a9

@github-actions GitHub Actions
Copy link

Preview ID generated: preview-llmcal-1759195522-41a72bb

hwchase17
hwchase17 reviewed Sep 30, 2025
View reviewed changes
Copy link
Contributor

@hwchase17 hwchase17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think table of contents can be better (too much under "Tracing a Model with a Custom Input/Output Format") . input/output format should be at a higher level. there should probably be more explicit setions for tool calls and content blocks. O

jacoblee93
jacoblee93 reviewed Sep 30, 2025
View reviewed changes
jacoblee93
jacoblee93 reviewed Sep 30, 2025
View reviewed changes
src/langsmith/log-llm-trace.mdx
class LangSmithOutputs(BaseModel):
"""The output format LangSmith expects."""

def process_inputs(inputs: dict) -> dict:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a real example?

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 3, 2025

Preview ID generated: preview-llmcal-1759474889-0b7785c

katmayb
katmayb approved these changes Oct 3, 2025
View reviewed changes
Copy link
Contributor

@katmayb katmayb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, couple of edits to consider.

src/langsmith/log-llm-trace.mdx
]
chat_model_choices(inputs)
```
<Expandable title="the code">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could leave in a regular, open code block — but your call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jacoblee93 jacoblee93 jacoblee93 left review comments

@hwchase17 hwchase17 hwchase17 left review comments

@katmayb katmayb katmayb approved these changes

@lnhsingh lnhsingh Awaiting requested review from lnhsingh lnhsingh is a code owner

Assignees
No one assigned
Labels
langsmith For docs changes to LangSmith
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@tanushree-sharma @jacoblee93 @hwchase17 @katmayb