Improve Observability ( LLMEvents and other System Events) in AutoGen Studio

[victordibia]

Currently, AGS shows messages exchanged by Agents (these are only the messages explicitly yielded in the agent/team implementation).
In many cases there are LLMCallEvents that are not necessarily yielded by the agents/teams (e.g., speaker selection), or even the yielded events are a subset of the actual LLMCallEvents.

As a developer tool, it is important that AGS can let users make sense of underlying system behaviour. This can also double as data collection infrastructure.

This PR is to track efforts to show all events that occur as a team is run.

Implementation

• Enable streaming of events to front end during run, enable storage to DB
• Show a filterable log of all Events in AGS UI.

[vladtsit]

I would like to support @victordibia with this issue. I recently started my journey with AutoGen and was excited about the potential of using AutoGen Studio to build collaborative AI agents without coding.

I began developing a selector team with LLM agents and MultimodalWebSurfer. However, I encountered challenges in debugging the agents' behavior due to limitations in message logging within the studio. While I can see the agents' outputs, I cannot access the underlying LLM calls or selector calls, making it difficult to understand their decision-making process.

Additionally, debugging the MultimodalWebSurfer has been particularly challenging. It performs actions while browsing web pages but encounters errors that are not visible in the logs. Without insight into these errors, troubleshooting the agent's behavior becomes very difficult.

It would be immensely helpful if more detailed logging and debugging tools could be provided to improve visibility into LLM interactions, selector calls, and the internal workings of MultimodalWebSurfer.

[ekzhu]

I think this is a very important issue. Let's enable a configuration to specify the log destination from AGS frontend?

For filtering, processing and visualization of logs, Do you think we should create our own solution, or we should consider leaning into existing tools.

[victordibia]

I was thinking of starting with just showing what is in LLMCallEvents for now. WDYT?

autogen/python/packages/autogen-core/src/autogen_core/logging.py

Line 9 in 51b601b

class LLMCallEvent:

[ekzhu]

Yes. Sounds good I think this can already be very valuable.