Support Approval Func in BaseTool in AgentChat

[victordibia]

Background

Agents can act via tools. The BaseTool interface in AutoGen provides an excellent scaffold for building tools across the framework. Extensions of BaseTool like FunctionTool, PythonCodeExecutionTool, and others demonstrate the flexibility and value of this tool.

The Approval Challenge and Opportunity

An important capability for a framework is the ability to allow users to "approve" actions before they are "executed". The opportunity here is to build this into the BaseTool library by adding an approval mechanism that maintains backward compatibility.

Proposed Design Sketch [discussion welcome]

1. Add Approval Function to BaseTool

# Add to BaseTool init
def __init__(
    self,
    args_type: Type[ArgsT],
    return_type: Type[ReturnT],
    name: str,
    description: str,
    strict: bool = False,
    approval_func: Optional[Callable[[str, Mapping[str, Any]], Awaitable[ToolApprovalResult]]] = None,
) -> None:
    # ... existing initialization ...
    self._approval_func = approval_func or default_approval

2. Create an Approval Result Model

class ToolApprovalResult(BaseModel):
    """Result of a tool approval request."""
    approved: bool
    reason: Optional[str] = None

# Default approval function that always approves
async def default_approval(tool_name: str, arguments: Mapping[str, Any]) -> ToolApprovalResult:
    return ToolApprovalResult(approved=True)

3. Modify run_json to Check Approval

async def run_json(self, args: Mapping[str, Any], cancellation_token: CancellationToken) -> Any:
    # Approval check
    approval_result = await self._approval_func(self.name, args)
    
    # Log the approval event
    event = ToolCallApprovalEvent(
        tool_name=self.name,
        arguments=dict(args),
        approved=approval_result.approved,
        reason=approval_result.reason
    )
    logger.info(event)
    
    # If not approved, return early
    if not approval_result.approved:
        return f"Tool execution not approved: {approval_result.reason or 'No reason provided'}" 
   
        
    # Execute tool if approved (existing functionality)
    return_value = await self.run(self._args_type.model_validate(args), cancellation_token)
    # ... existing logging ...
    return return_value

4. Add to AssistantAgent

def __init__(
    self,
    # ... existing parameters ...
    tool_approval_func: Optional[Callable[[str, Mapping[str, Any]], Awaitable[ToolApprovalResult]]] = None,
):
    # ... existing initialization ...
    self._tool_approval_func = tool_approval_func
    
    # When processing tools, apply the approval func
    for tool in tools:
        if isinstance(tool, BaseTool) and self._tool_approval_func:
            tool._approval_func = self._tool_approval_func
        elif callable(tool) and self._tool_approval_func:
            # For function tools, wrap with approval
            function_tool._approval_func = self._tool_approval_func

Benefits

1. Maintains backward compatibility
2. Provides consistent approval mechanism across all tools
3. Allows for centralized approval at agent level or per-tool
4. Enables logging of approval decisions

Implementation Notes / Open Questions

• Default behavior should match current behavior (auto-approve)
• Should emit events to notify of approval requests and decisions
• Both BaseTool and AssistantAgent should accept approval functions
• We need to check for side effects (e.g, if reflect_on_tool call , we should skpi this if tool denied)
• Should we add tool_call_approval_func to assistant agent?
• what happens with parrallel tool calls?
• can approval/disapproval be constrained .. e.g,. provide feedback e.g., on poor tool call parameters or let the approval function modify the parameters ?
• How do we pass context to the approval_function? E.g., if the function is soemthign that uses an LLM, we might need to provide plumbing for the context to be passed to the approval func

Thoughts welcome @ekzhu @jackgerrits @husseinmozannar

[husseinmozannar]

Eric @ekzhu and I were just discussing this exactly in the morning.

I like this design, I like passing it to the agents, I can see potential use cases when passed to tools but might make it a bit complicated. In your example you are only using approve in the agent vs the tool, do you think we can get away with approval only for the agent and not the tools?

[victordibia]

do you think we can get away with approval only for the agent and not the tools?

Excellent question. IIUC, the question is if the approval logic should live in the agent or in the tool.

• in agent: assistant agent responsible for evaluating approval func, emiting approval events, etc.
  Con here is that any BaseAgent in agent chat or core that uses BaseTool will need to do this themselves.
• in tool: we have a clean story for "action approval" across the framework (important IMO) and we can write documentation around this. The tool_approval_func just needs to be passed down to the tools. When none is passed, we emit an "ToolCallApprovalResult" where the reason is "auto approved".

can see potential use cases when passed to tools but might make it a bit complicated

What are the complications you had in mind e.g, if they are things we can improve with a better design?

[husseinmozannar]

with the tools approach, suppose the user refuses the tool execution, we would need to hardcode the refusal of action to a specific string like you had "f"Tool execution not approved: {approval_result.reason or 'No reason provided'}"", or will we throw a specific exception? do you add a flag for the tool call result to say refused that the agent can decipher and handle accordingly if needed?

In the case of refusal, do we expect that most agents would still need to have code that handles both refusal or approvals differently, or do you imagine that the basic assistantagent would just add the toolcallrefusal to the chat history?

The main point is whether we will have two pieces of code that handle refusals/approvals in any case in both tool/agent. If we include it in the tool, it should make the agent life easier with how to receive the refusal/approval