add tool_call_summary_msg_format_fct and test (microsoft#6460)

## Why are these changes needed?

This change introduces support for dynamic formatting of tool call
summary messages by allowing a user-defined
`tool_call_summary_format_fct`. Instead of relying solely on a static
string template, this function enables runtime generation of summary
messages based on the specific tool call and its result. This provides
greater flexibility and cleaner integration without introducing any
breaking changes.

### My Use Case / Problem

In my use case, I needed concise summaries for successful tool calls and
detailed messages for failures. The existing static summary string
didn't allow conditional formatting, which led to overly verbose success
messages or inconsistent failure outputs. This change allows customizing
summaries per result type, solving that limitation cleanly.

## Related issue number

Closes microsoft#6426

## Checks

- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.

---------

Co-authored-by: Chris Wieczorek <Chris.Wieczorek@iav.de>
Co-authored-by: EeS <chiyoung.song@motov.co.kr>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Mehrsa Golestaneh <mehrsa.golestaneh@gmail.com>
Co-authored-by: Mehrsa Golestaneh <mgolestaneh@microsoft.com>
Co-authored-by: Zhenyu <81767213+Dormiveglia-elf@users.noreply.github.com>

Author: 7 people

python/packages/autogen-agentchat/src/autogen_agentchat/agents/_assistant_agent.py

@@ -131,17 +131,26 @@ class AssistantAgent(BaseChatAgent, Component[AssistantAgentConfig]):
 
     * If the model returns no tool call, then the response is immediately returned as a :class:`~autogen_agentchat.messages.TextMessage` or a :class:`~autogen_agentchat.messages.StructuredMessage` (when using structured output) in :attr:`~autogen_agentchat.base.Response.chat_message`.
     * When the model returns tool calls, they will be executed right away:
-        - When `reflect_on_tool_use` is False, the tool call results are returned as a :class:`~autogen_agentchat.messages.ToolCallSummaryMessage` in :attr:`~autogen_agentchat.base.Response.chat_message`. `tool_call_summary_format` can be used to customize the tool call summary.
+        - When `reflect_on_tool_use` is False, the tool call results are returned as a :class:`~autogen_agentchat.messages.ToolCallSummaryMessage` in :attr:`~autogen_agentchat.base.Response.chat_message`. You can customise the summary with either a static format string (`tool_call_summary_format`) **or** a callable (`tool_call_summary_formatter`); the callable is evaluated once per tool call.
         - When `reflect_on_tool_use` is True, the another model inference is made using the tool calls and results, and final response is returned as a :class:`~autogen_agentchat.messages.TextMessage` or a :class:`~autogen_agentchat.messages.StructuredMessage` (when using structured output) in :attr:`~autogen_agentchat.base.Response.chat_message`.
         - `reflect_on_tool_use` is set to `True` by default when `output_content_type` is set.
         - `reflect_on_tool_use` is set to `False` by default when `output_content_type` is not set.
     * If the model returns multiple tool calls, they will be executed concurrently. To disable parallel tool calls you need to configure the model client. For example, set `parallel_tool_calls=False` for :class:`~autogen_ext.models.openai.OpenAIChatCompletionClient` and :class:`~autogen_ext.models.openai.AzureOpenAIChatCompletionClient`.
 
     .. tip::
-        By default, the tool call results are returned as response when tool calls are made.
-        So it is recommended to pay attention to the formatting of the tools return values,
-        especially if another agent is expecting them in a specific format.
-        Use `tool_call_summary_format` to customize the tool call summary, if needed.
+
+        By default, the tool call results are returned as the response when tool
+        calls are made, so pay close attention to how the tools’ return values
+        are formatted—especially if another agent expects a specific schema.
+
+        * Use **`tool_call_summary_format`** for a simple static template.
+        * Use **`tool_call_summary_formatter`** for full programmatic control
+          (e.g., “hide large success payloads, show full details on error”).
+
+        *Note*: `tool_call_summary_formatter` is **not serializable** and will
+        be ignored when an agent is loaded from, or exported to, YAML/JSON
+        configuration files.
+
 
     **Hand off behavior:**
 
@@ -199,13 +208,22 @@ class AssistantAgent(BaseChatAgent, Component[AssistantAgentConfig]):
             If this is set, the agent will respond with a :class:`~autogen_agentchat.messages.StructuredMessage` instead of a :class:`~autogen_agentchat.messages.TextMessage`
             in the final response, unless `reflect_on_tool_use` is `False` and a tool call is made.
         output_content_type_format (str | None, optional): (Experimental) The format string used for the content of a :class:`~autogen_agentchat.messages.StructuredMessage` response.
-        tool_call_summary_format (str, optional): The format string used to create the content for a :class:`~autogen_agentchat.messages.ToolCallSummaryMessage` response.
-            The format string is used to format the tool call summary for every tool call result.
-            Defaults to "{result}".
-            When `reflect_on_tool_use` is `False`, a concatenation of all the tool call summaries, separated by a new line character ('\\n')
-            will be returned as the response.
-            Available variables: `{tool_name}`, `{arguments}`, `{result}`.
-            For example, `"{tool_name}: {result}"` will create a summary like `"tool_name: result"`.
+        tool_call_summary_format (str, optional): Static format string applied to each tool call result when composing the :class:`~autogen_agentchat.messages.ToolCallSummaryMessage`.
+            Defaults to ``"{result}"``. Ignored if `tool_call_summary_formatter` is provided. When `reflect_on_tool_use` is ``False``, the summaries for all tool
+            calls are concatenated with a newline ('\\n') and returned as the response.  Placeholders available in the template:
+            `{tool_name}`, `{arguments}`, `{result}`, `{is_error}`.
+        tool_call_summary_formatter (Callable[[FunctionCall, FunctionExecutionResult], str] | None, optional):
+            Callable that receives the ``FunctionCall`` and its ``FunctionExecutionResult`` and returns the summary string.
+            Overrides `tool_call_summary_format` when supplied and allows conditional logic — for example, emitting static string like
+            ``"Tool FooBar executed successfully."`` on success and a full payload (including all passed arguments etc.) only on failure.
+
+            **Limitation**: The callable is *not serializable*; values provided via YAML/JSON configs are ignored.
+
+    .. note::
+
+        `tool_call_summary_formatter` is intended for in-code use only. It cannot currently be saved or restored via
+        configuration files.
+
         memory (Sequence[Memory] | None, optional): The memory store to use for the agent. Defaults to `None`.
         metadata (Dict[str, str] | None, optional): Optional metadata for tracking.
 
@@ -652,6 +670,7 @@ def __init__(
         model_client_stream: bool = False,
         reflect_on_tool_use: bool | None = None,
         tool_call_summary_format: str = "{result}",
+        tool_call_summary_formatter: Callable[[FunctionCall, FunctionExecutionResult], str] | None = None,
         output_content_type: type[BaseModel] | None = None,
         output_content_type_format: str | None = None,
         memory: Sequence[Memory] | None = None,
@@ -756,6 +775,7 @@ def __init__(
                 stacklevel=2,
             )
         self._tool_call_summary_format = tool_call_summary_format
+        self._tool_call_summary_formatter = tool_call_summary_formatter
         self._is_running = False
 
     @property
@@ -803,6 +823,7 @@ async def on_messages_stream(
         model_client_stream = self._model_client_stream
         reflect_on_tool_use = self._reflect_on_tool_use
         tool_call_summary_format = self._tool_call_summary_format
+        tool_call_summary_formatter = self._tool_call_summary_formatter
         output_content_type = self._output_content_type
         format_string = self._output_content_type_format
 
@@ -873,6 +894,7 @@ async def on_messages_stream(
             model_client_stream=model_client_stream,
             reflect_on_tool_use=reflect_on_tool_use,
             tool_call_summary_format=tool_call_summary_format,
+            tool_call_summary_formatter=tool_call_summary_formatter,
             output_content_type=output_content_type,
             format_string=format_string,
         ):
@@ -976,6 +998,7 @@ async def _process_model_result(
         model_client_stream: bool,
         reflect_on_tool_use: bool,
         tool_call_summary_format: str,
+        tool_call_summary_formatter: Callable[[FunctionCall, FunctionExecutionResult], str] | None,
         output_content_type: type[BaseModel] | None,
         format_string: str | None = None,
     ) -> AsyncGenerator[BaseAgentEvent | BaseChatMessage | Response, None]:
@@ -1078,6 +1101,7 @@ async def _process_model_result(
                 inner_messages=inner_messages,
                 handoffs=handoffs,
                 tool_call_summary_format=tool_call_summary_format,
+                tool_call_summary_formatter=tool_call_summary_formatter,
                 agent_name=agent_name,
             )
 
@@ -1232,22 +1256,30 @@ def _summarize_tool_use(
         inner_messages: List[BaseAgentEvent | BaseChatMessage],
         handoffs: Dict[str, HandoffBase],
         tool_call_summary_format: str,
+        tool_call_summary_formatter: Callable[[FunctionCall, FunctionExecutionResult], str] | None,
         agent_name: str,
     ) -> Response:
         """
         If reflect_on_tool_use=False, create a summary message of all tool calls.
         """
         # Filter out calls which were actually handoffs
         normal_tool_calls = [(call, result) for call, result in executed_calls_and_results if call.name not in handoffs]
-        tool_call_summaries: List[str] = []
-        for tool_call, tool_call_result in normal_tool_calls:
-            tool_call_summaries.append(
-                tool_call_summary_format.format(
-                    tool_name=tool_call.name,
-                    arguments=tool_call.arguments,
-                    result=tool_call_result.content,
-                )
+
+        def default_tool_call_summary_formatter(call: FunctionCall, result: FunctionExecutionResult) -> str:
+            return tool_call_summary_format
+
+        summary_formatter = tool_call_summary_formatter or default_tool_call_summary_formatter
+
+        tool_call_summaries = [
+            summary_formatter(call, result).format(
+                tool_name=call.name,
+                arguments=call.arguments,
+                result=result.content,
+                is_error=result.is_error,
             )
+            for call, result in normal_tool_calls
+        ]
+
         tool_call_summary = "\n".join(tool_call_summaries)
         return Response(
             chat_message=ToolCallSummaryMessage(

python/packages/autogen-agentchat/tests/test_assistant_agent.py

@@ -32,7 +32,7 @@
     SystemMessage,
     UserMessage,
 )
-from autogen_core.models._model_client import ModelFamily
+from autogen_core.models._model_client import ModelFamily, ModelInfo
 from autogen_core.tools import BaseTool, FunctionTool, StaticWorkbench
 from autogen_ext.models.openai import OpenAIChatCompletionClient
 from autogen_ext.models.replay import ReplayChatCompletionClient
@@ -56,10 +56,68 @@ async def _fail_function(input: str) -> str:
     return "fail"
 
 
+async def _throw_function(input: str) -> str:
+    raise ValueError("Helpful debugging information what went wrong.")
+
+
 async def _echo_function(input: str) -> str:
     return input
 
 
+@pytest.fixture
+def model_info_all_capabilities() -> ModelInfo:
+    return {
+        "function_calling": True,
+        "vision": True,
+        "json_output": True,
+        "family": ModelFamily.GPT_4O,
+        "structured_output": True,
+    }
+
+
+@pytest.mark.asyncio
+async def test_run_with_tool_call_summary_format_function(model_info_all_capabilities: ModelInfo) -> None:
+    model_client = ReplayChatCompletionClient(
+        [
+            CreateResult(
+                finish_reason="function_calls",
+                content=[
+                    FunctionCall(id="1", arguments=json.dumps({"input": "task"}), name="_pass_function"),
+                    FunctionCall(id="2", arguments=json.dumps({"input": "task"}), name="_throw_function"),
+                ],
+                usage=RequestUsage(prompt_tokens=10, completion_tokens=5),
+                thought="Calling pass and fail function",
+                cached=False,
+            ),
+        ],
+        model_info=model_info_all_capabilities,
+    )
+
+    def conditional_string_templates(function_call: FunctionCall, function_call_result: FunctionExecutionResult) -> str:
+        if not function_call_result.is_error:
+            return "SUCCESS: {tool_name} with {arguments}"
+
+        else:
+            return "FAILURE: {result}"
+
+    agent = AssistantAgent(
+        "tool_use_agent",
+        model_client=model_client,
+        tools=[_pass_function, _throw_function],
+        tool_call_summary_formatter=conditional_string_templates,
+    )
+    result = await agent.run(task="task")
+
+    first_tool_call_summary = next((x for x in result.messages if isinstance(x, ToolCallSummaryMessage)), None)
+    if first_tool_call_summary is None:
+        raise AssertionError("Expected a ToolCallSummaryMessage but found none")
+
+    assert (
+        first_tool_call_summary.content
+        == 'SUCCESS: _pass_function with {"input": "task"}\nFAILURE: Helpful debugging information what went wrong.'
+    )
+
+
 @pytest.mark.asyncio
 async def test_run_with_tools(monkeypatch: pytest.MonkeyPatch) -> None:
     model_client = ReplayChatCompletionClient(