Serialization with tools does not work properly with non-built-ins

[jmerkow]

What happened?

Describe the bug

When serializing and deserializing an agent that uses a tool implemented as a non-builtin (custom) Python function, the tool does not work properly after deserialization. Specifically, after calling dump_component() and then load_component() on an agent that uses a custom tool (e.g., get_magic_number_tool from my_tools.py), the agent fails to execute the tool as expected.

To Reproduce

Steps to reproduce the behavior:

1. Define a custom tool in my_tools.py thats uses a function:

   # my_tools.py

def get_magic_number():
return 42

def get_magic_number_tool(
description: str = "Gets the magic number!",
) -> str:
return str(get_magic_number())
```

2. Create and run an agent with the tool:

   from my_tools import get_magic_number_tool
   from autogen_agentchat.agents import AssistantAgent
   from autogen_agentchat.ui import Console
   
   magic_number_agent = AssistantAgent(
       name="magic_number_agent",
       description="Returns the magic number using the magic number tool.",
       system_message="You are an assistant that returns the magic number.",
       tools=[get_magic_number_tool],
       model_client=model_client
   )

3. Serialize and deserialize the agent:

   component = magic_number_agent.dump_component()
   up_new = AssistantAgent.load_component(component)
   stream = up_new.run_stream(task="what is the magic number?")
   await Console(stream)

4. Observe that after deserialization, the agent cannot use the custom tool and fails to answer the question.

Output/Error:

---------- user ----------
what is the magic number?
/anaconda/envs/radflow/lib/python3.10/site-packages/autogen_core/_component_config.py:252: UserWarning: 
⚠️  SECURITY WARNING ⚠️
Loading a FunctionTool from config will execute code to import the provided global imports and and function code.
Only load configs from TRUSTED sources to prevent arbitrary code execution.
  instance = component_class._from_config(validated_config)  # type: ignore
---------- magic_number_agent ----------
[FunctionCall(id='call_unQ9yUCFuPOCGlga8jfkfo12', arguments='{}', name='get_magic_number_tool')]
---------- magic_number_agent ----------
[FunctionExecutionResult(content="Error: name 'get_magic_number' is not defined", name='get_magic_number_tool', call_id='call_unQ9yUCFuPOCGlga8jfkfo12', is_error=True)]
---------- magic_number_agent ----------
Error: name 'get_magic_number' is not defined

Expected behavior

---------- user ----------
what is the magic number?
---------- magic_number_agent ----------
[FunctionCall(id='call_VHwXWFAU3zxq7SWL8IQ3tJoJ', arguments='{}', name='get_magic_number_tool')]
---------- magic_number_agent ----------
[FunctionExecutionResult(content='42', name='get_magic_number_tool', call_id='call_VHwXWFAU3zxq7SWL8IQ3tJoJ', is_error=False)]
---------- magic_number_agent ----------
42

I understand why this is happening: The global imports is not being populated, but its still a bug.

Are there any work arounds?

Which packages was the bug in?

Python AgentChat (autogen-agentchat>=0.4.0)

AutoGen library version.

Python 0.5.7

Other library version.

No response

Model used

No response

Model provider

Azure OpenAI

Other model provider

No response

Python version

3.10

.NET version

None

Operating system

None

[ekzhu]

In FunctionTool you can specify the global imports manually: https://microsoft.github.io/autogen/stable/reference/python/autogen_core.tools.html#autogen_core.tools.FunctionTool

[jackgerrits]

Yep, to add to what Eric said, imports are not automatically resolved since they are captured by source when serialized. Use the global_imports field to resolve when creating the tool.

[jmerkow]

Yep, to add to what Eric said, imports are not automatically resolved since they are captured by source when serialized. Use the global_imports field to resolve when creating the tool.

Could you point me to the docs for this part?

[victordibia]

@jmerkow

Can you take a look at the example in ags tools e.g., the image generation tool here?