[Agent, LLM] Make sure codeact agent produce message in u/a/u/a order (#3193)

xingyaoww · enyst · web-flow · commit 2e60d25eae16 · 2024-08-02T00:17:53.000+08:00
* make sure codeact agent produce message in u/a/u/a order

* integration tests

* sync message changes to codeact swe

* fix integration tests

---------

Co-authored-by: Engel Nyst &lt;enyst@users.noreply.github.com&gt;
diff --git a/agenthub/codeact_agent/codeact_agent.py b/agenthub/codeact_agent/codeact_agent.py
@@ -222,7 +222,13 @@ def _get_messages(self, state: State) -> list[dict[str, str]]:
 
             # add regular message
             if message:
-                messages.append(message)
+                # handle error if the message is the SAME role as the previous message
+                # litellm.exceptions.BadRequestError: litellm.BadRequestError: OpenAIException - Error code: 400 - {'detail': 'Only supports u/a/u/a/u...'}
+                # there should not have two consecutive messages from the same role
+                if messages and messages[-1]['role'] == message['role']:
+                    messages[-1]['content'] += '\n\n' + message['content']
+                else:
+                    messages.append(message)
 
         # the latest user message is important:
         # we want to remind the agent of the environment constraints
diff --git a/agenthub/codeact_swe_agent/codeact_swe_agent.py b/agenthub/codeact_swe_agent/codeact_swe_agent.py
@@ -173,7 +173,13 @@ def _get_messages(self, state: State) -> list[dict[str, str]]:
 
             # add regular message
             if message:
-                messages.append(message)
+                # handle error if the message is the SAME role as the previous message
+                # litellm.exceptions.BadRequestError: litellm.BadRequestError: OpenAIException - Error code: 400 - {'detail': 'Only supports u/a/u/a/u...'}
+                # there should not have two consecutive messages from the same role
+                if messages and messages[-1]['role'] == message['role']:
+                    messages[-1]['content'] += '\n\n' + message['content']
+                else:
+                    messages.append(message)
 
         # the latest user message is important:
         # we want to remind the agent of the environment constraints
diff --git a/tests/integration/mock/CodeActAgent/test_browse_internet/prompt_001.log b/tests/integration/mock/CodeActAgent/test_browse_internet/prompt_001.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Browse localhost:8000, and tell me the ultimate answer to life. Do not ask me for confirmation at any point.
 
 ENVIRONMENT REMINDER: You have 14 turns left to complete the task. When finished reply with <finish></finish>
diff --git a/tests/integration/mock/CodeActAgent/test_browse_internet/prompt_005.log b/tests/integration/mock/CodeActAgent/test_browse_internet/prompt_005.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Browse localhost:8000, and tell me the ultimate answer to life. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_edits/prompt_001.log b/tests/integration/mock/CodeActAgent/test_edits/prompt_001.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ENVIRONMENT REMINDER: You have 14 turns left to complete the task. When finished reply with <finish></finish>
diff --git a/tests/integration/mock/CodeActAgent/test_edits/prompt_002.log b/tests/integration/mock/CodeActAgent/test_edits/prompt_002.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_edits/prompt_003.log b/tests/integration/mock/CodeActAgent/test_edits/prompt_003.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_edits/prompt_004.log b/tests/integration/mock/CodeActAgent/test_edits/prompt_004.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_edits/prompt_005.log b/tests/integration/mock/CodeActAgent/test_edits/prompt_005.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_ipython/prompt_001.log b/tests/integration/mock/CodeActAgent/test_ipython/prompt_001.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Use Jupyter IPython to write a text file containing 'hello world' to '/workspace/test.txt'. Do not ask me for confirmation at any point.
 
 ENVIRONMENT REMINDER: You have 14 turns left to complete the task. When finished reply with <finish></finish>
diff --git a/tests/integration/mock/CodeActAgent/test_ipython/prompt_002.log b/tests/integration/mock/CodeActAgent/test_ipython/prompt_002.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Use Jupyter IPython to write a text file containing 'hello world' to '/workspace/test.txt'. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_ipython/prompt_003.log b/tests/integration/mock/CodeActAgent/test_ipython/prompt_003.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Use Jupyter IPython to write a text file containing 'hello world' to '/workspace/test.txt'. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_ipython_module/prompt_001.log b/tests/integration/mock/CodeActAgent/test_ipython_module/prompt_001.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Install and import pymsgbox==1.0.9 and print it's version in /workspace/test.txt. Do not ask me for confirmation at any point.
 
 ENVIRONMENT REMINDER: You have 14 turns left to complete the task. When finished reply with <finish></finish>
diff --git a/tests/integration/mock/CodeActAgent/test_ipython_module/prompt_002.log b/tests/integration/mock/CodeActAgent/test_ipython_module/prompt_002.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Install and import pymsgbox==1.0.9 and print it's version in /workspace/test.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_ipython_module/prompt_003.log b/tests/integration/mock/CodeActAgent/test_ipython_module/prompt_003.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Install and import pymsgbox==1.0.9 and print it's version in /workspace/test.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_ipython_module/prompt_004.log b/tests/integration/mock/CodeActAgent/test_ipython_module/prompt_004.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Install and import pymsgbox==1.0.9 and print it's version in /workspace/test.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_write_simple_script/prompt_001.log b/tests/integration/mock/CodeActAgent/test_write_simple_script/prompt_001.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Write a shell script 'hello.sh' that prints 'hello'. Do not ask me for confirmation at any point.
 
 ENVIRONMENT REMINDER: You have 14 turns left to complete the task. When finished reply with <finish></finish>
diff --git a/tests/integration/mock/CodeActAgent/test_write_simple_script/prompt_002.log b/tests/integration/mock/CodeActAgent/test_write_simple_script/prompt_002.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Write a shell script 'hello.sh' that prints 'hello'. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_write_simple_script/prompt_003.log b/tests/integration/mock/CodeActAgent/test_write_simple_script/prompt_003.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Write a shell script 'hello.sh' that prints 'hello'. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActAgent/test_write_simple_script/prompt_004.log b/tests/integration/mock/CodeActAgent/test_write_simple_script/prompt_004.log
@@ -396,8 +396,6 @@ The server is running on port 5000 with PID 126. You can access the list of numb
 
 NOW, LET'S START!
 
-----------
-
 Write a shell script 'hello.sh' that prints 'hello'. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_001.log b/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_001.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ENVIRONMENT REMINDER: You have 14 turns left to complete the task.
diff --git a/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_002.log b/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_002.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_003.log b/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_003.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_004.log b/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_004.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_005.log b/tests/integration/mock/CodeActSWEAgent/test_edits/prompt_005.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Fix typos in bad.txt. Do not ask me for confirmation at any point.
 
 ----------
@@ -601,13 +599,13 @@ open_file("bad.txt")
 ----------
 
 OBSERVATION:
-[File: /workspace/bad.txt (4 lines total)]
-(this is the beginning of the file)
-1|This is a stupid typoo.
-2|Really?
-3|No mor typos!
-4|Enjoy!
-(this is the end of the file)
+[File: /workspace/bad.txt (4 lines total)]
+(this is the beginning of the file)
+1|This is a stupid typoo.
+2|Really?
+3|No mor typos!
+4|Enjoy!
+(this is the end of the file)
 
 
 ----------
@@ -629,12 +627,12 @@ edit_file_by_replace(
 ----------
 
 OBSERVATION:
-[No exact match found in bad.txt for
-```
-This is a stupid typoo.
-No mor typos!
-```
-]
+[No exact match found in bad.txt for
+```
+This is a stupid typoo.
+No mor typos!
+```
+]
 
 
 ----------
@@ -656,14 +654,14 @@ edit_file_by_replace(
 ----------
 
 OBSERVATION:
-[File: /workspace/bad.txt (4 lines total after edit)]
-(this is the beginning of the file)
-1|This is a stupid typo.
-2|Really?
-3|No mor typos!
-4|Enjoy!
-(this is the end of the file)
-[File updated (edited at line 1). Please review the changes and make sure they are correct (correct indentation, no duplicate lines, etc). Edit the file again if necessary.]
+[File: /workspace/bad.txt (4 lines total after edit)]
+(this is the beginning of the file)
+1|This is a stupid typo.
+2|Really?
+3|No mor typos!
+4|Enjoy!
+(this is the end of the file)
+[File updated (edited at line 1). Please review the changes and make sure they are correct (correct indentation, no duplicate lines, etc). Edit the file again if necessary.]
 
 
 ----------
@@ -680,14 +678,14 @@ edit_file_by_replace(
 ----------
 
 OBSERVATION:
-[File: /workspace/bad.txt (4 lines total after edit)]
-(this is the beginning of the file)
-1|This is a stupid typo.
-2|Really?
-3|No more typos!
-4|Enjoy!
-(this is the end of the file)
-[File updated (edited at line 3). Please review the changes and make sure they are correct (correct indentation, no duplicate lines, etc). Edit the file again if necessary.]
+[File: /workspace/bad.txt (4 lines total after edit)]
+(this is the beginning of the file)
+1|This is a stupid typo.
+2|Really?
+3|No more typos!
+4|Enjoy!
+(this is the end of the file)
+[File updated (edited at line 3). Please review the changes and make sure they are correct (correct indentation, no duplicate lines, etc). Edit the file again if necessary.]
 
 
 ENVIRONMENT REMINDER: You have 10 turns left to complete the task.
diff --git a/tests/integration/mock/CodeActSWEAgent/test_ipython/prompt_001.log b/tests/integration/mock/CodeActSWEAgent/test_ipython/prompt_001.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Use Jupyter IPython to write a text file containing 'hello world' to '/workspace/test.txt'. Do not ask me for confirmation at any point.
 
 ENVIRONMENT REMINDER: You have 14 turns left to complete the task.
diff --git a/tests/integration/mock/CodeActSWEAgent/test_ipython/prompt_002.log b/tests/integration/mock/CodeActSWEAgent/test_ipython/prompt_002.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Use Jupyter IPython to write a text file containing 'hello world' to '/workspace/test.txt'. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActSWEAgent/test_ipython_module/prompt_001.log b/tests/integration/mock/CodeActSWEAgent/test_ipython_module/prompt_001.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Install and import pymsgbox==1.0.9 and print it's version in /workspace/test.txt. Do not ask me for confirmation at any point.
 
 ENVIRONMENT REMINDER: You have 14 turns left to complete the task.
diff --git a/tests/integration/mock/CodeActSWEAgent/test_ipython_module/prompt_002.log b/tests/integration/mock/CodeActSWEAgent/test_ipython_module/prompt_002.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Install and import pymsgbox==1.0.9 and print it's version in /workspace/test.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActSWEAgent/test_ipython_module/prompt_003.log b/tests/integration/mock/CodeActSWEAgent/test_ipython_module/prompt_003.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Install and import pymsgbox==1.0.9 and print it's version in /workspace/test.txt. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActSWEAgent/test_write_simple_script/prompt_001.log b/tests/integration/mock/CodeActSWEAgent/test_write_simple_script/prompt_001.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Write a shell script 'hello.sh' that prints 'hello'. Do not ask me for confirmation at any point.
 
 ENVIRONMENT REMINDER: You have 14 turns left to complete the task.
diff --git a/tests/integration/mock/CodeActSWEAgent/test_write_simple_script/prompt_002.log b/tests/integration/mock/CodeActSWEAgent/test_write_simple_script/prompt_002.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Write a shell script 'hello.sh' that prints 'hello'. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActSWEAgent/test_write_simple_script/prompt_003.log b/tests/integration/mock/CodeActSWEAgent/test_write_simple_script/prompt_003.log
@@ -587,8 +587,6 @@ exit
 
 NOW, LET'S START!
 
-----------
-
 Write a shell script 'hello.sh' that prints 'hello'. Do not ask me for confirmation at any point.
 
 ----------
diff --git a/tests/integration/mock/CodeActSWEAgent/test_write_simple_script/prompt_004.log b/tests/integration/mock/CodeActSWEAgent/test_write_simple_script/prompt_004.log
