You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A clear and concise description of what the bug is.
If the test class name contains a forbidden character, the file path will be incorrect and will not be created.
To Reproduce
(not very reliable because it depends on the output of an LLM. But we can force it.)
Steps to reproduce the behavior:
Generate a test on a simple hello-world func main()
Manually update the test class name to add a forbidden character on Windows (I don't know if replicable on Linux)
Run the test
See error
Expected behavior
A clear and concise description of what you expected to happen.
We need to sanitize the filename to avoid forbidden characters for all OS.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
The filename path is generated using the name of the generated class. It is excepted to follow the default naming rule [MethodUnderTest][Scenario]Test by default. This convention should not contain any forbidden characters. But if the LLM doesn't follow the rule (happened to me with a low parameters model), it's a possibility.
The only way to add forbidden characters to the name is with the following syntax which doesn't follow the Kotlin conventions.
class `MyClass("extra")`
To avoid this problem, I see two techniques:
Ask the LLM to not use this syntax for the class (not very reliable)
Simply sanitize the filename in the code. This is far more reliable. And cover the case where a user asks for non-conventional class name.
The text was updated successfully, but these errors were encountered:
The question now is: Where is the best place to sanitize the path?
The easiest place to do it is inside org.jetbrains.research.testspark.tools.TestProcessor.saveGeneratedTest. But this method asks for a filename. So, is it its responsibility to sanitize the filename or the caller?
Because we construct the full path inside this method, I think it's pretty safe to sanitize it inside, too.
Involved Module
Describe the bug
A clear and concise description of what the bug is.
If the test class name contains a forbidden character, the file path will be incorrect and will not be created.
To Reproduce
(not very reliable because it depends on the output of an LLM. But we can force it.)
Steps to reproduce the behavior:
func main()
Expected behavior
A clear and concise description of what you expected to happen.
We need to sanitize the filename to avoid forbidden characters for all OS.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
The filename path is generated using the name of the generated class. It is excepted to follow the default naming rule
[MethodUnderTest][Scenario]Test
by default. This convention should not contain any forbidden characters. But if the LLM doesn't follow the rule (happened to me with a low parameters model), it's a possibility.The only way to add forbidden characters to the name is with the following syntax which doesn't follow the Kotlin conventions.
To avoid this problem, I see two techniques:
The text was updated successfully, but these errors were encountered: