Issue: Invalid Session Errors During VAD Closed-Loop Evaluation

[curryqka]

Description

During the closed-loop evaluation using VAD, both run_evaluation_debug.sh and multi-GPU evaluation scripts encounter errors after running for some time. These errors include invalid session messages and failures to destroy actors.

Error Messages

The following errors occur during execution:

ERROR: Invalid session: no stream available with id 3 
ERROR: Invalid session: no stream available with id 7 
ERROR: Invalid session: no stream available with id 5 
ERROR: Invalid session: no stream available with id 9
ERROR: failed to destroy actor 3696 : unable to destroy actor: not found 
ERROR: Invalid session: no stream available with id 10 
ERROR: Invalid session: no stream available with id 6 
ERROR: Invalid session: no stream available with id 8 
ERROR: session 3580901 : error retrieving stream id : End of file 
ERROR: Invalid session: no stream available with id 5 
ERROR: Invalid session: no stream available with id 10 
ERROR: Invalid session: no stream available with id 7 
ERROR: failed to destroy actor 3697 : unable to destroy actor: not found 
ERROR: Invalid session: no stream available with id 8 
ERROR: Invalid session: no stream available with id 4 
ERROR: Invalid session: no stream available with id 6 
ERROR: Invalid session: no stream available with id 9 
ERROR: failed to destroy actor 3698 : unable to destroy actor: not found 
ERROR: Invalid session: no stream available with id 10 
ERROR: Invalid session: no stream available with id 8 
ERROR: Invalid session: no stream available with id 7 
ERROR: Invalid session: no stream available with id 6 
ERROR: Invalid session: no stream available with id 9 
ERROR: session 3580913 : error retrieving stream id : End of file 
ERROR: failed to destroy actor 3699 : unable to destroy actor: not found 
ERROR: Invalid session: no stream available with id 7 
ERROR: Invalid session: no stream available with id 6 
ERROR: Invalid session: no stream available with id 8 
ERROR: Invalid session: no stream available with id 9 
ERROR: failed to destroy actor 3700 : unable to destroy actor: not found 
ERROR: Invalid session: no stream available with id 10 
ERROR: Invalid session: no stream available with id 7 
ERROR: Invalid session: no stream available with id 8 
ERROR: Invalid session: no stream available with id 9 
ERROR: failed to destroy actor 3701 : unable to destroy actor: not found 
ERROR: failed to destroy actor 3702 : unable to destroy actor: not found 
ERROR: failed to destroy actor 3703 : unable to destroy actor: not found 
ERROR: failed to destroy actor 3704 : unable to destroy actor: not found 
ERROR: session 3580924 : error retrieving stream id : End of file 
ERROR: Invalid session: no stream available with id 9 
ERROR: Invalid session: no stream available with id 10 

[jayyoung0802]

Hi @curryqka, it was caused by the simulator crash, stop and resume it.

[newbrains1]

我在运行TCP的评估程序时遇到了相同的问题，我尝试了很多次停止和恢复，但是每次都在route3785的相同位置遇到一样的问题？我的英伟达驱动是550