Skip to content

Commit 8319209

Browse files
authored
docs(perception_online_evaluator): update metrics explanation (#6819)
Signed-off-by: kosuke55 <kosuke.tnp@gmail.com>
1 parent 859776d commit 8319209

File tree

4 files changed

+875
-5
lines changed

4 files changed

+875
-5
lines changed

Diff for: evaluator/perception_online_evaluator/README.md

+30-5
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,20 @@ This module allows for the evaluation of how accurately perception results are g
88

99
## Inner-workings / Algorithms
1010

11-
- Calculates lateral deviation between the predicted path and the actual traveled trajectory.
12-
- Calculates lateral deviation between the smoothed traveled trajectory and the perceived position to evaluate the stability of lateral position recognition.
13-
- Calculates yaw deviation between the smoothed traveled trajectory and the perceived position to evaluate the stability of yaw recognition.
14-
- Calculates yaw rate based on the yaw of the object received in the previous cycle to evaluate the stability of the yaw rate recognition.
11+
The evaluated metrics are as follows:
12+
13+
- predicted_path_deviation
14+
- predicted_path_deviation_variance
15+
- lateral_deviation
16+
- yaw_deviation
17+
- yaw_rate
1518

1619
### Predicted Path Deviation / Predicted Path Deviation Variance
1720

18-
Compare the predicted path of past objects with their actual traveled path to determine the deviation. For each object, calculate the mean distance between the predicted path points and the corresponding points on the actual path, up to the specified time step. In other words, this calculates the Average Displacement Error (ADE). The target object to be evaluated is the object from $T_N$ seconds ago, where $T_N$ is the maximum value of the prediction time horizon $[T_1, T_2, ..., T_N]$.
21+
Compare the predicted path of past objects with their actual traveled path to determine the deviation for **MOVING OBJECTS**. For each object, calculate the mean distance between the predicted path points and the corresponding points on the actual path, up to the specified time step. In other words, this calculates the Average Displacement Error (ADE). The target object to be evaluated is the object from $T_N$ seconds ago, where $T_N$ is the maximum value of the prediction time horizon $[T_1, T_2, ..., T_N]$.
22+
23+
> [!NOTE]
24+
> The object from $T_N$ seconds ago is the target object for all metrics. This is to unify the time of the target object across metrics.
1925
2026
![path_deviation_each_object](./images/path_deviation_each_object.drawio.svg)
2127

@@ -64,6 +70,25 @@ $$
6470

6571
The actual metric name is determined by the object class and time horizon. For example, `predicted_path_deviation_variance_CAR_5.00`
6672

73+
### Lateral Deviation
74+
75+
Calculates lateral deviation between the smoothed traveled trajectory and the perceived position to evaluate the stability of lateral position recognition for **MOVING OBJECTS**. The smoothed traveled trajectory is calculated by applying a centered moving average filter whose window size is specified by the parameter `smoothing_window_size`. The lateral deviation is calculated by comparing the smoothed traveled trajectory with the perceived position of the past object whose timestamp is $T=T_n$ seconds ago. For stopped objects, the smoothed traveled trajectory is unstable, so this metric is not calculated.
76+
77+
![lateral_deviation](./images/lateral_deviation.drawio.svg)
78+
79+
### Yaw Deviation
80+
81+
Calculates the deviation between the recognized yaw angle of an past object and the yaw azimuth angle of the smoothed traveled trajectory for **MOVING OBJECTS**. The smoothed traveled trajectory is calculated by applying a centered moving average filter whose window size is specified by the parameter `smoothing_window_size`. The yaw deviation is calculated by comparing the yaw azimuth angle of smoothed traveled trajectory with the perceived orientation of the past object whose timestamp is $T=T_n$ seconds ago.
82+
For stopped objects, the smoothed traveled trajectory is unstable, so this metric is not calculated.
83+
84+
![yaw_deviation](./images/yaw_deviation.drawio.svg)
85+
86+
### Yaw Rate
87+
88+
Calculates the yaw rate of an object based on the change in yaw angle from the previous time step. It is evaluated for **STATIONARY OBJECTS** and assesses the stability of yaw rate recognition. The yaw rate is calculated by comparing the yaw angle of the past object with the yaw angle of the object received in the previous cycle. Here, t2 is the timestamp that is $T_n$ seconds ago.
89+
90+
![yaw_rate](./images/yaw_rate.drawio.svg)
91+
6792
## Inputs / Outputs
6893

6994
| Name | Type | Description |

Diff for: evaluator/perception_online_evaluator/images/lateral_deviation.drawio.svg

+296
Loading

Diff for: evaluator/perception_online_evaluator/images/yaw_deviation.drawio.svg

+302
Loading

0 commit comments

Comments
 (0)