Fix integration between classwise wrapper and metric tracker #6363

Job	Run time
check-diff / eval-diff	13s
pytester (ubuntu-22.04, 3.10, 2.0.1)	6m 28s
pytester (ubuntu-22.04, 3.10, 2.1.2)	5m 33s
pytester (ubuntu-22.04, 3.10, 2.2.2)	6m 14s
pytester (ubuntu-22.04, 3.10, 2.3.1)	6m 4s
pytester (ubuntu-22.04, 3.10, 2.4.1)	5m 53s
pytester (ubuntu-22.04, 3.10, 2.5.0)	5m 28s
pytester (ubuntu-20.04, 3.9, 2.0.1, oldest)	5m 45s
pytester (ubuntu-22.04, 3.11, 2.4.1)	5m 39s
pytester (ubuntu-22.04, 3.12, 2.5.0)	6m 39s
pytester (macOS-13, 3.10, 2.0.1)	11m 39s
pytester (macOS-14, 3.10, 2.0.1)	6m 7s
pytester (macOS-14, 3.12, 2.6.0)	6m 13s
pytester (windows-2022, 3.10, 2.0.1)	13m 12s
pytester (windows-2022, 3.12, 2.6.0)	13m 57s
testing-guardian	1s
merge-pkg-artifacts / merge-caches	6s
	1h 45m 11s

Provide feedback