i want to note that currently Pending Cup 2024 is blocked from displaying results on the contest page until support for this is added (though the results have been announced in other channels and a news post is also ready soon)

@ppy/team-web any chance we could pls get some priority on this? standardized scoring is a requirement for contests official support (see this clause), and having our official system contradict with that is not good.

I'll do it if @venix12 isn't currently on it?

judge X's final score = (judge X's raw score - judge X's average raw score) / judge X's standard deviation

so a negative score is possible?
5 - 6 / 1.2 = -0.83?

yep see https://mappersguild.com/contests/results?contest=66be99c3e1f8c3da9e8a995b for an example

I'll do it if @venix12 isn't currently on it?

don't mind, probably quite quick to do, though busy with other stuffs at the very moment

(@notbakaneko i think that's permission for you to get this done)

Is the standard deviation across all the judge's scores across every category, or per category, because each category can have a different max_value which skews the averages and deviation?

@notbakaneko here's a demo in google sheets https://docs.google.com/spreadsheets/d/1uZS2Zp0tirJHEBc7s6wqSwofoEedg4lCFZRCSWPO9xM/edit?usp=sharing

sum of judge's category scores for an entry = judge's raw score for that entry

both the average and standard deviation is across judge raw scores per entry (where categories are summed together)

the max_values can be different, and they usually add up to 100 (organisers i've worked with define category weights as percentages), but this doesn't matter for the calculations

@venix12 @Walavouchey what's the process for this when it's time to get the scores? (i.e. when/who collects the scores, is there anyone who can see the scores before they're published, etc?)

@notbakaneko contest hosts need visibility on the final results and scoring/comments before they can publish results on their end. judge comments may need inspection before publishing for moderation purposes

current process:

1. fetch judge score data (including comments) once the judging period ends by pinging peppy to run

SELECT `id`, `masked_name` FROM `contest_entries`
WHERE `contest_id` = 'CONTEST_ID';

SELECT `id`, `name`, `max_value` FROM `contest_scoring_categories`
WHERE `contest_id` = 'CONTEST_ID';

SELECT * FROM `contest_judge_votes`
WHERE `contest_entry_id` IN (
	SELECT `id` FROM `contest_entries` WHERE `contest_id` = 'CONTEST_ID'
);

SELECT * FROM `contest_judge_scores`
WHERE `contest_judge_vote_id` IN (
	SELECT `id` FROM `contest_judge_votes` WHERE `contest_entry_id` IN (
		SELECT `id` FROM `contest_entries` WHERE `contest_id` = 'CONTEST_ID'
	)
);

2. calculate the the leaderboard with standardised scoring from the data (i do this with a not-so-pretty script but the linked pr above has this functionality now)
   • ideally these first two steps should be an ASS command or website admin export button
   • results along with all scoring data and comments are sent to the contest hosts
   • idk if site admins can see scores and comments on the contest page at this point already (should later be visible to and exportable by the contest hosts, imo)
3. contest hosts publish results on discord/livestream/news/wiki/etc when they're ready
4. contest page results are made public (whenever standardised scoring is supported)
5. due to Add a way to lock the judging before results are displayed on contest judging system #11393, i'll need to compare the exported data (from when the judging period ended) with contest page results (once they're published) and manually revert any discrepancies (was an issue with kyoku 2024)