Add standardised scoring for judged contests #12014

notbakaneko · 2025-03-17T14:33:05Z

the option is "is_score_standardised": true.
I'm not sure how/when the scoring should be calculated (interop/admin page, etc), so I added a a command contest:calculate-scores for now.

store judge's standard deviation so it doesn't need to get recalculated every time
order by standardised score
better display in results?
add something to trigger calculation

Walavouchey · 2025-03-17T15:29:13Z

resources/lang/en/contest.php

+        'score_std' => 'Standardised Score',
        'total_score' => 'total score',


this one is shown with Score not total score

Walavouchey · 2025-03-17T15:30:36Z

app/helpers.php

+    $mean = array_sum($values) / $size;
+
+    return [
+        sqrt(array_sum(array_map(fn ($value) => pow($value - $mean, 2), $values)) / $size),


looks correct

Walavouchey · 2025-03-17T15:34:18Z

app/Models/ContestJudge.php

+
+    public function stdDev()
+    {
+        // TODO: treat missing scores as 0?


all judges are supposed to judge all entries and they're poked if they haven't
default of 0 could make sense if it's to indicate that something is wrong, though ideally we'd know by counting the number of votes

nanaya · 2025-03-24T04:26:00Z

app/Console/Commands/ContestCalculateScores.php

+{
+    protected $signature = 'contest:calculate-scores {contestId}';
+
+    protected $description = 'Cleans up expired chat keep alives.';


nice description

nanaya · 2025-03-25T07:18:49Z

app/Models/ContestEntry.php

+        return $query;
+    }
+
+    public function scopeWithScore(Builder $query, array $options): Builder


shouldn't this accept Contest model instead? So the isset($options...) and all other craps don't need to be reimplemented (apparently that's how it's initially done? but then changed a few times until this one but there's no clear reason in the commit messages)

nanaya · 2025-03-25T07:48:41Z

app/Models/ContestJudgeScore.php

+            ContestJudgeVote
+                ::where('user_id', $user->getKey())
+                ->whereIn('contest_entry_id', $contest->entries()->select('id'))
+            ->select('id')


that's a weird indent

nanaya · 2025-03-25T07:52:32Z

app/helpers.php

@@ -965,6 +965,17 @@ function ujs_redirect($url, $status = 200)
    }
 }

+function std_dev(array $values): array
+{
+    $size = count($values);


can probably use a minimum value of 1

nanaya · 2025-03-26T01:39:16Z

database/migrations/2025_03_21_000001_add_std_dev_to_contest_judges copy.php

+            $table->double('mean')->nullable();
+            $table->double('std_dev')->nullable();


can't these just be stored into variable during calculation?

something like this in contest calculateScores (or probably more like calculateStdScores)

(also added explicit count option for stddev calculation)

public function calculateScores(): void { $entryCount = $this->entries()->count(); foreach ($this->contestJudges as $judge) { $votesQuery = ContestJudgeVote ::where('user_id', $judge->user_id) ->whereHas('entry', fn ($q) => $q->where('contest_id', $this->getKey())); $entryTotalScores = ContestJudgeScore ::whereIn('contest_judge_vote_id', (clone $votesQuery)->select('id')) ->groupBy('contest_judge_vote_id') ->selectRaw('contest_judge_vote_id, SUM(value) AS total') ->get() ->keyBy('contest_judge_vote_id'); [$stdDev, $mean] = std_dev($entryTotalScores->pluck('total')->all(), $entryCount); foreach ($votesQuery->get() as $vote) { $entryTotalScore = $entryTotalScores[$vote->getKey()]?->total ?? 0; $vote->update([ 'total_score_std' => $stdDev === 0.0 ? 0 : ($entryTotalScore - $mean) / $stdDev, ]); } } }

nanaya · 2025-03-26T03:27:18Z

resources/js/contest-judge-results/header.tsx

@@ -29,6 +29,7 @@ export default class Header extends React.PureComponent<Props> {

  render() {
    const totalScore = `${this.props.entry.results.votes}/${this.props.contest.max_total_score}`;
+    const totalScoreStd = this.props.entry.judge_votes.reduce((total, vote) => (vote.total_score_std ?? NaN) + total, 0);


shouldn't the default when missing be 0 instead?

...or maybe just actually include and use the this.props.entry.results.score_std value

nanaya · 2025-03-26T03:27:29Z

resources/js/contest-judge-results/header.tsx

+            <ValueDisplay
+              label={trans('contest.judge_results.total_score_std')}
+              modifiers='judge-results'
+              value={totalScoreStd.toFixed(2)}


formatNumber

nanaya · 2025-03-26T03:29:32Z

resources/css/bem/contest-judge-results-scores.less

@@ -4,14 +4,15 @@
 .contest-judge-results-scores {
  display: grid;
  align-items: center;
-  grid-gap: 5px;
+  gap: 5px;
  // category-name value
  grid-template-columns: minmax(auto, 120px) auto;


unrelated but this probably should be auto 1fr otherwise this happens:

nanaya · 2025-03-26T03:36:59Z

resources/js/contest-voting/entry.coffee

relativeVotePercentage should be adjusted to be something like 100 * (entry.results.score_std - min_score_std) / (max_score_std - min_score_std)

nanaya · 2025-03-26T03:45:27Z

resources/js/contest-judge-results/vote.tsx

            <div key={score.id} className='contest-judge-results-scores__row'>
              <div className='contest-judge-results-scores__col'>
-                <div className='contest-judge-entry__description-icon' title={category?.description}>
+                <div className='contest-judge-entry__description-icon' title={category.description}>


unrelated but the class doesn't really do anything useful apart of being a random element of an entirely different block

notbakaneko added the area:contests label Mar 17, 2025

notbakaneko self-assigned this Mar 17, 2025

Walavouchey reviewed Mar 17, 2025

View reviewed changes

notbakaneko force-pushed the feature/contents-score-std branch from 6a991b2 to 95f49e3 Compare March 18, 2025 19:26

notbakaneko marked this pull request as ready for review March 21, 2025 13:34

notbakaneko added 19 commits March 24, 2025 06:44

add standard deviation calculation helper

b12526f

add standardized score to score

f9aae4f

render standardised score

f92871e

update to using transformer class explicitly

5934746

split into functions

e820d60

move to scope

6a34b72

also move to scope

81e441c

pre-calculate standardised scores

6020767

pass flag into scope instead of Contest

6fcaa0c

include and order by calculated standardised score

49eae94

can just use withSum

7ab824b

show on same line

341aa29

null category makes no sense

7affd2f

add a gap

079a75c

unneeded

efa47b1

show total standardised score

cc9643e

add migrations

a89a47f

add command to calculate scores and stddev

a489e9f

handle case if std dev is some how 0

691584b

notbakaneko force-pushed the feature/contents-score-std branch from f7d14e6 to 691584b Compare March 23, 2025 21:44

Walavouchey mentioned this pull request Mar 24, 2025

Add an option for standardized scoring in contest judging system #11390

Open

simpler query (also applies soft deletes global scope)

44ef869

notbakaneko force-pushed the feature/contents-score-std branch from 8eaba8b to 44ef869 Compare March 25, 2025 10:18

notbakaneko added 2 commits March 25, 2025 19:19

doesn't need to be qualified?

aae1c60

lint

748ca54

nanaya requested changes Mar 26, 2025

View reviewed changes

notbakaneko added 6 commits March 26, 2025 13:02

nice migration name

a0d074f

just string

f6b9b45

use formatNumber

c873dce

remove unused leftover from testing

bc6e675

fix copypasta description

e6431a5

not really useful

21631a4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add standardised scoring for judged contests #12014

Add standardised scoring for judged contests #12014

notbakaneko commented Mar 17, 2025 •

edited

Loading

Walavouchey Mar 17, 2025

notbakaneko Mar 18, 2025 •

edited

Loading

Walavouchey Mar 17, 2025

Walavouchey Mar 17, 2025

nanaya Mar 24, 2025

nanaya Mar 25, 2025

nanaya Mar 25, 2025

nanaya Mar 25, 2025

nanaya Mar 26, 2025

nanaya Mar 26, 2025

nanaya Mar 26, 2025

nanaya Mar 26, 2025

nanaya Mar 26, 2025

nanaya Mar 26, 2025

		'score_std' => 'Standardised Score',
		'total_score' => 'total score',

		$table->double('mean')->nullable();
		$table->double('std_dev')->nullable();

Add standardised scoring for judged contests #12014

Are you sure you want to change the base?

Add standardised scoring for judged contests #12014

Conversation

notbakaneko commented Mar 17, 2025 • edited Loading

Choose a reason for hiding this comment

notbakaneko Mar 18, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

notbakaneko commented Mar 17, 2025 •

edited

Loading

notbakaneko Mar 18, 2025 •

edited

Loading