Best practices for ranked contest judging of about 18-25 students?

Hi all, hopefully I'm posting in the right place!

I recently judged a local music competition, where the students are eventually given a ranking from 1-18. 1 being the best, 18 being the worst.

I'm concerned that the way the final ranking is produced is actually not extremely accurate, however, the organization has been using the same system for 40 years. Can someone offer some insight into what the best method of computing scores accurately is?

Here are the details: There were 18 students. Two Judges.

Each of the judges was tasked with giving the students a "Score" from 1-100, 1 being the worst possible, 100 being the best possible score.

The way the organization computes the final ranking of the students is by taking the individual scores (1-100) and adding them together. The highest scoring student gets 1st place and so on.

However, it's my impression that because the range of scoring is so large, and the scoring numbers are arbitrary (perhaps the best and worst scores I give are 60-78, and the other judge might score from 10-93), simply adding the scores together is not an accurate way of obtaining a final result. Won't the arbitrary differences in the range and the distribution of scores among different judges cause the influence of each judge to be uneven?

For example, if I wanted to place a certain student in 1st place, overriding the other judge entirely, all I would have to do is give them 100, and give very low scores to everyone else. I imagine that even accidentally, some bizarre results would be produced using this method.

Would it not be more accurate to obtain individual rankings (1-18) from each judge, and then calculate final rankings from those similarly ranged numbers?

Thanks for your help. I'd like to be able to express clearly, the significance of both methods of scoring. I also wonder if there is some other method that I'm overlooking?

Posted Jan 4, 2016, 2:14 PM UTC · edited

25
Loading comments…