Trialing a Temporary Random Sort for Episode Discussion Comments

What is Being Changed?

From February 17 to April 1, all Episode discussion threads will be sorted by random for the first six hours before returning to the best sort. We are testing whether this reduces bias towards early commenters in Episode discussion threads.

This only changes the default sort, so you can change the sort back to best if you wish (either per-thread or by overriding all suggested sorts), just like you can with any other sorting override.

Data & Graphs

This is a graph of how many comments were made per minute per Episode discussion thread in 2025 (truncated to the first 24 hours). This is a zoom in of the first 60 minutes and this is the first 7 days. Looking at prior years (using averages to account for year to year volume differences), we can see that this is remarkably consistent past the first hour.

Looking at karma values, we see a different trend. This is a graph of a comment's average karma based on how many minutes after the thread was posted it was made. It shows all comments made in the first 24 hours of an Episode discussions post's life for the year of 2025. This is a zoom in of the first 60 minutes.

Our Interpretation of the Data

It's clear to us that the comments that eventually end up gaining the most karma are disproportionately made extremely early in an Episode discussion thread's life. In less time than it takes to watch an average episode of anime, the average karma drops almost 50%. And then almost by another 50% by 50 minutes.

That's just the nature of reddit's best sort. The earliest comments start near the top of the thread, which gives them more views, which leads to more upvotes, which leads to more views, and so on. Being early to a thread or getting a few early upvotes to boost your comment usually ends up being enough to snowball into a permanent top spot, regardless of if someone else said it better later.

Why Random?

By setting the default comment sort to random, we break that comment snowballing. Every comment will be shown at the top of the comment list sometimes. Early comments will still have a minor advantage over later comments, as they'll have more opportunities to be at the top of the thread. But while the random sort is active, everyone's comments will be on a more even playing field, hopefully resulting in higher quality comments (even if they're a bit late) bubbling up higher than where they'd normally be once the sort is set to best.

Why Not Keep Random Forever?

While using the random sort does help promote mildly later high quality comments, it also gives higher visibility to lower quality comments. Additionally, the dynamic nature of the random sort means that it's much harder to keep track of what you've already read on a refresh. So the advantage that the random sort gives to comment writers is at the expense of comment readers.

As such, we want to eventually set the comment sort to best as the thread transitions from being writing heavy, to reading heavy. That way, the more equitable mild karma boosts that random sort gives can be snowballed into permanent top spots.

Feedback

You are welcome to voice your feedback and opinions on the trial in this thread for the duration of the trial. We are most interested in how it feels, if you notice any changes in the top karma comments after the temporary sort expires, and if this affects your desire to make longer / higher effort / later comments in the episode discussion threads.

We will monitor the data as the trial is active and at the end, decide on whether we should make these changes permanent, trial other sorts and/or timings, or revert to our old settings.