“We’re #1!…or are we?”: Analyzing Roller Derby Ranking Systems

By: John “Fragglepuss” Evans

As It currently stands there are a few ranking systems used in Roller Derby. These ranking systems each use their own unique system to come up with their opinion on how good each league is compared to other leagues. There is only one official WFTDA ranking system, but that does not stop others from voicing their opinions on where leagues currently stand. Below is a list of four popular ranking systems used in Derby, with some of the pros and cons associated with them.

Flattrackstats.com – Leagues are constantly ranked against all other leagues and also separated into the four Regions. Each league holds a rating number which ranks them in comparison to all other leagues and is used in predicting the outcome of their future bouts. A mathematical prediction of each bout is made by FTS and leagues gain or lose points based on how they perform compared to the prediction. The prediction is based on each league’s current ranking (according to FTS), home vs. away, and regular or tournament bout. For a full explanation of the mathematical formulation, visit http://flattrackstats.com/about/algorithm/detailed. Long story short, the formulation is based on the Elo rating system, a scoring metric, a prediction algorithm, and home vs. away bias. There are six people that come up with all the information for FTS.

Pros – computer-based rankings mean that every league is ranked using the same formula and takes out personal opinions. Constant ranking updates for current rankings.

Cons – the computer-based rankings are created by someone’s personal opinions on how leagues should be judged. A new league jumps into the rankings after playing one bout and can jump into the very top or the very bottom of the rankings. If the rankings were truly unbiased, flattrackstats and Derbytron would produce the same rankings, which is clearly not the case.

Derbytron.com – Leagues are constantly ranked against all other leagues with no separation of Regions. Each league holds a rating number which ranks them in comparison to all other leagues and is used in predicting the outcome of their future bouts. Each league’s base rating is made using 3 factors: 1- Effective winning percentage using the Pythagorean Expectation, 2- Opponent’s effective winning percentage, and 3- Opponent’s opponent’s effective winning percentage. Derbytron also makes a prediction of each bout’s outcome factoring in each league’s current rating, Where it was played (home vs. away), when it was played (more current bouts are weighted heavier than old bouts), confidence level (combining the number of bouts a league has played with the number of bouts their opponents have played). Each league’s base rating is made using 3 factors: 1- Effective winning percentage using the Pythagorean Expectation, 2- Opponent’s effective winning percentage, and 3- Opponent’s opponent’s effective winning percentage. For further information, check out http://derbytron.com/2012/03/05/updated-methodology/. Derbytron is the person that comes up with all the information for derbytron.com.

Pros – computer-based rankings mean that every league is ranked using the same formula and takes out personal opinions. There are constant updates for up-to-date rankings. In other words, see flattrackstats pros

Cons – the computer-based rankings are created by someone’s personal opinions on how leagues should be judged. If the rankings were truly unbiased, Derbytron and flattrackstats would produce the same rankings, which is clearly not the case. Once again, in other words, see flattrackstats cons

WFTDA – Leagues are ranked solely by the four Regions (West, North Central, South Central, East) once a Quarter. Leagues are voted on by members of their own Region. The votes are weighted and averaged to create a region score, which assigns a team’s ranking for the Quarter. Leagues vote once a Quarter with rankings released the last week of January, April, July, and October. A league must play two sanctioned bouts against any other ranked WFTDA league to be eligible for rankings in any given Quarter. This is the currently the only official factor in determining which teams advance to the playoffs. The top 10 ranked teams in the July rankings from each Region advance to their appropriate Regional Playoff tournament. WFTDA member leagues determine the rankings. For more information, check out http://wftda.com/policies-rankings.

Pros – Multiple opinions used from multiple locations to determine an average league ranking.

Cons – Personal bias unrelated to a league’s performance can factor into ranking leagues. Rankings only updated once every 3 months. It is difficult to rank a league that you have not seen play, therefore a lot of guesswork can factor in, resulting in familiar leagues retaining higher spots and unfamiliar leagues having a difficult time receiving recognition.

DNN Power Rankings – The top 25 leagues in the nation are ranked on a monthly basis. The Derby News Network staff determine the top 25 leagues worldwide each month. These rankings are based off the opinions of the DNN staff and explained in detail each month. It is based on how they think each league would beat every league ranked below them and lose to every league ranked higher than them. See www.derbynewsnetwork.com for more details.

Pros – Multiple opinions to determine an average ranking. Derby media watches a lot of Derby and is able to make educated decisions on rankings.

Cons – Personal bias unrelated to a league’s performance can factor into ranking leagues. Rankings only updated once a month. It is difficult to rank a league that you have not seen play, therefore guesswork can factor in, resulting in familiar leagues staying in the top 25 and unfamiliar leagues having a difficult time getting recognition.

There are pros and cons to every ranking system and although there are four separate Roller Derby ranking systems out there, there are really only two systems, the human ranking system and the mathematical ranking system. To break it down even further, every single ranking system out there boils down to a human ranking system. Even the mathematical rankings are made by people, and the people that formulate these mathematical equations are using their personal opinions on what factors are important to rank a league. As I have mentioned previously, flattrackstats and Derbytron both use mathematical equations to rank leagues, but the factors in the equations are completely different, resulting in two completely different ranking systems.

There is no simple fix to how Roller Derby leagues are ranked. Roller Derby is differs from professional and college sports, in which teams are divided into conferences or divisions and teams play every other team in their conference/division a set amount of times. This shows where every team stands because they all play each other. This is not possible with the 156 WFTDA leagues currently in existence, with new leagues constantly being added. There are simply too many leagues and it would require too much time and travel. Roller Derby is much like college football, there are over 100 teams/leagues competing for the opportunity to be the best. College football has several ranking systems and Roller Derby is getting more ranking systems as time goes on. The advantage that Roller Derby has on college football is that a playoff system is already in place. All that Derby needs to do now is figure out how to accurately rank all these leagues.

I mention my ideas for change in “Time for a Change to the System”. You can read it online at https://fragglepuss.wordpress.com/2012/07/12/time-for-a-change-to-the-system/ .

Advertisements
This entry was posted in Roller Derby and tagged . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s