2020 Methodology

When a new poll debuts, and the results aren’t exactly what’s expected, people are quick to doubt its integrity. This is especially the case when the specifics of how the poll works is unknown. As a result, each year a page detailing the methodology and limitations of Vote Coasters will be made available.

Data Cleaning

Before running the poll for the first time the data must go through a significant amount of cleaning. The aim of this process is to remove suspicious rankings with the intention of increasing the accuracy of the poll. All users that ranked 0 or 1 roller coaster are removed. Users that rank roller coasters in a suspicious manner, e.g. all of the default search coasters at the top of their list, are also removed from the poll.

To ensure high quality rankings, Vote Coasters also removes all roller coasters from the poll that had been ranked by less than 5 users. Furthermore, each comparison between two roller coasters has to be featured in at least 3 rankings. Finally, roller coasters require at least 100 comparisons against other roller coasters to be ranked. All rides that didn’t meet these requirements are removed from the poll entirely. The results are then re-calculated, ensuring the poll was only based on rides that met the minimum requirements.

Ranking Process

The community is only permitted to rank roller coasters they’ve actually ridden. This way each roller coaster’s position in the final results will be as truthful and accurate as possible. By making it easy for many people to contribute their lists, Vote Coasters is able to accumulate a large sample which represents everyone.

Once the community has voted, the numbers are crunched. Our method involves directly comparing the rank of two individual roller coasters across all of the submitted lists. As Vote Coasters makes direct comparisons between individual roller coasters, the poll is not a popularity contest. Even obscure roller coasters that few people have had the chance to ride yet can do well!

An Example:

To explain this system in more detail, let’s take a look at four randomly generated example lists:

In the case of Maverick, it would be set up against El Toro to see which one was ranked higher than the other more times across all the submitted lists. This process is repeated for every single ride listed in the poll. For each of these one-on-one battles, Maverick would be rewarded either a win, a loss, or a draw. The individual matches only occur in lists that have both rides ranked.

Let’s look at this example in more detail. Maverick is paired up against El Toro. For every list where Maverick is ranked above El Toro, Maverick will be assigned a point and vice versa:

Maverick (3) - (1) El Toro - based on the example lists above

If Maverick is paired up against all the mentioned roller coasters in the example lists, the results would be as follows:   

Maverick (3) - (1) El Toro
Maverick (1) - (2) Skyrush
Maverick (1) - (3) Twisted Timbers
Maverick (2) - (1) Fury 325

Based on this example set of results, Maverick would be assigned 2 wins and 2 loses, resulting in a win rate of 50%. This would continue for all the other roller coasters, producing a win rate for each. Win rates would then be used to inform the final results of the poll. The roller coaster with the highest win rate would take the number one spot.

Limitations

Unlike real world surveys or polls, Vote Coasters is not constricted to a balanced sample of voters. Though this allows as many people to take part as possible, it also means the poll has innate biases. In an ideal world, the spread of people taking part in Vote Coasters would perhaps follow the spread of roller coaster around the world. If Germany has 4.4% of the world’s roller coasters, 4.4% of people taking part in Vote Coasters would be from Germany. Unfortunately, this is not the case. Multiple countries out-perform this balance, including the USA and the UK.

As a result

Future improvements

Currently we have no plans to improve the methodology of the poll in the future.

More people from a wider selection of the world. Let us know