Recommend
1 
 Thumb up
 Hide
11 Posts

BoardGameGeek» Forums » Gaming Related » General Gaming

Subject: Rating oddity rss

Your Tags: Add tags
Popular Tags: ratings_episode_999,999 [+] [View All]
AJ Cooper
msg tools
Avatar
mbmbmb
I noticed something odd on the ranks. Arkham Horror: The Card Game has both a higher average user rating, and more total votes than either Great Western Trail or Mechs vs. Minions. Yet the latter two both have a higher geek rating, and hence a higher rank. How does this work?

There may be other examples but I didn't look for them.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Derry Salewski
United States
Augusta
Maine
flag msg tools
badge
I'm only happy when it rains...
Avatar
mbmbmbmbmb
hidden math.

(that's kinda broken for any useful purpose these days anyway.)
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Rex Stites
United States
Lawrence
Kansas
flag msg tools
Avatar
mbmbmbmbmb
The standard deviation for Arkham is slightly higher. 1.32 to 1.19 (Great Western Trail) to 1.29 (Mechs).

Perhaps that figures into the ratings calculation.
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Oliver Dienz
United States
Shelburne
Vermont
flag msg tools
mbmbmbmbmb
rstites25 wrote:
The standard deviation for Arkham is slightly higher. 1.32 to 1.19 (Great Western Trail) to 1.29 (Mechs).

Perhaps that figures into the ratings calculation.

Yep, when looking at the ratings breakdown for Arkham versus GWT for example it shows that the former has a more even distribution of 8, 9, and 10 votes while GWT has a clearer peak between 8 and 9. Thus, the algorithm probably assumes that the average rating of GWT is closer to the "true" rating than the one of Arkham. In the end, the differences are so marginal that it should not really matter. I have not seen any user yet that rates his/her games down to the third decimal. Thus, showing the average that way assumes a level of precision that the underlying data do not provide. My math prof would not have agreed. devil
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Chuck Harrison
United States
Beavercreek
OH
flag msg tools
mbmbmbmbmb
It's the shillbusting algorithm. While we don't know exactly how it works, we know it ignores some ratings that it suspects are fake.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Maarten D. de Jong
Netherlands
Zaandam
flag msg tools
badge
Avatar
There is NO factoring in of standard deviation of any kind. Nor is there any weighing of user rates. What we're seeing is the shill buster algorithm which decides whether or not to trust a user based on quantities which are kept secret on purpoase. I presume these to be things like login times, user activity, overall user rating profile. Do not assume that only users with low rates get excluded: removal happens at all levels.

For the raw average everything is left in place. For the geek rate determining rank 'suspect' rates are silently excluded.

There is some evidence to assume that the algorithm deciding which users are suspect is only run once a month.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Robert Stewart
United Kingdom
Newcastle-upon-Tyne
flag msg tools
mbmbmbmbmb
odie73 wrote:
I have not seen any user yet that rates his/her games down to the third decimal. Thus, showing the average that way assumes a level of precision that the underlying data do not provide. My math prof would not have agreed. devil


With sufficient quantity of data, the justified precision of statistical measures of that data is greater than the precision of individual data points.

Over large samples, errors tend to cancel out.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Daniel Corban
Canada
Newmarket
Ontario
flag msg tools
badge
Avatar
mbmbmbmbmb
Dude. The games in question differ in calculated rating by a microscopic 0.012 points. I don't think even the most anal users here bother to rate their games to the thousandth decimal place.

In other words, the top 100 games are all great. 😎
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ryan Keane
United States
Medford
Massachusetts
flag msg tools
Avatar
mbmb
odie73 wrote:
rstites25 wrote:
The standard deviation for Arkham is slightly higher. 1.32 to 1.19 (Great Western Trail) to 1.29 (Mechs).

Perhaps that figures into the ratings calculation.

Yep, when looking at the ratings breakdown for Arkham versus GWT for example it shows that the former has a more even distribution of 8, 9, and 10 votes while GWT has a clearer peak between 8 and 9. Thus, the algorithm probably assumes that the average rating of GWT is closer to the "true" rating than the one of Arkham. In the end, the differences are so marginal that it should not really matter. I have not seen any user yet that rates his/her games down to the third decimal. Thus, showing the average that way assumes a level of precision that the underlying data do not provide. My math prof would not have agreed. :devil:


Great point. I have argued previously that the ranks should be the same for games that are essentially tied. Since except for a very small minority of users, the most they use is 1 decimal point, they should be treated as rounded off to 1 decimal for ranking. E.g. Twilight Struggle and Terra Mystica are tied for #3 at 8.2. SW: Rebellion, Gloomhaven, and Scythe are tied for #5 at 8.1. But that would probably generate more threads teasing out the ranks than threads like this one. I am saddened by how much energy I waste talking about BGG ratings. Stop smoking the crack!
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Det var bara en hake...
Sweden
flag msg tools
Help, I'm being held prisoner in an overtext typing facility! I don't have much time, they could find out at any m
badge
I'm that weirdo whose number of badges sold prior to yesterday Bail Organa is keeping track of
Avatar
mbmbmbmb
Tineren wrote:
It's the shillbusting algorithm. While we don't know exactly how it works, we know it ignores some ratings that it suspects are fake.


The best evidence suggests it cuts off low and high ratings based on the standard deviation, and nothing else.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
marc lecours
Canada
ottawa
ontario
flag msg tools
mbmbmbmb
Alternative reality math !!!
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.