Recommend
2 
 Thumb up
 Hide
13 Posts

BoardGameGeek» Forums » BoardGameGeek Related » BGG General

Subject: Could someone please explain the Rankings again? rss

Your Tags: Add tags
Popular Tags: [View All]
Jeff Miller
United States
Ogden
Utah
flag msg tools
badge
Working through the night Jeff fixed the broken link. After being fixed the broken link became Jeff's BGG profile.
Avatar
mbmbmbmbmb
First off, I know none of this matters in the grand scheme of things but please indulge my curiosity...

I understand there's a Bayesian Averaging thing going on that adds something like 250 dummy votes at a 5.5 rating. (The actually numbers matter less then the understood effect.) It makes sense that this would favor games with more ratings since that would lessen the impact of the dummy votes.

What puzzles me is that Arkham Horror, Ranked: 51, with an average 7.63 rating with 7461 voters is Ranked below the following:

Taj Mahal, Rank:49, 7.59 avg, 3604 votes
-- .04 lower avg and half as many votes

Ticket to Ride Europe, Rank:47, 7.59 avg, 7152 votes
-- .04 lower avg and 300 less votes

Samurai, Rank:46, 7.58 avg, 5522 votes
-- .05 lower avg and almost 2,000 less votes (heck, I don't even know why this is ranked higher then TTR:E)


I heard somewhere that there was something in place to help control shill votes or whatever. But looking at the bell-curve for all four games, there doesn't seem to be anything that stands out that would affect one game more then the others.

My last thought was maybe there's some sort of grand-fathering rule, but AH and TTR:E were published in the same year.

If it's just something in the system that isn't updating, calculating right, etc., I really don't care if it gets fixed. I would just like to know... for curiosity sake.

So what is it? What's with the ranking?

...and thank you ahead of time for any insight.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tom Dickson
United States
National City
California
flag msg tools
badge
Avatar
mbmbmbmbmb
My guess that part of the "shill prevention" code is code that slows down how fast a game would change positions or something.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Esa Junttila
Finland
Helsinki
flag msg tools
badge
Avatar
http://www.boardgamegeek.com/thread/193803
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Darren M
Canada
Fort Vermilion
AB
flag msg tools
badge
Avatar
mb
A shill busting algorithm and possibly other filters that Aldie and crew feels are appropriate are applied to the raw ratings to come up with the final "adjusted average" which is used to rank all the games. They keep the exact formulas private I assume so there is less chance of people trying to manipulate the ratings in ways to circumvent these filters.

The BGG ranking system is just one of many possible ranking methods though... and many people have proposed alternatives that are arguably much better in showing "truer" rankings of all the boardgames in the database... keeping in mind of course that ALL ranking systems have biases and none will necessarily be appropriate for any specific individual to base their game purchasing decisions on.
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Steve Duff
Canada
Ottawa
Ontario
flag msg tools
badge
Avatar
mbmbmbmbmb
Certainly it's good to have the shill rating stuff, but there comes a point when it's done it's job.

Having a higher rating and 4000 extra people that think so ought to be more than enough to rank you ahead of another game.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Herb Petro
United States
INDIAN TRAIL
North Carolina
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
You could just use the raw averages is you wish:
http://www.boardgamegeek.com/browse/boardgame?sort=avgrating...
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Seriously, turn off Facebook. You'll be happier.
United States
Riva
Maryland
flag msg tools
Avatar
mbmb

    Worrying about a position or two in a table of thousands of games is not particularly valuable use of your time. I'd submit that when you get down to that level of granularity you're far better off focusing on the individual games at hand and deciding which will appeal more to you personally based on theme, mechanics, and so on.

             Sag.


3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Randy Cox
United States
Clemson
South Carolina
flag msg tools
designer
1024x768 works just fine - Don't Wide the Site!
badge
Missing old BGG
Avatar
mbmbmbmbmb
UnknownParkerBrother wrote:
Certainly it's good to have the shill rating stuff, but there comes a point when it's done it's job.

Having a higher rating and 4000 extra people that think so ought to be more than enough to rank you ahead of another game.

Not if a significant portion of those extra 4000 voters are one-dimensional (e.g. they're AH fanboys or they play only horror themed games). Their votes MAY not count for as much as those who actually play and rate a variety of games. I say "may" because no one knows how the Shill Detector works. And it's not really a "shill" detector. It's finding less meaningful votes, many of which happen to be shill votes.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Randy Cox
United States
Clemson
South Carolina
flag msg tools
designer
1024x768 works just fine - Don't Wide the Site!
badge
Missing old BGG
Avatar
mbmbmbmbmb
Again, I know no inside information, but let's assume that the "rating corrections software" is trying to weed out votes that seem well outside the norm. One easy way to do that is to knock out all votes where someone ranked only a single game (the true fanboy). Or who rated only a few games. We don't know how many Arkham Horror players fall into that group, but I would be willing to bet that it's a much larger percentage of all AH raters than it would be for a more mainstream popular game like, say, Settlers of Catan.

Another method might have to do with some sort of validation or normalization routine. Maybe a person's set of votes are compared to "the norm" to see how far outside "the norm" their votes fall. If they're way out there, maybe such votes are reduced. Maybe they look like shill votes.

Let's compare the uberfans of Arkham Horror to those of Settlers of Catan. If you look at those who rated each game a 10 and find other games that 15% of those raters collectively rated, here are the games, in order of preference, liked by each game's super fans...

AH=================================SOC===========================
Arkham Horror - Dunwich Horror_____Puerto Rico
Arkham Horror - The King in Yellow_Settlers - 5-6 Plr Extension
Arkham Horror - Kingsport Horror___Catan: Cities & Knights
War of the Ring____________________Agricola
Battlestar Galactica ______________Power Grid
Arkham Horror - Curse/Dark Pharaoh_Tigris & Euphrates
Agricola __________________________Cities/Knights - 5-6 Plyr Exp
Twilight Imperium 3rd Edition _____Catan: Seafarers
Race for the Galaxy _______________El Grande
Dominion __________________________Dominion
Pandemic __________________________Age of Steam
Descent: Journeys in the Dark _____Ticket to Ride
Power Grid ________________________Carcassonne
Fury of Dracula ___________________Carcassonne - Inns&Cathedrals
___________________________________Race for the Galaxy
___________________________________Seafarers - 5-6 Plyr Exp
___________________________________Ra
___________________________________The Princes of Florence
___________________________________Memoir '44
___________________________________Pandemic
___________________________________Caylus


I've highlighted common entries in red and greyed out the expansions of each game. You'll notice that what is left is War of the Ring, Battlestar Galactica, Twilight Imperium, Decent, and Fury of Dracula on the Arkham side of the ledger. Compare those to Puerto Rico, Tigris & Euphrate, El Grande, Age of Steam, Ticket to Ride, Carcassonne, Ra, Princes of Florence, Caylus, and Memoir '44 which are very well-liked (8.0 rating or above) by the same proportion of Settlers fanboys.

Now, I expect fans of Arkham to like more of the fantasy/sci-fi. But the fact is that those games, here on BGG, are not as mainstream as the games enjoyed by the fans of Settlers. Does that make some of the Arkham fanboy votes register less than Settlers. Probably so. Because it's hard to say that one person who rates only Arkham, Descent, Fury, and BSG a 10 and thumbs down the more popular games isn't trying to manipulate statistics. That person may just be a super sci-fi fanboy. But by rating in such a manner, it makes them LOOK like a shill.

It would look much better if there were a few more Top 20 games in the AH list. As it stands, it sure looks like fans of AH, for the most part, either a) hate most all of the games BGGers say are "good", b) are ignorant of most of the highest rated BGG games, and/or c) are ignorant of non-sci-fi/fantasy games. It may truly be a case of ignorance and extremely outside-the-mainstream preferences. But some of those, if you were to look, might well appear to be shills. Some of those fans surely rated AH (and expansions) 10 and most run-of-the-mill great games less than 8 (hence, no TTR or El Grande or Ra or Age of Steam on the list). Possibly WAY less than 8. Again, that could still just be ignorance and fanaticism. But it registers as shilling. And, in a way, it is.

So, don't think of it as anti-shill. Think of it as anti-well-rounded and experienced gamer. :)
1 
 Thumb up
0.25
 tip
 Hide
  • [+] Dice rolls
Steve Duff
Canada
Ottawa
Ontario
flag msg tools
badge
Avatar
mbmbmbmbmb
Randy Cox wrote:
It would look much better if there were a few more Top 20 games in the AH list.


There would be, if the system wasn't arbitrarily lowering the rankings of those games.

I dispute the basic concept that someone's opinion is worth less because he mostly likes one kind of game, or someone's opinion is worth more because he likes euros.

And I say that as a dyed in the wool eurogamer, myself.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Randy Cox
United States
Clemson
South Carolina
flag msg tools
designer
1024x768 works just fine - Don't Wide the Site!
badge
Missing old BGG
Avatar
mbmbmbmbmb
I'm not saying that an outlier's ratings SHOULD have less value. I'm just saying that I think they do. I think that a person who plays only three games--whatever those three games are--is more likely to have their ratings dismissed as someone who plays and rates hundreds of games. And niche gamers (the ones who play only 3 games) are much more likely to rate fringe games than to rate only Puerto Rico, Settlers, and Ra.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jeff Miller
United States
Ogden
Utah
flag msg tools
badge
Working through the night Jeff fixed the broken link. After being fixed the broken link became Jeff's BGG profile.
Avatar
mbmbmbmbmb
Randy Cox wrote:
So, don't think of it as anti-shill. Think of it as anti-well-rounded and experienced gamer.

That would go a little ways towards explaining how Samurai is rated higher then TTR:E too since I'm sure there's more less-rounded gamers that rate the TTR series over Samurai.

Makes me wonder what kind of hit all these War Games are taking. (They, I'm guessing, are far more 'exclusive' then Ameritrash.) I've always just chalked it up to less ratings and a greater Bayesian disadvantage.

This is assuming you're really onto something and it makes sense that you are.
thumbsup
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Matthew M
United States
New Haven
Connecticut
flag msg tools
admin
8/8 FREE, PROTECTED
badge
513ers Assemble!
Avatar
mbmbmbmbmb
As you can imagine, we aren't going to divulge exactly what our shill-busting algorithms are doing. However I will say that there is no consideration of genre or any other category affecting weightings. The ranking system treats all games in the database the same.

-MMM
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.