Recommend
50 
 Thumb up
 Hide
97 Posts
1 , 2 , 3 , 4  Next »   | 

BoardGameGeek» Forums » Gaming Related » General Gaming

Subject: Are the BGG Game Rankings Meaningless? An Experiment..... rss

Your Tags: Add tags
Popular Tags: [View All]
Tony Ackroyd
United Kingdom
Brighton
E Sussex
flag msg tools
badge
Avatar
mbmbmbmbmb
Michael Debije
Netherlands
Eindhoven
The Netherlands
flag msg tools
wrote in his interesting geeklist Opinions from a curmudgeon "Why does ANYONE care what the ranking of a game is? The hue and cry over the rise of Agricola sent people into fits. The rankings here are meaningless to many grognards that browse here, for example. I've seen the statement that any game ranked under 1000 was not worth looking at. What utter rot. There is a world of good games outside the top 100, folks."

I'd like to do an experiment to see how useful the BGG rankings are at predicting how much we like the games in them. If you want to take part then you need to go and look at groups of 25 games and note down how many in that group you like or would like to try. Then come back and complete the poll below.

The Top 25 games
The games ranked 101-125
The games ranked 501-525
The games ranked 1001-1025
The games ranked 2001-2025
The games ranked 3001-3025
The games ranked 4001-4025
The games ranked 5001-5025

Poll: How many games did you like or want to try?
Out of each of these 25 games
  0-5 6-10 11-15 16-20 21-25
The Top 25
The games ranked 101-125
The games ranked 501-525
The games ranked 1001-1025
The games ranked 2001-2025
The games ranked 3001-3025
The games ranked 4001-4025
The games ranked 5001-5025
      345 answers
Poll created by 1000rpm

Poll: And did you have any zeros?
Were there any where there were no games you wanted to play?
  Zero 1+
The Top 25
The games ranked 101-125
The games ranked 501-525
The games ranked 1001-1025
The games ranked 2001-2025
The games ranked 3001-3025
The games ranked 4001-4025
The games ranked 5001-5025
      290 answers
Poll created by 1000rpm


Update: This has been a lot more popular than I'd expected, so lets have a closer look at the Top 1000 as that seems to be where the action is:

The games ranked 1-100 -> Look at them in groups of 25.
The games ranked 101-125
The games ranked 201-225
The games ranked 301-325
The games ranked 401-425
The games ranked 501-525
The games ranked 601-625
The games ranked 701-725
The games ranked 801-825
The games ranked 901-925
Poll
How many games do you like or would like to try out of each group of 25 games?
  0 1-5 6-10 11-15 16-20 21-25
The Top 25
Games ranked 26-50
Games ranked 51-75
Games ranked 76-100
Games ranked 101-125
Games ranked 201-225
Games ranked 301-325
Games ranked 401-425
Games ranked 501-525
Games ranked 601-625
Games ranked 701-725
Games ranked 801-825
Games ranked 901-925
      73 answers
Poll created by 1000rpm
16 
 Thumb up
1.00
 tip
 Hide
  • [+] Dice rolls
Nick McElveen
United States
Alexandria
Virginia
flag msg tools
badge
Mmmmmm..... Beer. Glug glug glug....
Avatar
mbmbmbmbmb
1000rpm wrote:
I'd like to do an experiment to see how useful the BGG rankings are at predicting how much we like the games in them. If you want to take part then you need to go and look at groups of 25 games and note down how many in that group you like or would like to try. Then come back and complete the poll below.


Isn't this circular? The reason that I wanted to try a lot of the games in the top 100 was because they were in the top 100, and once I tried them I liked them, for the most part.
22 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tony Ackroyd
United Kingdom
Brighton
E Sussex
flag msg tools
badge
Avatar
mbmbmbmbmb
nickmcelveen wrote:
1000rpm wrote:
I'd like to do an experiment to see how useful the BGG rankings are at predicting how much we like the games in them. If you want to take part then you need to go and look at groups of 25 games and note down how many in that group you like or would like to try. Then come back and complete the poll below.


Isn't this circular? The reason that I wanted to try a lot of the games in the top 100 was because they were in the top 100, and once I tried them I liked them, for the most part.

For you the Rankings obviously aren't meaningless. For others they may be. I'm trying to get an objective assessment. Who knows if it is the right approach, I thought it would be interesting anyway....
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
United States
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
They're definitely a cool feature of the site and clearly people find them important but I'd agree that the reality is such that they really don't mean much. Mass market popularity rarely equates (with 100% accuracy) to quality.

Each to his own, right?
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tony Ackroyd
United Kingdom
Brighton
E Sussex
flag msg tools
badge
Avatar
mbmbmbmbmb
wytefang wrote:
They're definitely a cool feature of the site and clearly people find them important but I'd agree that the reality is such that they really don't mean much. Mass market popularity rarely equates (with 100% accuracy) to quality.

Each to his own, right?

I agree with the sentiment, but I think we are talking about niche market popularity rather than mass market, so that may be quite different.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Dean Glencross
Australia
Geelong
VIC
flag msg tools
0 should have been an option. It would be interesting to see how many that voted 0-5 actually liked or wanted to play any of the games.

I know that I had 3 0s but would like to see how many others had similar thoughts on those sections of games.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
United States
flag msg tools
Avatar
mbmbmbmbmb
Until Aldie removes the bayesian crap that penalizes smaller print run games, they will remain completely useless.
9 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Krzysztof Zięba
Poland
Kraków
flag msg tools
designer
badge
Check out my first published board game: In the Name of Odin!
Avatar
mbmbmbmbmb
Lortarg wrote:
0 should have been an option. It would be interesting to see how many that voted 0-5 actually liked or wanted to play any of the games.

I know that I had 3 0s but would like to see how many others had similar thoughts on those sections of games.


Yeah, seconded. 0-5 is not a very good option, because basically I got 0, 1 or 2 every time I have chosen 0-5. Also, I barely made it to 6-10 in the second part - I got 6.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
brian
United States
Cedar Lake
Indiana
flag msg tools
badge
Avatar
mbmbmb
Works for me. I wanted to try nearly all the top 25 and reduced by 1 column as I went down each row (skipped the 11-15 column though) until I hit the 0-5 column. Then I stayed there.

After hitting 0 in the 3000, and 4000 row, I got close to moving back up in the 5000 row. Had 3 that I liked or wanted to try!
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tony Ackroyd
United Kingdom
Brighton
E Sussex
flag msg tools
badge
Avatar
mbmbmbmbmb
Lortarg wrote:
0 should have been an option. It would be interesting to see how many that voted 0-5 actually liked or wanted to play any of the games.

I know that I had 3 0s but would like to see how many others had similar thoughts on those sections of games.

Good point.
Second poll added (because polls can't be edited)
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Gabe Alvaro
United States
Berkeley
California
flag msg tools
badge
Avatar
mbmbmbmbmb
I think it's all about cause and effect. When you consider the ranking system, try considering each separately. On the one hand you have the rankings as an effect. And on the other hand you have the rankings as a cause.

If you look at the rankings as an effect of having been voted to their positions then there is much to argue about including what people do and the "bayesian crap".

But if you simply omit any concern for them as an effect of anything (just presume someone makes them up completely) and only consider them as a cause unto themselves, then the meaning is quite clear. The meaning of the rankings is to increase interest in each game. It's only natural that top ranked games are going to get the most interest. That is simply the human pyschology of a ranked list. Therefore in terms of the rankings as a cause unto themselves, their meaning is huge.


2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Anthony Sr
United States
Burnsville
Minnesota
flag msg tools
Avatar
mbmbmbmbmb
Lortarg wrote:
0 should have been an option. It would be interesting to see how many that voted 0-5 actually liked or wanted to play any of the games.

I know that I had 3 0s but would like to see how many others had similar thoughts on those sections of games.

I agree. I personally had about:

10 in the top 25
6 in the 100 - 125
5 in the 500 - 525
3 in the 1000 - 1025
0 in the 2000 - 2025
0 in the 3000 - 3025
0 in the 4000 - 4025
1 in the 5000 - 5025

So only the 1-5, & 6-10 were relevant to me.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ryan Newell
Canada
Regina
Saskatchewan
flag msg tools
Avatar
mbmb
Your experiment is obviously going to show a preference for the higher ranked games.

A) You're sampling from the population who made the rankings the way they are in the first place.

B) Your sample probably gets most of its gaming info from this site. The games that aren't so highly rated get almost no buzz here. I wasn't aware of most of the games in the 1001-1025 range so I can't really say I want to play them. Doesn't mean I wouldn't like them. On the other hand, I'd like to try something like Dominion because I've seen mention of it so many times. Doesn't mean I'd like it.
14 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Werner Bär
Germany
Karlsruhe
Baden
flag msg tools
badge
Avatar
mbmbmbmbmb

A funny coincidence: All games i've rated in the 3001+, 4001+ and 5001+ blocks, i rated a 4.

(i've seen other games on these pages, farther down, that i rated differently - most 2-5, but a childs game in there i've given a 7)
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Chris Ferejohn
United States
Mountain View
California
flag msg tools
badge
Pitying fools as hard as I can...
Avatar
mbmbmbmbmb
Zaphod wrote:
Until Aldie removes the bayesian crap that penalizes smaller print run games, they will remain completely useless.


Umm, no. So the highest ranked game will have one rating of 10? No thanks.
27 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Steve Duff
Canada
Ottawa
Ontario
flag msg tools
badge
Avatar
mbmbmbmbmb
My numbers were:
13
14
7
4
3
1
1
0

Now, this isn't fair to a lot of the later categories of course, because I didn't spend 10 minutes to investigate every game and read the rules etc, like I've done for a lot of other games. So only those low rated games that I had previously run across and done that investigation of got a vote here.

Personally, I do have a lot of games in the multi-thousand ratings I do have great interest in, it's just that they didn't fall into the 25 we're looking at.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Daniel Chen
United States
Green Bay
Wisconsin
flag msg tools
designer
This looks tasty...
badge
Yuck!
Avatar
mbmbmbmbmb
Ratio of top 100 games liked and dislike = 62:38
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Scott S.
United States
Lawrence
Kansas
flag msg tools
badge
Avatar
mbmbmbmbmb
Zaphod wrote:
Until Aldie removes the bayesian crap that penalizes smaller print run games, they will remain completely useless.


I agree that the Bayesian should be removed, but there would need to be a minimum number of votes for it to count in the rankings to prevent the scenario Chris mentioned. It would have to be in the hundreds for sure, but I don't know an optimal number. Personally, I am much more interested in average rating than the weighted BGG score, so long as the game has a significant number of ratings.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bruce Murphy
Australia
Pyrmont
NSW
flag msg tools
badge
Avatar
mbmbmbmbmb
The ratings are helpful, but they're not definitive for all the reasons that people have pointed out, different tastes or groups, different numbers of raters etc.

However, I'm fairly sure that there are more interesting games in 1-100 than there are in 1001-1100 for almost everyone interested in mainstream games. If you're a wargamer, then looking at the rankings in the purely wargame filtered view might be of interest, although I feel wargamers are more likely to dive deep into a particular series or system and not be interested in playing dozens of games the way Eurogamers might.

B>
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
J.L. Robert
United States
Sherman Oaks
California
flag msg tools
designer
badge
Follow me for wargames!
Avatar
mbmbmbmbmb
My tallies were:

8
8
13
9
5
4
1
0

There's a definite curve for me. But, then again, the bottom of the rankings is rife with small, obscure, or foreign games with little to no chance of me ever seeing them, much less playing them.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
United States
flag msg tools
Avatar
mbmbmbmbmb
cferejohn wrote:
Zaphod wrote:
Until Aldie removes the bayesian crap that penalizes smaller print run games, they will remain completely useless.


Umm, no. So the highest ranked game will have one rating of 10? No thanks.


Thats not how it would work all, but thanks for the "insightful" comment. There of course would be a required minimum number of ratings before the entry became statistically significant, assuming something resembling a normal distribution.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Thomas Taylor
United States
Castro Valley
California
flag msg tools
badge
Avatar
mbmbmbmbmb
Yes they are meaningless.

Reviews are meaningful.

The end.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Paul DeStefano
United States
Long Island
New York
flag msg tools
designer
badge
It's a Zendrum. www.zendrum.com
Avatar
mbmbmbmbmb
This is way to much work to check out the lists, so I'll just vote they're meaningless.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Patrick Korner
Canada
Coquitlam
British Columbia
flag msg tools
badge
Stop poking me! Ow! I mean it! That hurts!
Avatar
mbmbmbmbmb
Ratings value depends on the granularity you're willing to accept.

Is the #7 game 'better' than the #17 game? Depends on your tastes and play preferences. It might be, then again it might not.

Is the #7 game 'better' than the #700 game? Probably, but there's still a healthy chance you will feel differently.

Is the #7 game 'better' than the #7000 game? Almost certainly.

I generally discount smaller differences - for all the reasons listed above. I also take some exception to the site's obvious bias towards recent releases - it makes the ratings suspect and feel like more of a popularity contest. But that's ok, since I know how to research and find games that interest me in other ways that just scanning the high-ranked games.

pk
8 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Drew Spencer
United States
Tucson
Arizona
flag msg tools
badge
Avatar
mbmbmbmbmb
The people claiming the rankings are meaningless seem to be saying that a game's rank does not correlate well with their personal valuation of it. That doesn't make them meaningless; at most it means that their tastes are somewhat out of the mainstream for BGG members who rank games. Now that they know that, they can ignore the rankings. That's surely not the same as them being meaningless.

The rankings provide interesting statistical information. That's almost undeniable, (except maybe the word "interesting") so I don't see how anyone can think they're totally meaningless. The only wrench in the statistics is that different people seem to mean different things by the same ratings (a 7 means something different to me than to someone else for example), so the rankings are only reduced in their meaningfulness to the degree that people do not follow the guidelines for ratings.

I guess my point is that people who are so disappointed in the rankings' lack of meaningfulness are inflating what the rankings are supposed to be meaning.
8 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
1 , 2 , 3 , 4  Next »   | 
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.