Recommend
5 
 Thumb up
 Hide
16 Posts

BoardGameGeek» Forums » BoardGameGeek Related » BGG General

Subject: Better living through statistics rss

Your Tags: Add tags
Popular Tags: [View All]
Corey Butler
United States
Marshall
Minnesota
flag msg tools
badge
Avatar
mbmbmbmbmb
This is a followup to my recent post advocating that we change the rating system to a widely discussed and frequently preferred alternative...

http://www.boardgamegeek.com/thread/445353

I'm not going to rehash that discussion here, but I'd like to talk about how the thread quickly turned to the perennial question of whether or not the game ratings on BGG mean anything. As I stated there, my personal ratings are highly correlated with BGG ratings, and this means that the latter are useful for predicting the former. To pursue that line a little further, I offer the following concrete examples.

I played three new games this month and rated them. To the extent that BGG ratings are useful, I should be able to predict (or postdict) my ratings on the basis of previous data. Consequently, I entered all my other ratings along with the corresponding BGG average ratings and calculated the regression equation:

Y = bX + a,

Where Y is the predicted rating for a given game, X is the BGG average rating for that game, b is the slope of the regression line, and a is the Y intercept (a constant). The equation I came up with was:

Y = 1.32X - 2.56

How well does this equation predict my three new ratings from this month? The first new game I played was Through the Ages, which has a BGG rating of 8.33. According to the equation, my predicted rating is 8.44 and my actual rating was 9. My second new game was Martin Wallace's new release, Automobile. This game has a BGG rating of 7.83, a predicted rating of 7.78, and my actual rating was 8. My third new rating was for Arkham Horror, which is rated 7.62 on BGG, predicted at 7.50 from the equation, and actually rated 6 by me. Arkham was a bit of a disappointment, both statistically and as a gaming experience.

So how well did the statistics work? Are they worth considering when checking out new games? The prediction was close for Through the Ages and Automobile, but a full 1.5 points off for Arkham Horror. Maybe I need to give the game another chance. In any event, the average difference between the prediction and the actual rating was .76, which isn't bad for a test of only 3 games. The accuracy of the prediction will only go up with the addition of more games, so overall I should do pretty well, certainly within a point.

If you don't want to do regression, consider that simply predicting my rating on the basis of the BGG average was nearly as good. The average difference between my rating and the BGG ratings of these three games was .82. In practical terms, I am confident that I my rating will be plus or minus one point from the BGG average more often than not. I think that is useful information to know when it comes to what games I should spend my hard earned cash on.

2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
.
United States
flag msg tools
badge
Avatar
I don't get it. I don't know any statistics, so maybe I'm just dense. But you're saying that based on the BGG rating, I should be able to predict my own personal rating?

Your rating should be plus/minus 1 from the BGG average. That's great. But what about people whose tastes don't follow the rest of the crowd?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Corey Butler
United States
Marshall
Minnesota
flag msg tools
badge
Avatar
mbmbmbmbmb
ElAdoranSureshot wrote:
I don't get it. I don't know any statistics, so maybe I'm just dense. But you're saying that based on the BGG rating, I should be able to predict my own personal rating?

Your rating should be plus/minus 1 from the BGG average. That's great. But what about people whose tastes don't follow the rest of the crowd?


But the crowd knows best. Haven't you read The Wisdom of Crowds by James Surowiecki? Anyway what I'm saying is that you should be able to predict your future ratings for games you haven't played yet by using BGG ratings. Not perfectly of course, but pretty well for most games and most people.

On the other hand, people who's tastes tend to be considerably different from the mainstream should take this advice with a grain of salt.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Seitz
United States
Glen Allen
VA
flag msg tools
badge
Like water spilled on the ground, which cannot be recovered, so we must die. But God does not take away life; instead, he devises ways so that a banished person may not remain estranged from him. 2 Sam 14:14
Avatar
mbmbmbmbmb
That's a majorly flawed premise you got there.

Sure, there may be some correlation between My rating and BGG rating, but there's no causal relationship (except maybe for the weak-minded), and the correlation is not strong enough to use as a satisfying predictor.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Corey Butler
United States
Marshall
Minnesota
flag msg tools
badge
Avatar
mbmbmbmbmb
out4blood wrote:
That's a majorly flawed premise you got there.

Sure, there may be some correlation between My rating and BGG rating, but there's no causal relationship (except maybe for the weak-minded), and the correlation is not strong enough to use as a satisfying predictor.


The correlation between your ratings and the BGG averages can be calculated, so there is no need for debate or speculation. See this thread...

http://www.boardgamegeek.com/thread/372896

The higher the correlation, the better your prediction. And the causality issue is entirely irrelevant. Who can tell what multiple factors cause people to like and dislike games? From a practical point of view, it's irrelevant. All I need is a good correlation to make my prediction.

As with everything, your mileage may vary.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jonathan Morton
Sweden
Vällingby
Stockholm
flag msg tools
Avatar
out4blood wrote:
the correlation is not strong enough to use as a satisfying predictor.


Interestingly, your correlation to the BGG ratings is sky-high at .77.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Seitz
United States
Glen Allen
VA
flag msg tools
badge
Like water spilled on the ground, which cannot be recovered, so we must die. But God does not take away life; instead, he devises ways so that a banished person may not remain estranged from him. 2 Sam 14:14
Avatar
mbmbmbmbmb
Jonny5 wrote:
out4blood wrote:
the correlation is not strong enough to use as a satisfying predictor.


Interestingly, your correlation to the BGG ratings is sky-high at .77.

Perhaps I am one of the weak-minded. or perhaps you're ignoring the relevant data for the games that I have NOT rated.

This is probably a function of how I/we choose to a) play the games we do, and b) play games sufficiently to rate them.

There are many games in the top 100 I have never played, primarily because I know from observing them in our group or reading rules that they would not be for me. Out of the top 100, I've played and rated less than half. That is not because I don't have access to them, because I do. It's because I know that they are not for me. But if I went through the exercise to play them enough and then rate them, then my rating for those games would be much lower than the average BGG rating.

But I normally do not choose to play games that I know I won't enjoy.

Generally, I select games to play I already know I am going to be favorably disposed towards, and I avoid those games that I know I am not going to like. This has little to do with ratings, and everything to do with the type of game it is. Now, if it's true that other Geeks like the same games I do, and thus also rate it well, then perhaps our ratings for those games are correlated, but that does not mean the BGG rating will be an overall good predictor.

Edited to expand the point.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jonathan Morton
Sweden
Vällingby
Stockholm
flag msg tools
Avatar
out4blood wrote:
Perhaps I am weak-minded. I still don't find it a useful predictor for me personally.


Suppose you and I are going to play one of my traditional Euro games (not a lot of theme or randomness). I tell you I have 3 to choose from. You haven't played any of them. They're all recommended for 2 players, they have weights between 2.5 and 3.5, and all have a nominal length of 90 minutes. Game A is ranked #80 on BGG, Game B is #180, and Game C is #280 (hypothetically - I don't know what games are currently sitting at those ranks).

Are you telling me that absent other information, you'd choose Game B or Game C? Your .77 correlation to BGG ratings is saying that you're absolutely far more likely to prefer Game A. It's not a matter of being a sheep, it's the wisdom of crowds. Game A isn't ranked higher by a few random people, it's ranked higher by hundreds of people who really know board games.

On the other hand, if I told you that they weren't all Euro games - let's say Game A was a dungeon crawl, Game B was an perfect information abstract, and Game C was a worker placement game - Then you could safely throw the rankings out the window and just go with your preferred genre.

1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Seitz
United States
Glen Allen
VA
flag msg tools
badge
Like water spilled on the ground, which cannot be recovered, so we must die. But God does not take away life; instead, he devises ways so that a banished person may not remain estranged from him. 2 Sam 14:14
Avatar
mbmbmbmbmb
Quote:
On the other hand, if I told you that they weren't all Euro games - let's say Game A was a dungeon crawl, Game B was an perfect information abstract, and Game C was a worker placement game - Then you could safely throw the rankings out the window and just go with your preferred genre.

This.

I don't select games purely on rankings. Do you really use that as your selection tool?

There is a rating pre-screen that happens because I choose to NOT play certain games. Looking at correlation data for only the games I have rated ignores pertinent data.

In your first example, if the games were all in the genre that I enjoy playing, then I might select the highest rated one to play first, and it's entirely possible that I might also rate that one the highest, but that's not what you're modeling because you're ignoring the genre selection.

You should consider including mkgray's gamer categories into your predictive model.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jonathan Morton
Sweden
Vällingby
Stockholm
flag msg tools
Avatar
out4blood wrote:
I don't select games purely on rankings. Do you really use that as your selection tool?


One of the tools, absolutely. Luckily I'm not limited to a single tool.

If I were limited to a single tool, it would be genre. I love Carrom, RoboRally, Commands & Colors: Ancients, but in my case those are all exceptions from their genres. For the most part I'm just another Eurogamer. This is the pre-screen you're talking about.

But what if I was only allowed just one more tool? I think you'd be hard pressed to do better than overall rank.

Or, from another angle: suppose I felt like "broadening my horizons", ignoring my usual pre-screen and trying a pure Ameritrash game. Would I be more likely to enjoy an AT game that ranks #30 or one that ranks #300?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Seitz
United States
Glen Allen
VA
flag msg tools
badge
Like water spilled on the ground, which cannot be recovered, so we must die. But God does not take away life; instead, he devises ways so that a banished person may not remain estranged from him. 2 Sam 14:14
Avatar
mbmbmbmbmb
Jonny5 wrote:
out4blood wrote:
I don't select games purely on rankings. Do you really use that as your selection tool?


Or, from another angle: suppose I felt like "broadening my horizons", ignoring my usual pre-screen and trying a pure Ameritrash game. Would I be more likely to enjoy an AT game that ranks #30 or one that ranks #300?

LOL. Thanks for proving my point for me.

Sure, if you are "broadening your horizons" you probably try the "highest rated" game that you would otherwise not like, but do you really think that the BGG rating for a top 100 wargame (as an example) is going to predict YOUR rating for it? Particularly if YOU hate wargames?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Randall Bart
United States
Winnetka
California
flag msg tools
designer
Baseball been bery bery good to me
badge
This is a picture of a published game designer
Avatar
mbmbmbmbmb
out4blood wrote:
That's a majorly flawed premise you got there.

I am not sure there is a premise at all.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bill Eldard
United States
Burke
Virginia
flag msg tools
mbmbmbmbmb
shotokanguy wrote:
To pursue that line a little further, I offer the following concrete examples.

I played three new games this month and rated them. To the extent that BGG ratings are useful, I should be able to predict (or postdict) my ratings on the basis of previous data. Consequently, I entered all my other ratings along with the corresponding BGG average ratings and calculated the regression equation:

Y = bX + a,

Where Y is the predicted rating for a given game, X is the BGG average rating for that game, b is the slope of the regression line, and a is the Y intercept (a constant). The equation I came up with was:

Y = 1.32X - 2.56


My head hurts. zombie

I believe it was Samuel Clemens (Mark Twain) who said, "There are three kinds of lies: lies, damned lies, and statistics."
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Seitz
United States
Glen Allen
VA
flag msg tools
badge
Like water spilled on the ground, which cannot be recovered, so we must die. But God does not take away life; instead, he devises ways so that a banished person may not remain estranged from him. 2 Sam 14:14
Avatar
mbmbmbmbmb
Eldard wrote:
My head hurts. zombie

I believe it was Samuel Clemens (Mark Twain) who said, "There are three kinds of lies: lies, damned lies, and statistics."

It was Benjamin Disraeli, originally. MT was quoting him.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Randall Bart
United States
Winnetka
California
flag msg tools
designer
Baseball been bery bery good to me
badge
This is a picture of a published game designer
Avatar
mbmbmbmbmb
out4blood wrote:
Eldard wrote:
My head hurts. zombie

I believe it was Samuel Clemens (Mark Twain) who said, "There are three kinds of lies: lies, damned lies, and statistics."

It was Benjamin Disraeli, originally. MT was quoting him.

Twain attributes it to Disraeli, but there no record of the line before Twain quoted it.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Seitz
United States
Glen Allen
VA
flag msg tools
badge
Like water spilled on the ground, which cannot be recovered, so we must die. But God does not take away life; instead, he devises ways so that a banished person may not remain estranged from him. 2 Sam 14:14
Avatar
mbmbmbmbmb
Barticus88 wrote:
out4blood wrote:
Eldard wrote:
My head hurts. zombie

I believe it was Samuel Clemens (Mark Twain) who said, "There are three kinds of lies: lies, damned lies, and statistics."

It was Benjamin Disraeli, originally. MT was quoting him.

Twain attributes it to Disraeli, but there no record of the line before Twain quoted it.

Maybe he was *GASP* lying!
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.