$18.00
GeekGold Bonus for All Supporters: 131.57

7,726 Supporters

$15 min for supporter badge & GeekGold bonus
48.7% of Goal | left

Support:

Ryan Opp
United States
Greeley
Colorado
flag msg tools
BGG's Geek Rating, which determines the rank of a game on this site, is based on mathematics called Bayesisan Averages. The thing about Bayesian formulas is that every one is a little different and you can throw in a large amount of fudge factor, or base your formula on many many factors such as the time elapsed since the vote was cast, etc.

I am a curious, mathematics-minded kind of guy, and wanted to figure out what this secret formula is that BGG uses, because there seemed to be some "unfairness" in the rankings of certain games. On the FAQ, BGG says that it really only bases it's rankings on average user ranking and number of votes. The number of votes determines a "weight" of how much the average user ranking is diluted with 5.5 votes. This seems true in most cases, and I can find no evidence that they are using any other factors in their formula such as age of vote, etc. If someone knows otherwise, correct me if I'm wrong.

However, when graphing the number of votes versus the apparent weight, we can see that the result, though showing a distinct trend, is very sloppy, and is also peppered with outright anomalies. If these weights were to be brought into line--a concrete mathematical trend--then some of the rankings could change significantly.

Two days ago (August 1, 2016, so it's already out of date), I took the data of a union between the top 1000 board games and the 1000 games with the most votes (1209 entries), and analyzed it thoroughly, and applied a new formula to it, which is as close to what I believe BGG originally intended as I can get. The new graph is a lot nicer. The new rankings are not too dramatic for the upper echelon, but get a little crazier later on down the list. On average, each entry changed by 18 ranks!

Here is my data. The graphs I was talking about are at the bottom.
http://www.rytracer.com/DummyVotes.html

Edit: Okay everybody, I have decided to publish a complete list of this system weekly. Please see this thread: https://boardgamegeek.com/thread/1618548/gerbil-rating-syste...
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bryan Thunkd
United States
Florence
MA
flag msg tools
mbmbmbmbmb
I already knew this becuase every BGG rating that differs from mine is off.
28 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
that Matt
United States
Ann Arbor
Michigan
flag msg tools
I'm a quitter. I come from a long line of quitters. It's amazing I'm here at all.
badge
I can feel bits of my brain falling away like wet cake.
mbmbmbmbmb
RyTracer wrote:
On the FAQ, BGG says that it really only bases it's rankings on average user ranking and number of votes. The number of votes determines a "weight" of how much the average user ranking is diluted with 5.5 votes. This seems true in most cases, and I can find no evidence that they are using any other factors in their formula such as age of vote, etc. If someone knows otherwise, correct me if I'm wrong.

You mentioned user rating, vote count, and a dummy vote weight. Add to that an anti-shill variable intended to reduce the influence of extreme ratings. Now you have the minimum of what is used to calculate ranking.

As far as I know, no further details have been confirmed. But there are plenty of ways that one could imagine the ranking model being tweaked.

Your guesstimate of the BGG rating-to-ranking system is off, sometimes by a lot.
16 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
mbmbmbmbmb
The assumption that you know the secret ratings algorithm is a little bit off, or a lot.
11 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Caleb
United States
Seminole
Florida
flag msg tools
mbmbmbmbmb
I'd tell you where you went wrong, but then either I'd have to kill you or someone at BGG would have to kill me.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Byron S
United States
Ventura
California
flag msg tools
So many people think that being bigger makes you a better person.
badge
I'm (probably) bigger than you, but that doesn't make me any better. It's what's inside that counts.
mbmbmbmbmb
What is the median change in rank? That would be a lot more telling, given that it's a half-bounded range, and likely very skewed.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
10/₆
United States
Round Rock
Texas
flag msg tools
"Sometimes, the dark side overcomes what Lincoln called the better angels of our nature"
mb
Yeah, you don't know the secret "anti-shilling" sauce BGG uses.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
P.D. Magnus
United States
Albany
New York
flag msg tools
designer
mbmbmbmbmb
I don't think the Geek Rating is updated in real time. The current number of ratings might be out of synch when you took your data, and that would introduce noise.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ryan Opp
United States
Greeley
Colorado
flag msg tools
runtsta wrote:
What is the median change in rank? That would be a lot more telling, given that it's a half-bounded range, and likely very skewed.


Median change in rank is -1, median change in absolute rank is 12.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ryan Opp
United States
Greeley
Colorado
flag msg tools
MWChapel wrote:
Yeah, you don't know the secret "anti-shilling" sauce BGG uses.


I assumed there was one. A couple of arguments here on that:

1. Bayesian Averages are supposed to be their own anti-shilling sauce. Doing it twice is redundant.

2. Shilling and anti-shilling will probably cancel each other out in most cases. I read one of the linked posts here, and asking haters to change their vote while removing good votes is unfair. The anti-shillers still hate the game.

3. Often, the obvious nerfing is unnecessary. Killer Bunnies, original Risk, Monopoly, all have such terrible ratings anyway, and move very slightly when fixed, although their dummy weights are off the charts.

4. Shilling is a gray area. It's not just throwing 10s and 1s. Everybody sees the average rating right there on the same row as their vote when they place it, so they know whether their vote will change the rating up or down. They may throw one or two points up or down without even realizing it. This is the nature of the beast.

5. If someone is throwing all 1s and 10s, or has only voted on a single entry, this is an administrator issue, not to be contained in the math of the formula. Remove the vote entirely if it is suspect. Then it wouldn't have shown up in my data.

6. Any other nerfing or sauce application is arbitrary. This is very evident in the data. If it is based on a previous voting record of not adhering to norms, that is telling people what their taste in games SHOULD be. As a designer and retailer, I find that the BGG ratings are meaningless if they are biased.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ryan Opp
United States
Greeley
Colorado
flag msg tools
pmagnus wrote:
I don't think the Geek Rating is updated in real time. The current number of ratings might be out of synch when you took your data, and that would introduce noise.


I hadn't considered that, but I know they update the ranking at least daily, as I've been watching changes for a while. I feel like even if I caught it at the end of the cycle, the noise is crazy on things that probably didn't even get votes that day.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
that Matt
United States
Ann Arbor
Michigan
flag msg tools
I'm a quitter. I come from a long line of quitters. It's amazing I'm here at all.
badge
I can feel bits of my brain falling away like wet cake.
mbmbmbmbmb
RyTracer wrote:
1. Bayesian Averages are supposed to be their own anti-shilling sauce. Doing it twice is redundant.

Not really, no.

Quote:
2. Shilling and anti-shilling will probably cancel each other out in most cases. I read one of the linked posts here, and asking haters to change their vote while removing good votes is unfair. The anti-shillers still hate the game.

Why would they cancel each other out in every particular game? Do you have evidence that they do?

Quote:
4. Shilling is a gray area. It's not just throwing 10s and 1s. Everybody sees the average rating right there on the same row as their vote when they place it, so they know whether their vote will change the rating up or down. They may throw one or two points up or down without even realizing it. This is the nature of the beast.

Sure, but that has nothing to do with whether or not BGG attempts to adjust for it. They could adjust for it in a very black and white way (removing users or their ratings) or in very gray ways (weighting users, or particular ratings, based on patterns of ratings).

Quote:
5. If someone is throwing all 1s and 10s, or has only voted on a single entry, this is an administrator issue, not to be contained in the math of the formula. Remove the vote entirely if it is suspect. Then it wouldn't have shown up in my data.

There is at least one person who rates all 1s and 10s, but that doesn't mean that their votes should be removed. Why is that an administration issue? Let everyone rate everything however they want, use all the data, and arrive at results. Why should BGG spend a minute of administrative effort at all trying to chase down the bad faith ratings that, as you complained just a single point ago, are a gray area anyway?

Quote:
6. Any other nerfing or sauce application is arbitrary. This is very evident in the data. If it is based on a previous voting record of not adhering to norms, that is telling people what their taste in games SHOULD be. As a designer and retailer, I find that the BGG ratings are meaningless if they are biased.

You don't seem to have looked nearly carefully enough to conclude anything about how arbitrary the adjustments are. You don't even seem to have any idea what they are, so you've decided that they are based on "telling people what their tastes SHOULD be"? Now who's being arbitrary?


You started off this thread saying that you
RyTracer wrote:
wanted to figure out what this secret formula is that BGG uses

...but it sounds more like you have some weird kind of axe to grind.
6 
 Thumb up
0.05
 tip
 Hide
  • [+] Dice rolls
Ryan Opp
United States
Greeley
Colorado
flag msg tools
tumorous wrote:
RyTracer wrote:
1. Bayesian Averages are supposed to be their own anti-shilling sauce. Doing it twice is redundant.

Not really, no.


Yes, really, yes. That's actually exactly what it's for. If you can't trust the actual user average because of some oddball votes, you reduce its weight according to the few votes it has. If it has enough votes to get past your desired threshold, oddball votes don't count as much anyway. You know how an average works, right? The more voters you have, the less any one vote is heard. I only ask condescendingly because you seem to have no math background yourself, admit to having no idea how the BGG rating system works, and are fiercely loyal to it, and obnoxious in a simple discussion. So what is your weird axe to grind? Did one of your favorite games drop a few ranks in my analysis? Do you think all my favorites did better? No, I was not arbitrary, but used mathematics. It's fun, try it.

So, this number one point was really the only point I needed to make. Anti-shilling is not necessary if Bayesian is done correctly. Of course, throwing 30-100 dummy votes in each entry is not doing it correctly, and regardless of what you are told, is not what is happening anyway anymore, maybe it was 10 years ago when the post that was linked talked about it.

Also, the definition of a shill is someone who has an affiliation with the game and gets a take of its profits. Thus, fanatics do not count, and as to the designers' and publishers' family and friends who throw out lots of 10s, who can blame them? Even political candidates can vote for themselves, and I doubt that the few votes that that comes to make that much of a difference. The only problem is if there is some shadow organization of BGG members out there that meet in a dark room where the publisher of game X walks in and throws a bag of money on the table, and they are large enough to sway the rankings a few points. No, the anti-shilling algorithms, whatever they may be (pronounced "magic"), are doing more damage than good.

Quote:

Quote:
2. Shilling and anti-shilling will probably cancel each other out in most cases. I read one of the linked posts here, and asking haters to change their vote while removing good votes is unfair. The anti-shillers still hate the game.

Why would they cancel each other out in every particular game? Do you have evidence that they do?


No, actually the anti-shillers are more powerful for better-than-average games. For every 1 vote of 1 from a hater, and a 10 from a shill, it is like 2 dummy votes of 5.5 going into the pot, and since good games that we care about all fall in the 7ish area, that hurts the game. Somehow fixing the shill problem and asking the haters nicely to fix their end of the problem is irresponsible as a statistician. See the above paragraphs how a few oddball votes don't affect the outcome that much and there has to be a shadow organization to really throw things off if the math is done correctly.

But, you want examples? Everybody hates Monopoly, right? It's the cliché of bad games. It has a rank here of 12467 (which Hasbro probably doesn't care about as they whistle on their way to drop their money in the bank).
Monopoly is also 48th in all-time most votes gathered. It has an abysmal rating of 4.43, which would make it very bad anyway, but for some reason, BGG goes against their FAQ-explained formula, and moves the rating even further from the 5.5 median. Putting a dot on the chart way out away from everything else. It already sucked. Those 17000 votes made it suck. Why make it suck more?

Then there's Catan. Hands down the most voted on game ever, but #76 in the weight of those votes. The math here suggests that BGG doesn't trust 48000 of the 61000 votes as shills. Again, I'm not saying they're screwing things up intentionally, but it isn't mathematically done.

Quote:

Quote:

4. Shilling is a gray area. It's not just throwing 10s and 1s. Everybody sees the average rating right there on the same row as their vote when they place it, so they know whether their vote will change the rating up or down. They may throw one or two points up or down without even realizing it. This is the nature of the beast.

Sure, but that has nothing to do with whether or not BGG attempts to adjust for it.


You, and others, are the ones saying they are adjusting for it. I just said they made math errors, but didn't give a reason.

Quote:

They could adjust for it in a very black and white way (removing users or their ratings) or in very gray ways (weighting users, or particular ratings, based on patterns of ratings).


And either of these solutions would be statistically, and bordering on morally, wrong. 1 man, 1 vote.

Quote:

Quote:

5. If someone is throwing all 1s and 10s, or has only voted on a single entry, this is an administrator issue, not to be contained in the math of the formula. Remove the vote entirely if it is suspect. Then it wouldn't have shown up in my data.

There is at least one person who rates all 1s and 10s, but that doesn't mean that their votes should be removed. Why is that an administration issue?


Admins deal with unruly members, trolls and the like, this should be both in the forums and in the ratings. A person who is only voting 10s and 1s is definitely going against the spirit of the system. They cannot, with sound conscience and logic, declare that everything they rate is either Outstanding, they always want to play it, or Awful, the game is horribly broken and shouldn't be called a game. This person obviously has a better understanding of statistics than you, and is abusing the system. The admins should contact such a person and ask him about his voting and suggest that he change them in the spirit of this community, and boot him if he turns out to be a bot or a troll, votes removed. What you DON'T do, is change the formula drastically and over-correct this behavior.

Quote:
Let everyone rate everything however they want, use all the data, and arrive at results.


Now we're saying the same thing. Did you see the chart and numbers I posted?

Quote:

Quote:

6. Any other nerfing or sauce application is arbitrary. This is very evident in the data. If it is based on a previous voting record of not adhering to norms, that is telling people what their taste in games SHOULD be. As a designer and retailer, I find that the BGG ratings are meaningless if they are biased.

You don't seem to have looked nearly carefully enough to conclude anything about how arbitrary the adjustments are.


I crunched numbers for three days and posted all those numbers for you to look at yourself. I got all the data off the site a couple days ago, and didn't change any of the raw data you can all access. I have backed up everything I have said with numbers.

Quote:
You don't even seem to have any idea what they are, so you've decided that they are based on "telling people what their tastes SHOULD be"? Now who's being arbitrary?


You're right, I don't have any idea what they are, because they are too random. Thus I believe they are arbitrary, or magic, or made up. Most mathematical trends can be deciphered. There is no consistency here. I have not yet given a reason for it, only a fix. You supplied the reasons, and I argued them away.

I have rated 158 games on here. I could do many more when I have time. It turns out, that I prefer to give good scores to my favorites and not necessarily rip down the ones I dislike. Also, I love board games, and by the nature of my immersion in the hobby, find most of the ones I play to be well above average, getting reviews first from forums and word-of-mouth, I am able to avoid even playing most of the terrible ones. Anyway, my average rating is 8.25. I also like some games others do not, and dislike games that others love, and deviate from the mean on many of my choices (there is a tool you can use in your profile that shows where you disagree most with the masses). Anyway, if any of those things make the games I like take a ratings dive, or make my vote less important, then BGG is "telling me what my tastes SHOULD be." That is what I meant by that. You were the one who suggested they might be doing that.

The reason I think the ratings are all screwed up is because everything is bootstrapped from an earlier version. BGG obviously had problems with their ratings early on and have made attempts to fix them with bandaids and patches that maybe didn't quite cover all of the data, and some games are still governed by old systems. I think it's just a mess, not a con. The fact that they don't update the rankings in real time indicates that there is some human factor involved, and not a simple equation that is less than 16 characters long, like my proposed solution, which covers most of the data nicely and fixes the anomalies.

Quote:
You started off this thread saying that you
RyTracer wrote:
wanted to figure out what this secret formula is that BGG uses

...but it sounds more like you have some weird kind of axe to grind.


I think I've covered my motivations in the previous comments in this post. One of the hallmarks of BGG is the ranking of games. I bet that gets the most traffic of any of the things to do on here. It influences people's buying decisions and playing decisions. And awesomely, it is powered by our own ratings of the various games--a players' opinions poll, at least it claims to be. If this glorious piece of the site is broken, it needs to be fixed.
7 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Barry Harvey
United Kingdom
London
flag msg tools
mbmbmbmb
RyTracer wrote:
...If this glorious piece of the site is broken, it needs to be fixed.

Or ignored.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
J J
Australia
flag msg tools
RyTracer wrote:
Admins deal with unruly members, trolls and the like, this should be both in the forums and in the ratings. A person who is only voting 10s and 1s is definitely going against the spirit of the system. They cannot, with sound conscience and logic, declare that everything they rate is either Outstanding, they always want to play it, or Awful, the game is horribly broken and shouldn't be called a game. This person obviously has a better understanding of statistics than you, and is abusing the system. The admins should contact such a person and ask him about his voting and suggest that he change them in the spirit of this community, and boot him if he turns out to be a bot or a troll, votes removed. What you DON'T do, is change the formula drastically and over-correct this behavior.


Here's something you very much need to read. Note that this is from the head administrator, and has the full backing of the site owner.

You should also read this before you continue in your current vein.

Now, can we please stop having someone complain about this each week? Can we at least extend the refractory period to a fortnight (to begin with)?
10 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
mbmbmbmbmb
RyTracer wrote:
A person who is only voting 10s and 1s is definitely going against the spirit of the system. They cannot, with sound conscience and logic, declare that everything they rate is either Outstanding, they always want to play it, or Awful, the game is horribly broken and shouldn't be called a game.

As Jason noted, this is not necessarily true. The textual descriptions of ratings are only suggestions. (And some people have argued - with some validity - that they are paradoxically incoherent and impossible to follow sincerely in many cases, since they conflate personal desire to play a game with assessment of a game's objective quality.)

I've seen many users who are clearly not malicious trolls but rather simply sincerely don't rate things as fine-grained as 10 different values from 1 to 10, just as many people only rate using the 10 integers and don't put fractional decimal digits for even finer-grained ratings; rather, they rate simply in terms of (e.g.) only 3 values (dislike, indifferent, like) or even only 2 values (dislike, like).

Quote:
The fact that they don't update the rankings in real time indicates that there is some human factor involved,

You could suspiciously suppose that, but as far as I understand it based on various posts by Aldie and other admins, it is actually because there are multiple BGG servers doing different things, and certain automated global update processes run only once daily to help reduce server load. This seems far more likely to me than that human admins are manually intervening to selectively secretly manipulate the game ratings.
10 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Pasi Ojala
Finland
Tampere
flag msg tools
Get the Imperial Assault Campaign module for Vassal from http://www.vassalengine.org/wiki/Module:Star_Wars:_Imperial_Assault
badge
The next total solar eclipse holiday in 2017 in the USA.
mbmbmbmbmb
RyTracer wrote:
If this glorious piece of the site is broken, it needs to be fixed.

It isn't. It works exactly as intended.

1. A user can use their ratings in any way and form they wish. (As long as they rate each game only from a single account.)
2. BGG creates rankings in the way they like from the ratings. (Calculated once a day.)
3. There is no 3.
13 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Eddy Sterckx
Belgium
Vilvoorde
flag msg tools
designer
mbmbmbmbmb
russ wrote:

I've seen many users who are clearly not malicious trolls but rather simply sincerely don't rate things as fine-grained as 10 different values from 1 to 10


This is my rating system, it's all I need when I filter/search my collection :

10 : Super game
8 : Very good game
6 : Ok game
4 : Bad game
2 : Very bad game

2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
that Matt
United States
Ann Arbor
Michigan
flag msg tools
I'm a quitter. I come from a long line of quitters. It's amazing I'm here at all.
badge
I can feel bits of my brain falling away like wet cake.
mbmbmbmbmb
a1bert wrote:
2. BGG creates rankings in the way they like from the ratings. (Calculated once a day.)

If I recall correctly from a Pandemic Legacy megathread, there is also some kind of first-of-the-month adjustment.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
O Little Town of
Sweden
flag msg tools
Help, I'm being held prisoner in an overtext typing facility! I don't have much time, they could find out at any m
badge
I'm that weirdo whose number of badges sold prior to yesterday Bail Organa is keeping track of
mbmb
If we're going to be mathematically and statistically minded, I want to rerepeat again the always-overlooked fact that BGG average ratings are calculated from the arbitrary labels of an ordinal data set. That's not to say good points aren't being made here - on the contrary, I think there are, and many others have criticized the practice of flatly dumping 5.5 votes on all entries over the years. And it's not at all impossible ratings could be "improved" by degrees according to some notion of what ratings should ideally do, but they can never be made "correct", since they aren't really mathematically valid to begin with.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
that Matt
United States
Ann Arbor
Michigan
flag msg tools
I'm a quitter. I come from a long line of quitters. It's amazing I'm here at all.
badge
I can feel bits of my brain falling away like wet cake.
mbmbmbmbmb
Kaffedrake wrote:
If we're going to be mathematically and statistically minded, I want to rerepeat again the always-overlooked fact that BGG average ratings are calculated from the arbitrary labels of an ordinal data set. That's not to say good points aren't being made here - on the contrary, I think there are, and many others have criticized the practice of flatly dumping 5.5 votes on all entries over the years. And it's not at all impossible ratings could be "improved" by degrees according to some notion of what ratings should ideally do, but they can never be made "correct", since they aren't really mathematically valid to begin with.

I wouldn't call that "always-overlooked." Criticism of calculating simple averages from ordinal data has been a cornerstone of arguments about BGG ratings for at least eight years now.

Yes, BGG averages are calculated from ordinal data. But there's nothing fundamentally wrong about that. (Mathematics don't care how you add and divide numbers.) I disagree with your characterization of BGG's approach as "not mathematically valid," insofar as it implies that there is some way to change the process and make the models "mathematically valid." There isn't. All models are wrong. This one could probably be made more useful in certain ways.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
O Little Town of
Sweden
flag msg tools
Help, I'm being held prisoner in an overtext typing facility! I don't have much time, they could find out at any m
badge
I'm that weirdo whose number of badges sold prior to yesterday Bail Organa is keeping track of
mbmb
tumorous wrote:
I wouldn't call that "always-overlooked." Criticism of calculating simple averages from ordinal data has been a cornerstone of arguments about BGG ratings for at least eight years now.


I sincerely believe that if this statistical point was widely understood, the arguments and aspirations of these discussions would change noticeably, even if they would obviously not go away or anything.

tumorous wrote:
Yes, BGG averages are calculated from ordinal data. But there's nothing fundamentally wrong about that. (Mathematics don't care how you add and divide numbers.)


But they aren't real numbers, which is the fundamental point. They've just been given labels that look like numbers. That's the nature of ordinal data.

tumorous wrote:
I disagree with your characterization of BGG's approach as "not mathematically valid," insofar as it implies that there is some way to change the process and make the models "mathematically valid." There isn't.


I have certainly not meant to claim there is!
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bryan Thunkd
United States
Florence
MA
flag msg tools
mbmbmbmbmb
Kaffedrake wrote:
tumorous wrote:
Yes, BGG averages are calculated from ordinal data. But there's nothing fundamentally wrong about that. (Mathematics don't care how you add and divide numbers.)


But they aren't real numbers, which is the fundamental point. They've just been given labels that look like numbers. That's the nature of ordinal data.
While it's technically inappropriate to average ordinal utility, doing so works well enough. There's no way to appropriately get an average preference, but the result of averaging geek ratings is close enough to what I'd expect that I don't care if it's technically wrong. Useful beats correct in this case.

But honestly, I don't really put much stock in the average rating anyway. I'd much rather look at geekbuddy analysis than the BGG zeitgeist.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
that Matt
United States
Ann Arbor
Michigan
flag msg tools
I'm a quitter. I come from a long line of quitters. It's amazing I'm here at all.
badge
I can feel bits of my brain falling away like wet cake.
mbmbmbmbmb
Kaffedrake wrote:
I have certainly not meant to claim there is!



Kaffedrake wrote:
But they aren't real numbers, which is the fundamental point. They've just been given labels that look like numbers. That's the nature of ordinal data.

Well, it's worse than that, because you're still assuming that the data are actually ordinal. They are not necessarily ordered in a meaningful way. BGG admins have literally and repeatedly told users that any games can be rated by anyone for any reason. I could decide that for my ratings, my favorite games will be 5 (my favorite number), 9 is for the worst games (9th circle of hell), 3 is mediocre (3 is a boring number), 7 is for games with a lot of luck... and so on.

At some point, in this model, BGG must leap the gap from "your numbers can be whatever you want" to "your collective numbers are telling us something useful about your opinions of the quality of these games." A linear average with dummy votes and a security-through-obscurity shill-busting algorithm is pretty flimsy, but it's clearly a step in a direction that makes a degree of sense. It's not insane. (It's certainly not, as the OP argues, the Wrong Way vs. his Right Way.) There are definitely good arguments for a different starting point, such as that paired comparison model suggested back in 2007, elaborated in 2008.

(Standard BGG Proposed Changes Note: I haven't seen any inclination from staff to do anything to change the system that's been in place for a decade. Maybe unless we crowdfund them a statistician postdoc.)
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ryan Opp
United States
Greeley
Colorado
flag msg tools
Quote:

...If this glorious piece of the site is broken, it needs to be fixed.
Quote:

Or ignored.

If that's how you feel, you are welcome to it. I notice you don't rate games, and that's fine. I do, and it is something I'm concerned about, so I made this post which is now hotly contested by MOSTLY people who don't use the system at all or have once, so I don't see that their opinions are super relevant.
Quote:
Here's something you very much need to read. Note that this is from the head administrator, and has the full backing of the site owner.

Right, I totally agree with this. People can vote whatever they want. The referenced paragraph was a minutia of my argument which was brought out by tumorous. Statistically, oddball votes will be absorbed. I was mostly referencing multiple accounts, or votes from bots or immediately created accounts that are suspect in such ways.

My ultimate thought is that IF this behavior is to be dealt with, it is done through administration, not tampering with data.
Quote:

You should also read this before you continue in your current vein.

Yes, I apologize for directly calling out someone else's unfriendly behavior. I will try to be more subtle and indirectly condescending so as to remain within the guidelines in the future. I will use the following statement as a model:
Quote:

Now, can we please stop having someone complain about this each week? Can we at least extend the refractory period to a fortnight (to begin with)?

I did as extensive of a search as I could, but people post these topics all over, and it's hard to find them all. Perhaps there needs to be dedicated board for it. Also, my take is very fresh, see original post.

By the way, no one is forcing you to read this thread or comment in it. I'm sorry its presence bothers you.
Quote:

It isn't. It works exactly as intended.

1. A user can use their ratings in any way and form they wish. (As long as they rate each game only from a single account.)

Agreed.
Quote:

2. BGG creates rankings in the way they like from the ratings. (Calculated once a day.)

And so they have. I submit that it is a mix of a proposed mathematical model and arbitrary decisions or errors. If you're all fine with that, so be it. It's been "good enough" for most people's purposes for awhile. Like I've shown, the top ten is virtually unchanged, top forty not bad, etc., but I often look at ratings in the top thousand or more. I have no dog in this fight, but if I was the designer of a game that was ranked 100 places too low, I'd be pissed. The formula is "secret" because they don't want people to learn how to abuse the system. Well, how hard is it to figure out to affect the system? Lots of high votes make the rank go up, lots of low votes down. No, the formula is secret because they don't want commentary on their decisions and/or sloppy math. I love this site, but that's the truth. I'm for mathematical consistency and transparency.
Quote:

While it's technically inappropriate to average ordinal utility, doing so works well enough. There's no way to appropriately get an average preference, but the result of averaging geek ratings is close enough to what I'd expect that I don't care if it's technically wrong. Useful beats correct in this case.

This. Everyone knows they are contributing to a 1-10 scale of average rating when they vote, so there is no slight of hand in using the average, just arguing semantics. If one wants a nihilistic view that the system is meaningless, that person is in the same boat as those at the top of this post that simply don't care, and I ask again, then why are these people so opinionated?
Quote:
At some point, in this model, BGG must leap the gap from "your numbers can be whatever you want" to "your collective numbers are telling us something useful about your opinions of the quality of these games." A linear average with dummy votes and a security-through-obscurity shill-busting algorithm is pretty flimsy, but it's clearly a step in a direction that makes a degree of sense. It's not insane. (It's certainly not, as the OP argues, the Wrong Way vs. his Right Way.)

Yes, I agree that this is on the right track. There is an obvious trend here that works very well. I am just saying that BGG needs to be consistent and stick to that trend. I did not make up the formula that I used to bring the data back into line, I harvested it from the data that was already present and tried to keep the average rating change of the whole sample set at zero. Yes, I found the equation I was looking for, and believe me, it is short and not complicated. It takes two variables, average user rating and number of votes to determine a new geek rating. Simple. Elegant. What was intended. I'm still keeping it "secret" for whatever reasons BGG thinks it can be abused (again, I don't see how). Thus, it's not MY Right Way, it's BGG's Right Way (that is, without the anomalies. Again, have any of the haters even LOOKED at the graphs?).
Quote:

(Standard BGG Proposed Changes Note: I haven't seen any inclination from staff to do anything to change the system that's been in place for a decade. Maybe unless we crowdfund them a statistician postdoc.)

Longevity is not a valid argument, or we'd be having this conversation over snail mail. Also, I submit that it HAS changed, a decade ago is just the last time you got a press release. We also don't need a postdoc, this isn't rocket science. It's pretty simple, actually.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
1 , 2 , 3  Next »   | 
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.