Stephen Contakes
United States
Santa Barbara
California
flag msg tools
mbmbmbmbmb
I recently noticed that the rating for one of my favorite games was dropping by 0.02% per day due to a significant increase in the number of "1" ratings.

Now I don't doubt that a "1" rating my in some cases be appropriate, seeing as how the rating descriptions also focus on the subjective desire of the rater to play the game. However, I would submit that it is not appropriate to go through BGG and simply give "1" ratings to any game that does not have a theme, genre, mechanic, etc... that I personally enjoy or find appealing. If this were the case the ratings would say more about what themes, genres, and mechanics I found appealing than the inherent quality of any of these games and whether they might be worth the time and effort of someone who does enjoy those themes, mechanics, etc...

Moreover, I also worry about those raters give a game a "1' simply to knock it down and so "defend" other games they enjoy more (I remember this happening quite a bit over Peurto Rico, which BTW is a great game). What worries me is that these raters are being "bad neighbors" in skewing game ratings in such a way as to unnecessarily prejudice those of whose who depend on these ratings (among other things) to decide whether to take a chance spending our time, effort, and money on a game. Moreover, they are doing a disservice to the designers and publishers who put effort into the unfairly rated games as well as to the industry that supports their hobby. In short, these raters are hurting the rest of us by skewing numbers that we rely on and effectively maligning the designers and publishers who provide games that at least some of us enjoy.

I case some of you might wonder about whether I am somehow "upset" over a favorite game's ratings falling at around 0.02 per day; I am neither upset nor particularly worried; that game has enough ratings that its metrics are more likely to be affected in the first or second decimal place than in the ones position (although in the case of other games with less ratings the result could be even more serious). Moreover, at the end of the day I will continue to enjoy it.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Dom L
United States
Maine
flag msg tools
badge
READ A BOOK!
Avatar
mbmbmbmbmb
Ratings are useful in aggregate, and when given by people whose tastes you know. As you say, the rating descriptions specifically call out subjective desire to play, so bemoaning that they are not indicative of the "inherent quality" of a game is spitting into the wind. I don't give 1s to games I don't enjoy, but I'll certainly give a 2. Consider these rogue '1' ratings a counter to the many 10s that come from designers' friends, associates, and other people looking to boost them without regard for their "inherent quality." One practice is just as much a disservice to the ratings as the other, and in the long run neither make much of a dent.
18 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Mad Scientist Philip von Doomula
United States
Orono
ME
flag msg tools
"I'm a leaf on the wind. Watch how I soar."
badge
I got in everyone's hostile little face. Yes, these are wooden cubes from boardgaming. Yes, I'm comfortable with that. I am enlightened.
Avatar
mbmbmbmbmb
I'm pretty certain BGG already "filters" out these outliers in its ratings calculations. That still didn't help Space Hulk (third edition). You can only contain so much rage I guess.

Edit: By filter I mean "takes into consideration"
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jeff
United States
Linden
New Jersey
flag msg tools
designer
badge
Social Justice Wargamer
Avatar
mbmbmbmbmb
I rate "ratings wars" a 1.

That said, if you're buying games based on the aggregate "hive-mind" rating, you're doing it wrong, and you're going to end up with a lot of boring games. Much better to read the positive, negative and in-between comments to figure out whether a particular game appeals to your sensibilities.

10 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Paul DeStefano
United States
Long Island
New York
flag msg tools
designer
badge
It's a Zendrum. www.zendrum.com
Avatar
mbmbmbmbmb
The rating system is not swayed by a few rogue 1s.

Once a game has a certain threshold of ratings, the filters and weighting systems kick in.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Rockpaper Scissors
United States
flag msg tools
The only time it's genuinely annoying is when someone says "I'm rating this a 1 to counteract all the fake 10s."

Way to be part of the problem...
13 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
kat costa
United States
Paradise
California
flag msg tools
badge
Yup.
Avatar
mbmbmbmbmb
I don't assign low ratings to stop a game's ascent in the rankings. I rate games "1" or "2" because they're no fun to play. Couldn't care less about how it affects the rankings.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tomello Visello
United States
Reston
Virginia
flag msg tools
mbmbmbmbmb
scontakes wrote:
I recently noticed that the rating for one of my favorite games was dropping by 0.02% per day due to a significant increase in the number of "1" ratings.
I looked your collection for your highest rated games (presumably, therefor a good guess about "one of my favorites"). I looked at the number of ratings for the 4 highest of them (all rated 10) in search of a "significant increase" of 1s simply by checking the current count of 1s.

Middle-Earth Quest
9 out of 2862

The Lord of the Rings: The Card Game
29 out of 9994

Star Realms
10 out of 4737

Star Wars: Imperial Assault
8 out of 1334



Even if I add them all together that's a pretty thin cast of haters. I'm pretty sure that while I sleep tonight the world will still be safe for democracy ... even if some of the participants are inconsiderate.



11 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jason Sadler
United States
Olney
Maryland
flag msg tools
From the Halls of Montezuma...
badge
...to the Shores of Tripoli...
Avatar
mbmbmbmbmb
Any game worth it's salt will have a few 1 ratings. It gives them character and lets you know who the clowns are in this neck of the woods.
6 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
suPUR DUEper
United States
Villa Hills
Kentucky
flag msg tools
Avatar
mbmbmbmbmb
ExcitingJeff wrote:

That said, if you're buying games based on the aggregate "hive-mind" rating, you're doing it wrong, and you're going to end up with a lot of boring games. Much better to read the positive, negative and in-between comments to figure out whether a particular game appeals to your sensibilities.


Why do you rate your games?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
James Ludlow
United States
Saint Louis Park
Minnesota
flag msg tools
badge
Avatar
mbmbmbmbmb
TedW wrote:
ExcitingJeff wrote:

That said, if you're buying games based on the aggregate "hive-mind" rating, you're doing it wrong, and you're going to end up with a lot of boring games. Much better to read the positive, negative and in-between comments to figure out whether a particular game appeals to your sensibilities.


Why do you rate your games?


I rate and comment to hopefully help geekbuddies who chose to follow me either because we have strong agreement or strong disagreement about which games we enjoy.

8 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Stephen Contakes
United States
Santa Barbara
California
flag msg tools
mbmbmbmbmb
I hear you about the negligible effect of the "1" ratings, as well as the "10" ratings that come from the designers friends, family, and associates, which I agree are just as bad. It also bugs me to no end when kickstarter project creators ask you to rate their games in some poll, even though they haven't delivered it yet - as if we are supposed to rate a game just based on the fact that it sounded exciting and we backed it.

I also agree that it is a bad idea to go off ratings alone. That is why I read reviews (both good and bad) and pay attention to reviewers whose ratings tend to match my own feelings about games I have tried. My complaint stems perhaps more from the fact that I have limited time and so must choose which games to consider, explore, read reviews about, etc...and also from the fact that "hater 1 reviews" are to some extend inhospitable to all of the rest of us. It's not that the effects are large or that any one of us is unduly put out in a meaningful way but rather the spirit of the thing.

In fact, I must admit that perhaps it is more that I have a problem with the one-dimensionality of the ratings system itself. I don't mind ratings based on subjective preferences; after all that is probably unavoidable in this sort of thing. However, I don't find it all that helpful when game's overall ratings are based on how much users like a game's genre, mechanic, play time, producer, etc...since this information is available for me to look at on the game page itself. Nor is it helpful to me when a game's overall metric just tells me how much people like it or hate it relative to some other game or how much hey hate its publisher. Such things are rather like the poor amazon review saying that such and such book didn't arrive in time or had handwriting inside - they don't really get at whether one is dealing with a worthwhile item. Now don't get me wrong - it can be helpful to know that some great games are produced by awful publishers who, if rated in their own right would deserve to get truly awful scores. However, since the ratings get used as a sort of shorthand for whether games are great and so might deserve the time and effort I might spend consider buying, learning, and playing them, it might be nice if game ratings to reflected on the game on its own merits. I can always look for helpful reviews to tell me how games compare to other similar games or whether a game company employs a price-gouging or a money-milking expansion scheme in the Alien Frontiers mold. I probably would want to know that so I can decide which game to buy or decide whether a game might be good enough for me to put up with poor publisher support or consumer-unfriendly behavior. However, I would want to know whether a game is good or not in the first place first.

Further, I also don't think it is helpful to rate games based on my preferences and tastes alone (even though the system is explicitly set up that way). I could go through and rate all the 5+ hour wargames and themeless Euros a "1" just because I would never play them but that doesn't tell anyone who has the time and mental energy to spare anything useful about these games, unless perhaps they were somehow interested in studying me particularly. If everyone rated games this way then ratings would be more useful for a sociological analysis of the BGG community than for individual users to decide whether to read about and consider buying individual games - and as far as I can tell BGG's primary purpose is more the latter than to be a sociological data repository. Further, even if issues like play time are as much a showstopper for you as for me, such information is already helpfully displayed on each game's home page.

Note that I would not complain about a game rating system in which one assesses some sort of satisfaction vs. play time and weight since such a thing reflects inherently on the "greatness" of the game itself - i.e. whether it delivers on the high investment it requires. In fact, perhaps what I really want is a multifaceted rating system in which issues like fun factor, components, theme, comparison to similar games, publisher support, value for the price, and other measures are handled separately.

However, beyond this I am still arguing that here on BGG we should to some extent be good neighbors and to ask "haters" and "designer-supporter 10s" to recognize that they are to some extent being inhospitable to the rest of us.

That said, however, I can see now that my concerns about the effect of 1 ratings on game's overall ratings were perhaps less founded than I thought. I simply did not know at the time that BGG used a statistical filtering system that prevents "1s" from inappropriately skewing the overall ratings too much (thanks to all of you who pointed this out).

P.S. I should also clarify that by "significant" I meant a significant jump in the # of 10 ratings relative to what they were before, not a mathematically significant increase.

P.S.S. I actually thought the game in question's average rating was probably too high to begin with (even though my own rating was higher it was sort of a niche game). In fact I am not particularly troubled by the drop - just the manner in which I thought it had occurred.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Maarten D. de Jong
Netherlands
Zaandam
flag msg tools
badge
Avatar
scontakes wrote:
It also bugs me to no end when kickstarter project creators ask you to rate their games in some poll, even though they haven't delivered it yet - as if we are supposed to rate a game just based on the fact that it sounded exciting and we backed it.

Of course. They thrive on visibility to sell the crap they manufacture. Theirs is not a race for quality (ill-defined as that is), but for marketing. You know how I deal with that? I look at the game's family, see 'Kickstarter', and move on. If the game were financed with the author's own money (even if the components are simple), or released as a PnP title, I'd be far more inclined to take a look at it.

Quote:
It's not that the effects are large or that any one of us is unduly put out in a meaningful way but rather the spirit of the thing.

Perhaps, but in any group of people there will be idiots. There isn't much you can do about this.

Quote:
Further, I also don't think it is helpful to rate games based on my preferences and tastes alone (even though the system is explicitly set up that way).

On the contrary, it very much is. For then people can learn about your own personal preferences, and make you their Geekbuddy.

Quote:
I could go through and rate all the 5+ hour wargames and themeless Euros a "1" just because I would never play them but that doesn't tell anyone who has the time and mental energy to spare anything useful about these games, unless perhaps they were somehow interested in studying me particularly.

BGG always expressly allowed people to invent their own meaning for a rate; you don't have to follow the suggested guideline. As long as 10 is in some way 'highest' and 1 'lowest', you're okay. So I am at perfect liberty to restrict my rates to games I've played, and leave the rest alone. This makes the above scenario simply meaningless.

Quote:
Note that I would not complain about a game rating system in which one assesses some sort of satisfaction vs. play time and weight since such a thing reflects inherently on the "greatness" of the game itself - i.e. whether it delivers on the high investment it requires. In fact, perhaps what I really want is a multifaceted rating system in which issues like fun factor, components, theme, comparison to similar games, publisher support, value for the price, and other measures are handled separately.

In practice there would be no difference whatsoever to rating according to these quantities and a rate where this is all lumped together. You are probably not aware of it, but these factors are asking after a subjective personal opinion just as much as the current system.

Quote:
That said, however, I can see now that my concerns about the effect of 1 ratings on game's overall ratings were perhaps less founded than I thought. I simply did not know at the time that BGG used a statistical filtering system that prevents "1s" from inappropriately skewing the overall ratings too much (thanks to all of you who pointed this out).

Well, it's a bit more complicated than that. First you have the raw ratings. You can ask the system to rank games according to this number, and then it's no holds barred: rage rating, shill rating, retaliatory rating, asskissing rating, normal rating, it's all there. If the number of rates is below 30, nothing else is done; there's no ranking calculation. If it is higher than 30, BGG does some extra processing. First a secret algorithm is run on those numbers to filter out 'suspicious' rates. These are probably based on your activity here at BGG, whether or not you have a collection of games, and so forth. This shill filter was designed to remove shill rates (at both ends of the rate spectrum, by the way), and shills follow a slightly different pattern which makes them stand out more. The details are kept secret on purpose to avoid people from gaming that system, and to date noone has bothered to reverse engineer it (although one effort has been made to simply calculate the system's effect). Second, the Bayesian concrete straightjacket is fitted which had the effect of dampening wild rank changes so that a game's move up or down the ranks would be smoother. Unfortunately these days the Bayesian dummy votes are so numerous that in effect they turn the ranks into a popularity vote: only games with thousands of rates will have a shot, ever, to reach the Top-100 or Top-250 levels. (This is why all Geeklists meticulously tracking changes here are a waste of time.)

So the question effectively becomes whether the shill filter these days can handle a legit user rating erratically, and the answer to that is that we simply don't know. Fortunately, if it doesn't, the Bayesian straightjacket weighs down so heavily that the net effect of a screwy rating truly is neglegible. The best defense is simply to behave like a normal user.
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
For those interested in the mysteries of bizarre "1" spite ratings, you might enjoy perusing the "1" comments in this collection.
12 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ron
Austria
Vienna
flag msg tools
“It's all in the mind.” ― George Harrison
badge
Devoted Follower of the Most Holy Church of the Evil Bob. Possessed and down the road to become chaotic, evil & naughty. All hail the Evil Bob and his Stargate.
Avatar
mbmbmbmbmb
russ wrote:
For those interested in the mysteries of bizarre "1" spite ratings, you might enjoy perusing the "1" comments in this collection.

LOL! A funny read indeed, but I guess this user has forgotten to read the BGG rating descriptions.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tomello Visello
United States
Reston
Virginia
flag msg tools
mbmbmbmbmb
scontakes wrote:
P.S. I should also clarify that by "significant" I meant a significant jump in the # of 10 ratings relative to what they were before, not a mathematically significant increase.
If they jumped from a 4 count to an 8 count that might be called a "significant" change in isolation, yet also near meaningless for the overall calculation. (I am interpreting that you meant "1" there instead of "10")

Quote:
That said, however, I can see now that my concerns about the effect of 1 ratings on game's overall ratings were perhaps less founded than I thought.
It's still OK to post a rant. You just have to remember the rest of us like to challenge.


Quote:
However, beyond this I am still arguing that here on BGG we should to some extent be good neighbors and to ask "haters" and "designer-supporter 10s" to recognize that they are to some extent being inhospitable to the rest of us.
They already know. And by their actions show they don't care.


2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tomello Visello
United States
Reston
Virginia
flag msg tools
mbmbmbmbmb
scontakes wrote:
Further, I also don't think it is helpful to rate games based on my preferences and tastes alone (even though the system is explicitly set up that way).
I am way puzzled by that statement taken as standalone. As if we should instead rate them based on the "preferences and tastes" of other people?? Makes no sense to me.

Then I can look at the continuation,
Quote:
I could go through and rate all the 5+ hour wargames and themeless Euros a "1" just because I would never play them but that doesn't tell anyone who has the time and mental energy to spare anything useful about these games, unless perhaps they were somehow interested in studying me particularly.
and maybe that gives me some context about your thoughts that leads me to suggest; of course it is a silly waste of time to go around pretending to rate games with which you have no association. (and hey! that would just put you in league with the "Haters"...)

 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Eric Nolan
Ireland
Dublin
flag msg tools
mb

For good or bad, no matter how correct you are or are not, attempting to get people on the internet to act as you believe they should is this age's version of Kind Canute.

Back off sea, you're doing it wrong
7 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jeff
United States
Linden
New Jersey
flag msg tools
designer
badge
Social Justice Wargamer
Avatar
mbmbmbmbmb
TedW wrote:
ExcitingJeff wrote:

That said, if you're buying games based on the aggregate "hive-mind" rating, you're doing it wrong, and you're going to end up with a lot of boring games. Much better to read the positive, negative and in-between comments to figure out whether a particular game appeals to your sensibilities.


Why do you rate your games?


Uh... to remember how I feel when it comes time to start selling games? To give my friends a sense of what I'm interested in playing? To offer my take on a game in case someone finds it useful? I'm not sure what answer you're looking for here. It's certainly not "to alter the overall ranking by .001% so I can finally put X game where it belongs."
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Reuben Cronje
South Africa
Johannesburg
Gauteng
flag msg tools
Avatar
mbmbmbmbmb
TVis wrote:
scontakes wrote:
Further, I also don't think it is helpful to rate games based on my preferences and tastes alone (even though the system is explicitly set up that way).
I am way puzzled by that statement taken as standalone. As if we should instead rate them based on the "preferences and tastes" of other people?? Makes no sense to me.



I think you're incorrectly placing emphasis on the "my" part of that sentence. You should place the emphasis on "alone" - ie, he is saying that he shouldn't solely rate games on his preferences and tastes without having actually played the game. Seems quite clear when you see that he goes on to talk about simply trawling the database and down-voting Euros and wargames.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Eddie Meister
United States
Peoria
Illinois
flag msg tools
badge
Avatar
mbmbmbmbmb
I personally pay very little to attention to the ratings of a game. Nor does a game's rating have an impact on whether or not I buy it. Just because the game has a low rating and people don't like it doesn't necessarily mean I won't like it either..

I have rated some of my games, but I don't anymore. Everyone has different tastes. I take more stock in reviews, comments and session reports, and if the rules are online, I may peruse them as well. These will give me a much better idea as to whether or not I will like the game.

I love Star Trek, and I love Star Fleet Battles. But I'm sure there are many people who would hate this type of game. It could have hundreds of "10" ratings, but if someone has no interest in that type of game or genre, I doubt it would change their opinion of it. The same is true for me. If a game is rated very high, but it's some silly kids' game, I won't buy it. Just my .02
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
suPUR DUEper
United States
Villa Hills
Kentucky
flag msg tools
Avatar
mbmbmbmbmb
ExcitingJeff wrote:
TedW wrote:
ExcitingJeff wrote:

That said, if you're buying games based on the aggregate "hive-mind" rating, you're doing it wrong, and you're going to end up with a lot of boring games. Much better to read the positive, negative and in-between comments to figure out whether a particular game appeals to your sensibilities.


Why do you rate your games?


Uh... to remember how I feel when it comes time to start selling games? To give my friends a sense of what I'm interested in playing? To offer my take on a game in case someone finds it useful? I'm not sure what answer you're looking for here. It's certainly not "to alter the overall ranking by .001% so I can finally put X game where it belongs."


I was just curious. I do rate my games to make my contribution to the overall ranking (or the "hive mind" in your vernacular). I am pretty good at remembering what games I like and I generally discuss my preferences with my friends. Looks like we rate them for different reasons.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Canada
London
Ontario
flag msg tools
badge
"Federer is Betterer"
Avatar
mbmbmbmbmb
scontakes wrote:
I recently noticed that the rating for one of my favorite games was dropping by 0.02% per day due to a significant increase in the number of "1" ratings.


... Sorry, but if this seriously bothers you enough to post a long message about it ... your life has few worries ... be thankful.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Andy Leber
Canada
Orillia
ON
flag msg tools
Yin
badge
Yang
Avatar
mbmbmbmbmb
TVis wrote:
scontakes wrote:
Further, I also don't think it is helpful to rate games based on my preferences and tastes alone (even though the system is explicitly set up that way).
I am way puzzled by that statement taken as standalone. As if we should instead rate them based on the "preferences and tastes" of other people?? Makes no sense to me.

Then I can look at the continuation,
Quote:
I could go through and rate all the 5+ hour wargames and themeless Euros a "1" just because I would never play them but that doesn't tell anyone who has the time and mental energy to spare anything useful about these games, unless perhaps they were somehow interested in studying me particularly.
and maybe that gives me some context about your thoughts that leads me to suggest; of course it is a silly waste of time to go around pretending to rate games with which you have no association. (and hey! that would just put you in league with the "Haters"...)



You can rate/review something purely subjectively (I like/don't like it), and you can also review something with some level of objectivity. Like reviewing a film's direction, cinematography, etc..

It's often not difficult to objectively review something, and be able to distinguish it's pros, and who it's potential audience could be.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
United States
flag msg tools
I spent 200 GeekGold...
badge
...and all I got was this lousy overtext.
Avatar
mbmbmbmbmb
I stopped paying attention to ratings awhile ago. I mostly rely on suggestions from known friends that share similar tastes as me or, failing that, Rahdo or another video play through to get a feel for games.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.