Recommend
3 
 Thumb up
 Hide
30 Posts
1 , 2  Next »   | 

BoardGameGeek» Forums » Gaming Related » General Gaming

Subject: Rating System: Requesting Feeback rss

Your Tags: Add tags
Popular Tags: [View All]
Jak Varnado
United States
Mississippi
flag msg tools
mbmb
I am writing this post to show off and request feedback on a rating system that I have developed for rating board games. The purpose of the system is to be able to give a good, accurate rating for games in the interest of furthering the BGG community, informing my game group, and just having fun.

The system takes five categories and breaks each down into two or three elements. The elements of each category carry varying weights adding up to 10. The elements are summed up and calculated in the formula given later in this post. Furthermore, each category carries a different weight in the overall formula.

(A)The first category is Overall Fun. This is obviously the most important aspect of a board game and thus carries the most weight (15) in the formula.

1. The first element of this category is My impression. This element ranges from 0-5 in this category. This is essentially how much fun I have when playing the game. This value, as with several others, is of course subjective based on who is rating the game.

2. The second element in this category is Other's impression. The range here is 0-3. This is the overall impression that I got from others while playing that they were having fun. This is just as, if not more, important as my impression, but as I cant look inside their heads, it's more of a guess.

3. The final element of this category is Memorable Experiences. The range here is 0-2. This if for those times that a game leaves you saying "no way" or "I can't believe we pulled that off". Memorable experiences are just that; experiences that are still being talked about many sessions in the future.[/BGCOLOR]

(B)The second category is Theme. Theme is very important to me. Theme (in my opinion) is what can seperate a good game from a great one. For this reason, theme carries the second highest weight (7) in the formula. Again, this category is largely subjective.

1. The first element of the theme category is My impression. Ranges 0-7. As with overall fun, this is simply what I feel about the theme. Afterall, I'm the one rating the game right?

2. The second on last element of theme is Theme : Mechanic Interaction. Ranges 0-3 This is how well the theme works with how the game plays. A good illustration of this would be Agricola. Surely while playing this game you feel like there is a race against time to feed your family and make the most of you plight in the world...just as the theme intended.

(C)The third category is Table-Worthiness. This category carries a weight of (5). Though the weight of this category is not as high as previous ones, it is still an important aspect of the overall rating. Table-Worthiness has a direct impact in several ways on the overall fun of the game.

1. The heaviest weight (0-6) of this element is Likeliness. That is to say, how often will this game hit the table. Do people grumble when you mention it, or is it your go-to game in times of indecisiveness?

2. The final element here is Ergonomics with an intra-category weight of 0-4. Is the game big and clunky? Is the rule book easy to understand? How well do new players pick the game up? Complex games should not be stricken here, but rather games that don't do what they're supposed to do very well. Is the game intuitive?

(D)The Fourth category is Components. Just as the name indicates, this category deals exclusively with the quality of and number of components in a game. The overall weigh here is (4).

1. The first element of components is Component Quality. High score ranges from 0-6 with high scores going to solid components that don't tear or bend easily. They hold up to wear and feel heafty.

2. The second element is Enough components. Perhaps this shouldn't even be an element but it is a pet peave of mine with a score range of 0-2. There is nothing I hate more than having to use pennies to represent armies or whatever. If the game allows it, then there should be pieces.

3. The third and final element of components is Component Diversity. Ranges 0-2. A good example of this is Eldritch Horror. Love the game but am very sick of the same encouters over...and over...and over. Yes I know that there are expansion(s) to help deal with this but they're not being rated; the game is. Most games will score a maximum socre of 2 here, but I felt that it was significant enough to add in for the exceptions.

(E)The Fifth category is Art Design. This category carries a formula weight of (4). I felt that this category was significant enough to add because when looking into, buying, or playing a game, the quality and uniqueness of the art design is what sets games apart and helps to make them memorable.

1. Aesthetics. This is a composite of the quality and uniqueness of the art design. This is the most important element of the category with a range of 0-7.

2. Opinion. Simply put this is how much I like the design. I'm the boss afterall. 0-3.

The FINAL category is...well I dont know what it is. I call it Magic. It's that little element that cannot be explained or quantified. That thing that makes you like one game over another for no apparent reason other than that you just do. This category carries a weight of (3) but, given the totally non-emprical nature of it, it is not divided in the formula. Think of it as a bonus. This category should not have a score value on most games. This is for when "Game A" and "Game B" score similar scores but you don't feel that they should. Again, totally non-empirical but whatevs.

Here is an explanation of the formula.

Overall Fun = A
Theme = B
Table Worthiness = C
Components = D
Art Design = E
Magic = X

A = a1+a2+a3
B = b1+b2
C = c1+c2
D = d1+d2+d3
E = e1+e2
X = x1

Formula ((A*15)+(B*7)+(C*5)+(D*4)+(E*4)+(X*3)) / 35

Note that the total value of all possible scores is 38, but that magic element is not divided by. It's a bonus.

So that's it. Read over it and tell me what you think!
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bryan Thunkd
United States
Florence
MA
flag msg tools
badge
Avatar
mbmbmbmbmb
Jakthulhu wrote:
Read over it and tell me what you think!
I don't believe that your preferences will match up to the way you've laid out your categories. I imagine you'll find that something, like components, doesn't truly measure how you feel about a game very well. So you'll end up in situations where you have games with great components that you don't like and games with bad components that you do like. So you'll kick in a fudge factor to compensate, in your case 'magic', and you'll make the rating come out to what you want it to.

But the more and more that you rate games like this the more you'll find out that your preferences and feelings are much more complex than this simple formula.

I've seen many people try this, and in the end you're just trying to dress up personal preferences in statistics. If anything the numbers distort your true feelings without adding anything useful.


12 
 Thumb up
0.02
 tip
 Hide
  • [+] Dice rolls
Jak Varnado
United States
Mississippi
flag msg tools
mbmb
Thunkd wrote:
Jakthulhu wrote:
Read over it and tell me what you think!
I don't believe that your preferences will match up to the way you've laid out your categories. I imagine you'll find that something, like components, doesn't truly measure how you feel about a game very well. So you'll end up in situations where you have games with great components that you don't like and games with bad components that you do like. So you'll kick in a fudge factor to compensate, in your case 'magic', and you'll make the rating come out to what you want it to.

But the more and more that you rate games like this the more you'll find out that your preferences and feelings are much more complex than this simple formula.

I've seen many people try this, and in the end you're just trying to dress up personal preferences in statistics. If anything the numbers distort your true feelings without adding anything useful.




Thanks for the reply!

I see what you mean and have already experienced this to a degree.
I have tried to re-align weights where needed to get the results near where they would be if I just picked an arbitrary number. Though I guess, this being the case, I could have just picked a number.

What methods do you use when rating a game? Looking for any feedback.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Leonard Moses II
United States
Hixson
Tennessee
flag msg tools
badge
Avatar
mbmbmbmbmb
How about every board game played is just compared to every other one. And the criteria for deciding what goes higher than something else is how much the title paid you for your time.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Michael Carter
United States
Marion
Iowa
flag msg tools
Avatar
mbmbmbmbmb
Jakthulhu wrote:
Thunkd wrote:
Jakthulhu wrote:
Read over it and tell me what you think!
I don't believe that your preferences will match up to the way you've laid out your categories. I imagine you'll find that something, like components, doesn't truly measure how you feel about a game very well. So you'll end up in situations where you have games with great components that you don't like and games with bad components that you do like. So you'll kick in a fudge factor to compensate, in your case 'magic', and you'll make the rating come out to what you want it to.

But the more and more that you rate games like this the more you'll find out that your preferences and feelings are much more complex than this simple formula.

I've seen many people try this, and in the end you're just trying to dress up personal preferences in statistics. If anything the numbers distort your true feelings without adding anything useful.




Thanks for the reply!

I see what you mean and have already experienced this to a degree.
I have tried to re-align weights where needed to get the results near where they would be if I just picked an arbitrary number. Though I guess, this being the case, I could have just picked a number.

What methods do you use when rating a game? Looking for any feedback.


I use my gut feeling and compare the game to other games that I've played.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jak Varnado
United States
Mississippi
flag msg tools
mbmb
mlcarter815 wrote:
Jakthulhu wrote:
Thunkd wrote:
Jakthulhu wrote:
Read over it and tell me what you think!
I don't believe that your preferences will match up to the way you've laid out your categories. I imagine you'll find that something, like components, doesn't truly measure how you feel about a game very well. So you'll end up in situations where you have games with great components that you don't like and games with bad components that you do like. So you'll kick in a fudge factor to compensate, in your case 'magic', and you'll make the rating come out to what you want it to.

But the more and more that you rate games like this the more you'll find out that your preferences and feelings are much more complex than this simple formula.

I've seen many people try this, and in the end you're just trying to dress up personal preferences in statistics. If anything the numbers distort your true feelings without adding anything useful.




Thanks for the reply!

I see what you mean and have already experienced this to a degree.
I have tried to re-align weights where needed to get the results near where they would be if I just picked an arbitrary number. Though I guess, this being the case, I could have just picked a number.

What methods do you use when rating a game? Looking for any feedback.


I use my gut feeling and compare the game to other games that I've played.


Does that not lead to changes in a game's rating over time? Like you give a game a 9 but then play four games that are leaps and bounds better than the original?
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Leonard Moses II
United States
Hixson
Tennessee
flag msg tools
badge
Avatar
mbmbmbmbmb
Isn't anything else mostly lying in the end? You allow for the changes. Because that is more specific and more true.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Michael Carter
United States
Marion
Iowa
flag msg tools
Avatar
mbmbmbmbmb
Jakthulhu wrote:
mlcarter815 wrote:
Jakthulhu wrote:
Thunkd wrote:
Jakthulhu wrote:
Read over it and tell me what you think!
I don't believe that your preferences will match up to the way you've laid out your categories. I imagine you'll find that something, like components, doesn't truly measure how you feel about a game very well. So you'll end up in situations where you have games with great components that you don't like and games with bad components that you do like. So you'll kick in a fudge factor to compensate, in your case 'magic', and you'll make the rating come out to what you want it to.

But the more and more that you rate games like this the more you'll find out that your preferences and feelings are much more complex than this simple formula.

I've seen many people try this, and in the end you're just trying to dress up personal preferences in statistics. If anything the numbers distort your true feelings without adding anything useful.




Thanks for the reply!

I see what you mean and have already experienced this to a degree.
I have tried to re-align weights where needed to get the results near where they would be if I just picked an arbitrary number. Though I guess, this being the case, I could have just picked a number.

What methods do you use when rating a game? Looking for any feedback.


I use my gut feeling and compare the game to other games that I've played.


Does that not lead to changes in a game's rating over time? Like you give a game a 9 but then play four games that are leaps and bounds better than the original?


Sure. My ratings are all based on context. When that context changes, so do my ratings. As I have played more games over the last couple years and have broadened my experience, my tastes have refined and my past ratings have often lowered or increased.

Why is it a bad thing for ratings to change over time?
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Max
United States
Fuquay Varina
North Carolina
flag msg tools
Reign of Fire
badge
Bombs Away! Mission 9.
Avatar
mbmbmbmbmb
Its more of a gut feeling for me too. I've been using this rule set to guide me for awhile.

10: Magnificent game. One of my all-time personal favorites.
9: Excellent game. Always a pleasure to play. I definitely own/want it, and will actively suggest or recommend.
8: Very good game. I likely own/want it, and will actively suggest or recommend.
7: Good game. I might even suggest or recommend it. I may or may not own it.
6: Above average game. Somewhat enjoyable in the right circumstances.
5: Average game. I'm indifferent, but may be willing to play occasionally.
4: Below average game. I'll probably suggest something else, but might be persuaded to play.
3: Poor game. Will actively avoid playing.
2: Very poor game. Please don't ask me to play this.
1: Dreadful, terrible, awful game. The opposite of fun. Conjures feelings of hatred or anger
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jak Varnado
United States
Mississippi
flag msg tools
mbmb
xgamerms999 wrote:
Its more of a gut feeling for me too. I've been using this rule set to guide me for awhile.

10: Magnificent game. One of my all-time personal favorites.
9: Excellent game. Always a pleasure to play. I definitely own/want it, and will actively suggest or recommend.
8: Very good game. I likely own/want it, and will actively suggest or recommend.
7: Good game. I might even suggest or recommend it. I may or may not own it.
6: Above average game. Somewhat enjoyable in the right circumstances.
5: Average game. I'm indifferent, but may be willing to play occasionally.
4: Below average game. I'll probably suggest something else, but might be persuaded to play.
3: Poor game. Will actively avoid playing.
2: Very poor game. Please don't ask me to play this.
1: Dreadful, terrible, awful game. The opposite of fun. Conjures feelings of hatred or anger


Very nice. Thanks for the reply. I was doing something like this before the idea of a formula. Still probably a much better idea in the end...

Also, number 1 made me laugh.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Cornixt
msg tools
badge
Cheap as chips
Avatar
mbmbmbmbmb
I would have put Fun and Table-worthiness into the same category, same as components and art. They are so inter-related. This simplifies it all down to
1. Is it fun enough to play a lot?
2. Is it well-made?
3. Fudge-factor for liking/hating the game despite giving it meticulously accurate scores in the above categories.

You like what you like - no matter how you break it down to justify your feelings, they just are.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
S J
United States
Bloomington
Illinois
flag msg tools
Avatar
mbmbmbmbmb
As Bryan already said, the problem with these kinds of rating systems is that you are trying to quantify your preferences, when they are just that... preferences.

No matter how you "weigh" a game, what you'll eventually find is there will be some games that equal more than the sum of their parts, and some games that are worth less than the sum of their parts. This is that "magic" category you talk about. I've also heard it described as the "X Factor," and "Tilt."

Trouble is, in my experience, this is the only factor that matters. At the end of the day, I don't care that the theme of puerto rico is pasted on, or that the components are geometric shapes, or the it's long... I still really really like the game. Those other factors are irrelevant to me. I prefer Puerto Rico over lots of other games with better themes, components, and which are shorter.

I hate to poo-poo on your post... it's obvious you put a lot of work into this. And you know, if this system is helpful to you, then use it! Rate games using this rubric, and maybe you'll find some fulfillment out of it. But I don't see much hope of a universal rating system using these parameters. There's no accounting for taste, and that's really what games are about.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tomello Visello
United States
Reston
Virginia
flag msg tools
mbmbmbmbmb
xgamerms999 wrote:
Its more of a gut feeling for me too. I've been using this rule set to guide me for awhile.

10: Magnificent game. One of my all-time personal favorites.
9: Excellent game. Always a pleasure to play. I definitely own/want it, and will actively suggest or recommend.
8: Very good game. I likely own/want it, and will actively suggest or recommend.
7: Good game. I might even suggest or recommend it. I may or may not own it.
6: Above average game. Somewhat enjoyable in the right circumstances.
5: Average game. I'm indifferent, but may be willing to play occasionally.
4: Below average game. I'll probably suggest something else, but might be persuaded to play.
3: Poor game. Will actively avoid playing.
2: Very poor game. Please don't ask me to play this.
1: Dreadful, terrible, awful game. The opposite of fun. Conjures feelings of hatred or anger
See also this link from 2006 for something similar and which had been in consideration as replacement for the current BGG guideline.

Alternate rating guidelines

dakarp wrote:
Here is the proposed new scale (or at least one version of it--it went through a lot of drafts--apologies to all involved if I missed something that should have been included):
10: Outstanding game. One of my all-time personal favorites.
9: Excellent game. Always a pleasure to play. Shines under most circumstances.
8: Very good game. Rarely disappoints me. High on my request/recommend list.
7: Good game. Usually willing to play. I might even request or recommend it.
6: OK game. Some fun or challenge at least. Enjoyable in the right circumstances.
5: Average game. I'm indifferent, but may be willing to play.
4: Below average game. I avoid playing and would need to be persuaded.
3: Poor game. Will strongly resist playing.
2: Very poor game. I refuse to play this.
1: Dead game. Seriously negative entertainment value. Black Hole of Fun.



1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ian Taylor
United Kingdom
Epping
Essex
flag msg tools
badge
Avatar
mbmbmbmbmb
The Theme category seems a bit out of place to me. Does that mean that abstracts and other unthematic games are incapable of scoring a really good mark no matter how much they excel in every other area?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Curt Carpenter
United States
Kirkland
Washington
flag msg tools
badge
Avatar
mbmbmbmbmb
The only system I've found useful to is try to shoot for a bell curve centered at 5.5. This is useful when comparing games that typically fall within my radar of games I (or others who care about the same games I do) might play. It does generally mean shifting ratings down over time. As both games generally get better, and I also tire of many games. It would be interesting to be able to preserve a rating for my initial impression, and current rating. If aggregated, that would be really interesting to separate the games that get worse with more plays from the games that get better with more plays. The latter being games that tend to become all-time favorites.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ian Taylor
United Kingdom
Epping
Essex
flag msg tools
badge
Avatar
mbmbmbmbmb
I've noticed that a massive amount of games start life as a 7 for me. That seems to be my sort of 'holding score' for "I like this game, but I'm not sure how much yet".
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mb
I don't see many of the factors I'd consider terribly important:

Veracity to subject matter - if it ain't an abstract, it should represent
the topic well. No, 'theme' doesn't cut it - that's a nebulous term (at
best - usually just wrongly used here) more about flavor than anything: a
game can be strong on theme due to artwork or flavor text.

Balance - within the above, with asymmetrical situations, it's often
tough as hell to get the balance right. Notably, if the victory conditions
reflect an inherent unbalance, they may lead players to act in manners
which are not conducive to making decisions which were historically valid;
the thinking is not the same as faced by the real actors.

Tableworthiness may include what I'd call 'ease of play', but some
very long and complex games succeed well at reducing unnecessary
complexity without losing any of the detail they are attempting to
capture. It may be that one can seldom get a copy of DAK
on the table (due to size and time constraints) but within that confine,
it would seem to place much less burden on the players than equally
large games. It's this that I would worry about, rather than how often
you can actually set aside time.

Notably, 'fun' is lacking from my criteria - terrible games can often
spark a great deal of fun. Sometimes inconsistent rules are the source
of great mirth. It's such a subjective factor that, if I was trying
to deconstruct ratings, it just wouldn't have a place.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jak Varnado
United States
Mississippi
flag msg tools
mbmb
Rather than quote everyone and reply individually (I didn't expect to get so much feedback) I'm just going to do a general reply.

Thanks for the replies and advice. Most of it really makes a lot of sense. My main goal with the project was to TRY to quantify elements of a rating system. I am well aware that such a system will not, and could never accurately depict how someone feels about a particular game

That being said, I would like if a few people punched in some numbers of games that hey have rated and see how close to their arbitrary rating the formula gets them.

Thanks to all once again.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jak Varnado
United States
Mississippi
flag msg tools
mbmb
curtc wrote:
The only system I've found useful to is try to shoot for a bell curve centered at 5.5.


This is very interesting. I have work to do...
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mb
Jakthulhu wrote:
curtc wrote:
The only system I've found useful to is try to shoot for a bell curve centered at 5.5.


This is very interesting. I have work to do...



I do this too, but can see no terribly good reason for it (as opposed
to trying to generate linear results). In the aggregate, if people aim
for a bell curve on their own, they're going to amplify that effect on
the overall rating system. Still, I feel that mid-ranged games are
more common than those which really impress one way or another.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Andi Hub
Germany
Frankfurt
flag msg tools
Avatar
mbmbmbmbmb
xgamerms999 wrote:
...
6: Above average game. Somewhat enjoyable in the right circumstances.
5: Average game. I'm indifferent, but may be willing to play occasionally.
...

I am always amused when people use the term average and actually mean mediocre or even bad. In your case it is really striking, since you rate almost 94% of the games you played as "above average" or better. I rate average games somewhere between 6 and 7 (including the borders).

@topic: I am not a big fan to construct an overall rating based on sub categories. This may be valid for testing technical products, but it is not usefull for measuring something like fun or general appeal. I think the approach of board game geek to assess your own desire to play the game is quite clever. However, I also somehow deviate from this and factor in some nostalgia bonus, if I have played the game a lot in the past and remember the enjoyment I got out of it (e.g. Catan: Cities & Knights).

2 
 Thumb up
0.02
 tip
 Hide
  • [+] Dice rolls
Ian Taylor
United Kingdom
Epping
Essex
flag msg tools
badge
Avatar
mbmbmbmbmb
ringo84 wrote:

I am always amused when people use the term average and actually mean mediocre or even bad. In your case it is really striking, since you rate almost 94% of the games you played as "above average" or better. I rate average games somewhere between 6 and 7 (including the borders).

I don't find that to be particularly strange. Bear in mind that we screen out most of the games we won't like without ever playing them, either because we've heard they're bad (or not heard they're good) or because we know they contain elements that we won't enjoy. If you were given immortality and forced to play every game in existence, you would probably radically redefine your definition of 'average'.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Cornixt
msg tools
badge
Cheap as chips
Avatar
mbmbmbmbmb
Quote:
I am always amused when people use the term average and actually mean mediocre or even bad. In your case it is really striking, since you rate almost 94% of the games you played as "above average" or better.


Reminds me of video game ratings in the 90s - where anything with less than 80% was bad, and less than 60% meant it was unplayable.

The thing is, making 5/10 anything less than average gives you less room to discriminate amongst the top games, if that is any value to you. Having 50 games rated as 8/10 makes it a bit tougher to pick out the better ones.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Michael Carter
United States
Marion
Iowa
flag msg tools
Avatar
mbmbmbmbmb
cornixt wrote:
Quote:
I am always amused when people use the term average and actually mean mediocre or even bad. In your case it is really striking, since you rate almost 94% of the games you played as "above average" or better.


Reminds me of video game ratings in the 90s - where anything with less than 80% was bad, and less than 60% meant it was unplayable.

The thing is, making 5/10 anything less than average gives you less room to discriminate amongst the top games, if that is any value to you. Having 50 games rated as 8/10 makes it a bit tougher to pick out the better ones.


Ratings are still that way today. That method using letter grading to handle review ratings.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Cornixt
msg tools
badge
Cheap as chips
Avatar
mbmbmbmbmb
How much better is an 8 game versus a 7? 8.2 versus 8.1? You start splitting hairs at some point so it becomes meaningless, and that point is different for everyone.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
1 , 2  Next »   | 
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.