$20.00
Recommend
3 
 Thumb up
 Hide
7 Posts

BoardGameGeek» Forums » Board Game Design » Board Game Design

Subject: h-index as a metric for playtesting rss

Your Tags: Add tags
Popular Tags: [View All]
Jeff Warrender
United States
Averill Park
New York
flag msg tools
designer
publisher
mbmbmbmbmb
In the academic world, the "h-index" is emerging as a popular metric for one's scholarly productivity. Your h-index is the number of papers you've published that have at least h citations. It's a bit more informative than just counting total citations. If you have 1000 citations across all of your papers, but 990 are citations to one paper, and the other ten are from 10 other papers that each received one citation, then your h-index is 1.

I wonder if this wouldn't be helpful as a way of thinking about play testing. It's not unusual to hear publishers say "this game was blind-tested 200 times!" or "30 different groups played this game and loved it!". In play testing terms, the h-index would be, I suppose, the number of groups that played the game at least h times. So if those 200 tests were all a single group, the h index would be 1. If those 30 groups played at most two times per group, the h index would only be 2.

This figure is not informative in the same way as in publishing, I don't think. Having 200 tests or 30 individual groups IS significant. But those numbers alone don't tell the whole story. If you have 30 groups but an h-index of 1 or 2, you have a pretty good handle on what the first-time experience is like, but you may know relatively little about the balance in the game. Or anyway, you don't know any MORE about the balance of the game than the designer with a single group that played 200 times.

There may be a more useful way of expressing this quantitatively. I'm not proposing this number be mandatory for publishers to disclose or that it be added to the BGG database for each game. I'm simply suggesting that as designers, striving to achieve a high "play testing h-index" may be a good idea; and that publishers could be pushed further in response to claims of big playtest counts, to get at the information behind the numbers.
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Eric Brosius
United States
Needham Heights
Massachusetts
flag msg tools
mbmbmbmbmb
It sounds like a good idea to me, Jeff.

This is not quite the same, but some of us have started to use H-index to track the breadth of our game playing:

My H-Index List

My H-Index List
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
George Leach
United Kingdom
Godalming
Surrey
flag msg tools
designer
Perhaps it would be better expressed as h-players played the game h-times. Perhaps then divided by the max player count? Or maybe the publisher's intended "optimal player number".
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
mbmbmbmbmb
Jugular wrote:
Perhaps it would be better expressed as h-players played the game h-times. Perhaps then divided by the max player count? Or maybe the publisher's intended "optimal player number".

Or for each player count, compute the h-player-count at that number of players (e.g. the h-index for 2-player was 17 players played it 2-player at least 17 times, the h-index for 3-player was 15 players played it 3-player at least 15 times, etc), and then the minimum of those would be the overall statistic (e.g. 15) as a concise way of measuring overall playtest coverage?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jeremy Lennert
msg tools
designer
mbmbmbmbmb
jwarrend wrote:
Your h-index is the number of papers you've published that have at least h citations.

This sounds like you have defined a whole class of indexes, where your 1-index is the number of papers with at least 1 citation, and your 2-index is the number of papers with at least 2 citations, and your 3-index is the number of papers with at least 3 citations, etc.

But the way you are using it in the following examples makes me think that you actually mean "your h-index is the largest N for which you've published at least N papers with at least N citations each".
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jeff Warrender
United States
Averill Park
New York
flag msg tools
designer
publisher
mbmbmbmbmb
Antistone wrote:
jwarrend wrote:
Your h-index is the number of papers you've published that have at least h citations.

This sounds like you have defined a whole class of indexes, where your 1-index is the number of papers with at least 1 citation, and your 2-index is the number of papers with at least 2 citations, and your 3-index is the number of papers with at least 3 citations, etc.

But the way you are using it in the following examples makes me think that you actually mean "your h-index is the largest N for which you've published at least N papers with at least N citations each".


That's correct.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Michael Brettell
Australia
South Turramurra
NSW
flag msg tools
designer
mbmbmbmbmb
And applying it to a board game, you're saying its "The board game's h-index is the largest N for which N different groups have played the game N times"?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.