Recommend
9 
 Thumb up
 Hide
45 Posts
1 , 2  Next »   | 

BoardGameGeek» Forums » Board Game Design » Board Game Design

Subject: How's this for a playtester feedback form? rss

Your Tags: Add tags
Popular Tags: [View All]
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
I'm looking to get more structured feedback from blind playtesters. I've put together a single page scoresheet and feedback form.

It's based mostly on forms by Wade McGraith and Mike Compton found here:
https://www.reddit.com/r/boardgames/comments/1ej13y/

(EDIT - New version, scroll down!)


PDF Link

I'll also do a web form version.

What are others doing for feedback forms? Anything I'm missing, or wasting space with?

Thanks for any feedback and this feedback form!
7 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Harv Veerman
Netherlands
Zoetermeer
Zuid-Holland
flag msg tools
Avatar
mbmbmbmbmb
I more or less stumbled in here...

When I see 1 >> 2 >> 3 >> 4 ... ,my math brain reads 1 is far greater than 2 is far greater than 3 is...

After reading the form, I understand you mean exactly the opposite.

Not sure it matters...
7 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
Mad Math wrote:
When I see 1 >> 2 >> 3 >> 4 ... ,my math brain reads 1 is far greater than 2 is far greater than 3 is...
ok, is that what the double sign like that means? I was just looking for a convenient arrow in that font. I'll use a better arrow character. (I studied logic but no math, so arrow means 'if then' conditional to me.)

Thanks
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
DC
United States
Grand Rapids
Michigan
flag msg tools
badge
Avatar
mbmbmb
TimST1E wrote:
Mad Math wrote:
When I see 1 >> 2 >> 3 >> 4 ... ,my math brain reads 1 is far greater than 2 is far greater than 3 is...
ok, is that what the double sign like that means? I was just looking for a convenient arrow in that font. I'll use a better arrow character. (I studied logic but no math, so arrow means 'if then' conditional to me.)

Thanks


I had this same thought. Usually ">>" means "is MUCH bigger than". An arrow needs a stem, like "->".
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
My brother says it means French quotes.

Ok, I'll definitely switch it to a better arrow or simple dot or whatever looks best.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Charles Ward
Japan
Matsumoto
Nagano
flag msg tools
designer
publisher
Avatar
mbmbmbmbmb
Looks very good! I should step up my game
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
L S
Germany
flag msg tools
badge
Avatar
mbmbmbmbmb
Does the enumeration have a purpose? In my opinion, it just irritates the feedback by signaling some sort of basic comparability of answers between different playtesters when there's actually no sound reason to assume that two testers assign the same evaluation and weight to a "6" on any of these scales (nevermind the different value they assign to each of the categories). On top of that, some numbers can even have opposite meaning (e.g. too much and too little interaction), so abstacting the feedback into numbers actually adds ambivalence here.

Why limit the recommendations for game changes to just one option? What if an insightful playtester has multiple suggestions?

I find the wording "balance between players and luck" less than ideal.

Why is clarity of layout, wording, and rules rolled up into one question? What if the layout is clear but the rulebook is a mess?

This is just a gut feeling, so one might want to do a few pilot runs before a decision, but I don't think that asking whether the game was fun or not makes any sense in such a questionnaire. That seems counterintuitive, but let me explain: If I had fun playing the game, I would look for reasons to like it; otherwise, I would look for reasons why it sucks. There is ample opportunity for both in the rest of the questionnaire, so whether I had fun playing the game is redundant and possibly even a confounder for all other answers.
6 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
ex1st wrote:
Looks very good! I should step up my game
Let me know if you want the InDesign files. My form is based on that reddit I linked to.


Thanks L S, some good feedback.

Randombias wrote:
Does the enumeration have a purpose? In my opinion, it just irritates the feedback by signaling some sort of basic comparability of answers between different playtesters when there's actually no sound reason to assume that two testers assign the same evaluation and weight to a "6" on any of these scales (nevermind the different value they assign to each of the categories).
I think the main reasons to use numbers like this are faster and easier for playtesters to just circle something ensuring I have a quantifiable number for every category. Then the data adds up at the analysis stage and I'll have a good idea of general problem areas.

Randombias wrote:
On top of that, some numbers can even have opposite meaning (e.g. too much and too little interaction), so abstacting the feedback into numbers actually adds ambivalence here.
1 is "bad" and 7 is more like "the perfect level for this game." People have already told me they are confused by some of my shortened "examples." I'll try to re-word it, and make it clear that 1 could be too much or too little and 7 is "just right."

Randombias wrote:
Why limit the recommendations for game changes to just one option? What if an insightful playtester has multiple suggestions?
I think that forces them to choose just 1 important change, also by asking for 1 specifically you're more likely to get 1 than 0.

This form is not meant for pro playtesters, more like random people willing to try the game. Most people don't seem to like filling out forms so I want to make it easy as possible.

But I certainly want to hear more from people willing to give more feedback. I'll make sure I have links for "more feedback" for people willing to go further than the one sheet, if only just to my WIP thread.

Randombias wrote:
I find the wording "balance between players and luck" less than ideal.
You're right, I mean "balance between the players, which luck is a main element." Maybe just "BALANCE and luck" as it's obviously between players.

Randombias wrote:
Why is clarity of layout, wording, and rules rolled up into one question? What if the layout is clear but the rulebook is a mess?
Yeah, rules are important enough to have their own category. I'll cut layout and card text, hopefully fit into another category.

Randombias wrote:
This is just a gut feeling, so one might want to do a few pilot runs before a decision, but I don't think that asking whether the game was fun or not makes any sense in such a questionnaire.
I think you're right, it's the wrong word. Maybe the right word is "ENGAGEMENT and decision making."
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
Maybe I should re-word the categories to statements which you "strongly disagree" to "strongly agree." I could remove the examples and fit a few more statements.

Would that be better?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Justin Powell
United States
huntington
West Virginia
flag msg tools
mbmbmbmbmb
Flow: "Too much going on/too many exceptions" to "Right amount of stuff happening" leaves no room for a playtester's opinion that there is not ENOUGH stuff going on.
Balance: "Completely broken/runaway luck" to "balanced and fair/right amount of luck" leaves no room for a playtester's opinion that there is not ENOUGH luck elements.

5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jon Moffat
United States
Virginia
flag msg tools
designer
Avatar
mbmbmbmb
I suggest changing from 1-7 to 1-5 scale. Not that 1-7 is unclear or bad, but people are more used to the 1-5 (or 1-10) convention.

5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bastiaan Reinink
Netherlands
Utrecht
flag msg tools
Avatar
mb
I would reword all of them and focus on exactly 1 question per item:

-Layout
-Clarity of the rules
-Fun to play
-Etc.

Also reword exactly what you want people to tell you. The "I agree" vs "I disagree" suggestion from above is good. My preferred however would be:
1: Strongly dislike
...
7: Strongly like

This would make things like "tension" more clear as well (is tension good or bad?) This way you will get the opinion of the players and not some sort of abstract "good tension".

Finally, I think a form like this would be handy if somehow you were able to get a lot of people to play your game and without being able to have an "interview" with them afterwards. If however you have to do these playtests yourself, go for an interview, as you will be able to dig so much deeper!
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
L S
Germany
flag msg tools
badge
Avatar
mbmbmbmbmb
Quote:
I think the main reasons to use numbers like this are faster and easier for playtesters to just circle something ensuring I have a quantifiable number for every category. Then the data adds up at the analysis stage and I'll have a good idea of general problem areas.

I would recommend to leave the numbers out of the questionnaire then. If they only come into play at the stage of analysis, you can just as easily convert structured feedback into numerical values yourself, so there's no need for a double scaling (worded and enumerated) of the structured feedback in the questionnaire itself. That said, I still doubt that there's actually something to quantify here, or that quantification really helps the process of evaluation.

Quote:
This form is not meant for pro playtesters, more like random people willing to try the game. Most people don't seem to like filling out forms so I want to make it easy as possible.

This is an assumption that I always encounter among my students as well, and I really wonder where it comes from. Have you ever been in a doctor's anteroom? Check the free journals there and look how many of the "what type of lover are you"-tests are filled out. People fill out questionnaires for fun.


To be a bit more constructive: I like the boxes at the bottom of your sheet, and I think that this sufficiently structures the feedback while still being open enough for more meaningful answers. Personally, I would recommend to expand on this part of the questionnaire as much as possible - for instance, a question box like "Did you like the game's duration?" gives the option for the same answers as the scaled question, but it also gives the option to write something like "The overall duration was fine, but there was too much downtime between my turns."
Maybe just try to convert the scaled questions into such boxes and see if you like them; then, if there are topics left that don't translate well into such semi-open questions, leave only those as scaled questions.

Quick remark on the "one change": If you put it as "the most important change" instead, it remains basically the same question, but respondents won't feel as limited because they're still free to say "but that's both important". Even if you were going for exactly one suggestion, this would be the way to ask for it in a questionnaire.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Derek H
South Africa
Pretoria
Gauteng
flag msg tools
badge
Avatar
mbmbmbmbmb
wowbagger44 wrote:
I suggest changing from 1-7 to 1-5 scale. Not that 1-7 is unclear or bad, but people are more used to the 1-5 (or 1-10) convention.

Actually, from the form there are 4 sets of written (non-numeric) descriptions below each scale. Why not just link a scale to those:

1 = Not so good.
2 = OK.
3 = Good.
4 = Great.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
Awesome feedback! I've taken everybody's suggestions and done a new version.

I think the ideal for these types of "Likert Scales" is to have every option a labelled blank circle (like L S suggests). I don't have room to explain each option here, so I went with 1 to 5 and labelled the ends. I did try blank circles but they felt more ambiguous, the numbers at least suggest the degrees of variance.

I've taken your suggestions on allowing too little - perfect - too much on some questions. I split half the questions into a scale where "5 is the perfect amount." I wasn't sure the best way to handle this scale. I felt 5 should always mean BEST or else it could get confusing. I wanted 1 to be "too little", so that kinda left 9 as "too much."

Numbers make sense? Is their a better way to do the scale?



PDF Link
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
You know what would be very useful - some info on the boardgamer's demographic. Then I could analysis the numbers effectively. Something like:

Which best describes you and the games you like to play?
A) Eurogamer
B) War games and strategy
C) Casual and Party
D) ...

Dont know where I'd fit it though.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
L S
Germany
flag msg tools
badge
Avatar
mbmbmbmbmb
In principle, I would advise against using two different scales (one where the middle option is neutral, one where its the optimum) in a single questionnaire because it's a potential source of confusion; that said, since this is a questionnaire for board gamers, it's probably safe to assume that your playtesters will get along with this just fine. Perhaps I would put the second set of answers as 1-3-5-3-1 instead of 1-3-5-7-9 to further indicate that 1 is always the worst option and that "too short" and "too long" are equally bad. But, as I said, that's probably something you shouldn't worry too much about.

As for demographics, the question you put down would be ok but must allow for multiple answers (e.g. I like wargames and Euros). However, these kinds of classifications are very subjective, not everybody may think of the same set of games when they hear the word "Eurogamer", and some people may not even know this classification at all. Maybe a question like "What are your three favorite board games?" would serve the same purpose while avoiding the ambiguity of game categories?

On a side note, I think you've reached a point where you should do some test runs with your questionnaire. See how it gets filled out for some random games and if the provided information would be helpful for you.

For example, my answers for Race for the Galaxy would be:
Rules: 4
Text and symbols: 2
Engaged: 5
Mechanics: 5
Theme: 3
Time: 5
Turns: 5
Luck: 5
Decisions: 5
Interaction: 3

For Munchkin:
Rules: 4
Text and symbols: 5
Engaged: 3
Mechanics: 3
Theme: 5
Time: 9
Turns: 3
Luck: 7
Decisions: 3
Interaction: 5

And for Exploding Kittens:
Rules: 5
Text and symbols: 5
Engaged: 1
Mechanics: 3
Theme: 3
Time: 5
Turns: 5
Luck: 9
Decisions: 1
Interaction: 5

So that's one game which I deem pretty near perfect (RftG's only problem is the steep initial learning curve in regard to the obscure symbols), one game that has a serious flaw (Munchkin is a funny party game that drags on way too long), and one stupid game whose only redeeming feature is that it is over rather quickly (Kittens)... all in my opinion, of course.

In regard to the questionnaire, I had three problems while filling out the answers:
1.) While it's not pure solitaire due to interactive phase selection asf., RftG really isn't all that interactive. However, I don't see this indirect and low degree of interaction as a flaw while the questionnaire seems to suggest that there is a "golden mean" of interaction that each game has to strive for. I had a hard time to decide for an answer in that case.
2.) In regard to the question about "time", I seriously cannot think of a case where I would criticize a game for being too short. If a game is fun but over rather quickly, we usually just play another round (or more). So, short playtime doesn't bother me in the slightest - certainly not in the same way that Munchkin starts to bore me due to its long playtime. Hence, I suspect that the left end of the scale for the time question is inadequate.
3.) I have the same problem with the question about luck. I can't think of a game that I would dismiss because it has not enough luck in it; on the other hand, I hate games that depend too much on luck for my taste. In this case, the question can only be evaluated against the background of my "boardgame demographic" (see above). Since I would list Chess as my favorite game, it's easy to see that I'm not bothered by the absence of lucky elements - but you definitely need at least a little bit of information about my preferences to assess my response to this question meaningfully.

3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
Some good stuff RandomBias...

Randombias wrote:
In principle, I would advise against using two different scales (one where the middle option is neutral, one where its the optimum) in a single questionnaire because it's a potential source of confusion; that said, since this is a questionnaire for board gamers, it's probably safe to assume that your playtesters will get along with this just fine. Perhaps I would put the second set of answers as 1-3-5-3-1 instead of 1-3-5-7-9 to further indicate that 1 is always the worst option and that "too short" and "too long" are equally bad. But, as I said, that's probably something you shouldn't worry too much about.
What about emoticons for the very bad to very good, and a thermometer icon for the scale? I'll try it.

Randombias wrote:
As for demographics, ... Maybe a question like "What are your three favorite board games?" would serve the same purpose while avoiding the ambiguity of game categories?
Might try more like "types of games."

Randombias wrote:
On a side note, I think you've reached a point where you should do some test runs with your questionnaire. See how it gets filled out for some random games and if the provided information would be helpful for you.
Awesome thanks for the help.

Randombias wrote:
1.) While it's not pure solitaire due to interactive phase selection asf., RftG really isn't all that interactive. However, I don't see this indirect and low degree of interaction as a flaw while the questionnaire seems to suggest that there is a "golden mean" of interaction that each game has to strive for. I had a hard time to decide for an answer in that case.
And I'm not sure I gain anything from that question... as a card game it's solidly interaction anyways. Might cut it to make room.

Randombias wrote:
2.) In regard to the question about "time", I seriously cannot think of a case where I would criticize a game for being too short. If a game is fun but over rather quickly, we usually just play another round (or more). So, short playtime doesn't bother me in the slightest - certainly not in the same way that Munchkin starts to bore me due to its long playtime. Hence, I suspect that the left end of the scale for the time question is inadequate.
This game ends at a certain number of points. I now this doesn't always happen at the right time. Perhaps I can re-word the question t be more about victory conditions, satisfying ending.

Randombias wrote:
3.) I have the same problem with the question about luck. I can't think of a game that I would dismiss because it has not enough luck in it; on the other hand, I hate games that depend too much on luck for my taste. In this case, the question can only be evaluated against the background of my "boardgame demographic" (see above). Since I would list Chess as my favorite game, it's easy to see that I'm not bothered by the absence of lucky elements - but you definitely need at least a little bit of information about my preferences to assess my response to this question meaningfully.
In this case, my game is more like Exploding Kittens. It's more of a casual card game. You'd likely find it too luck based, but I do think a lot of my potential players would find chess not luck based enough.

I'm also hoping to get more causal players to be filling out this form.

That's why I think a question about the playtester like that is important.


Some more stuff to think about. Thanks!
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
The Chaz
United States
Arkansas
flag msg tools
designer
Avatar
mbmbmb
An "in focus photo of your filled in form" is significantly worse than, say, using the CamScanner app to create a clean pdf.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bastiaan Reinink
Netherlands
Utrecht
flag msg tools
Avatar
mb
I think there is room for further improvement, but this already looks good!

Keep us posted with further updates?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
The Chaz wrote:
An "in focus photo of your filled in form" is significantly worse than, say, using the CamScanner app to create a clean pdf.
Good idea. Somebody who already knows how to use a scanner app would probably think of it. I'll make a note of it on the site instructions.

This is what I've come up with for icons. Standard happy faces to good to bad, and bears for "just right" scale.



So this is the form now.



PDF Link


I'm not sure about the lost sale question, "Do you make the right amount of decision?" It's actually pretty similar to the previous 2 questions. What's another question that could go there?


When this is done, if anybody wants it I could make a generic version of this form for any game. Would that be useful? Should I include a generic scoresheet?
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Charles Ward
Japan
Matsumoto
Nagano
flag msg tools
designer
publisher
Avatar
mbmbmbmbmb
I prefer the previous NUMBERS over the new ICONS
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Davidson
Canada
flag msg tools
designer
Avatar
mbmbmb
ex1st wrote:
I prefer the previous NUMBERS over the new ICONS
Are the numbers easier to understand?

What about numbers vs the Bears? That's where I thought the numbers got confusing.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Charles Ward
Japan
Matsumoto
Nagano
flag msg tools
designer
publisher
Avatar
mbmbmbmbmb
TimST1E wrote:
ex1st wrote:
I prefer the previous NUMBERS over the new ICONS
Are the numbers easier to understand?

What about numbers vs the Bears? That's where I thought the numbers got confusing.


For me yes. I'm not so awake these days snore

So the bears could be numbers that go up then back down, like: 1 3 5 3 1
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Matt Stokes
United States
North Augusta
South Carolina
flag msg tools
Duck Season!
badge
I also believe that you met the King of the Forest, Mei, and meeting him is a sign of good luck.
Avatar
mbmbmbmbmb
TimST1E wrote:
ex1st wrote:
I prefer the previous NUMBERS over the new ICONS
Are the numbers easier to understand?

What about numbers vs the Bears? That's where I thought the numbers got confusing.


I think Goldilocks should be in the middle of the bears.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
1 , 2  Next »   | 
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.