michael brown
United States
SANDY
Ut
flag msg tools
designer
playtesthub.firebaseapp.com
badge
patreon.com/theTrueMikeBrown
Avatar
mbmbmbmbmb
Not to long ago I made a survey, and through it I decided to make a website to help organize a way for game designers to swap playtests in a fair way.

I have been developing the website, and it is coming along fairly well, however I have been having a hard time designing the feedback screen.

I want the feedback that the playtester provides to be good quality feedback that is actionable, and specific, but I fear that I might be missing the mark. Here are my current ideas for fields (mostly taken from Mike Compton's feedback form):

<edit>I have rewritten this based on feedback</edit>

Quote:
Feelings:
What was the most fun part of the game?
What was the worst part of the game?
Would you want to play again?
If competative: Did the winners like the game more, the same, or less than the losers?
What category would you put this game in?
Are you biased toward or against that category?
List some comparable games you have played.

General:
How many players did you play with?
Did you win?
How much would this game cost in a store?
Would you buy it?
How long did it take you to learn how to play?
How long did setup take?
How long did the game take?
Discuss the length of the game (Was the game the right length?, Was there bad down time?)

Art/Theme:
Did the art feel final?
Did the art match the theme?
Did the mechanics match the theme?
How could the theme be made to fit the game better?
Discuss the art (What worked, what didn't and what improvements would you suggest?)

Rules:
Did the rules make sense? If no, which parts were confusing?
Did you have to reference the rule document during the game? If so, what rules did you reference?
Were there any changes that you would make to the rule document?
How was the text on the components?

Mechanics:
Describe the game's balance.
What was the best mechanic in the game?
What was the worst mechanic in the game?
Do you think the game mechanisms interlocked well?
Do the components help the mechanics?
Is there more than one strategy to win?

Final Thoughts
Overall raiting (1-10), using the BGG descriptors for what the values mean.
What one change would you make to the game?
Any extra feedback that you would like to give?
May the designer contact you regarding this feedback? if so, enter your contact info

Is there anything else that you think that I should probably add to this?

I want to make it as useful as possible.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
James Perry
United States
Spring Hill
Tennessee
flag msg tools
badge
Ex-Term-In-Ate
Avatar
mbmbmbmbmb
I have been working on something similar. A way for me to objectively provide feedback on games I play.

Some questions I ask myself are:

What worked for me?
What didn't work for me?
Where did I find the fun?

Instead of using numbers for the other questions, I put the value into words. Because I had questions like Attention Required and Originality of Mechanics.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ian Ranney
United States
California
flag msg tools
Avatar
I like to know how quick people pick it up. A game shouldn't take 2-3 hours to learn how to play.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ken Shogren
United States
Rochester
Michigan
flag msg tools
badge
Avatar
mbmbmb


Does that count?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ken Shogren
United States
Rochester
Michigan
flag msg tools
badge
Avatar
mbmbmb
theTrueMikeBrown wrote:

Is there anything else that you think that I should probably add to this?


On a more serious note...

- Did you find yourself drawn to a single strategy or did you see multiple ideas you might explore in a replay?

- Did you win? If so, how did the other players feel?

- Did you lose? If so, do you know why the winner won?

- When you finished, which best describes how you felt:
(a) Awesome Sauce! I won. That's enough.
(a) Oh thank you, it's finally drawn its last breath and died of drowning in the vomited misery it created in my life!
(a) Do I look like Spock? ...because I am thinking either it or I am illogical.
(a) Well, that was... interesting.
(a) If it only had miniatures, dice, cards, a board, a scoring track, or basic rules - it would be a winner. But alas, it is only going to end up as the newest Hotness for a week.
(a) I think the final product will be awesome, especially after the theme is updated to Medieval/Renaissance Trading.
(a) So, like... which publisher did you say signed a contract on this?
(a) Wait, what?!?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
michael brown
United States
SANDY
Ut
flag msg tools
designer
playtesthub.firebaseapp.com
badge
patreon.com/theTrueMikeBrown
Avatar
mbmbmbmbmb
Thanks for your replies.

I think that I will add the following questions based on your feedback:
Quote:
What was the most fun part of the game?
How long did it take you to learn how to play?
Does the game seem to have replay value?
How much did the winner like the game?
How much did the losers like the game?

@Ken (and anyone else, really): if you are interested in seeing where this website is going to go, you can sign up to get updates here:
http://eepurl.com/c3x9AP

When it enters betatesting I will notify all people on that list, and invite them to try it out before I release it for real.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
James Williams
Australia
Marrickville (Sydney)
NSW
flag msg tools
badge
Avatar
mbmbmbmbmb
So it's a playtesting credit trading scheme/market... fascinating idea

You would end up being able to determine a dollar value for playtesting time. As a "free-market" setup, it would be subject to the laws of supply and demand (both games to test, and playtester availability). Therefore it may lead to playtesting credit speculation... buying/earning 'em when they're cheap, and selling/spending 'em when they're expensive.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
J C Lawrence
United States
Campbell
California
flag msg tools
designer
badge
Avatar
I look for two things:

1) Has it found the game?

2) Does it actually implement that game or is it a near or wide miss?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Adrian Pillai
France
Paris
flag msg tools
badge
Avatar
mbmbmb
I just wanna suggest that maybe​ put the 'was it fun' question 1st, followed by why? What was most fun part etc. I think if you lead with more feeling based questions first, you could capture a more emotive reaction before it gets tempered by more cerebral questions such as genre and bias.

Also, i agree that assigning ratings numbers is both a little limiting and too subjective to learn from. It's the what testers feel that we want to know. The why can hopefully be gleaned from the testers expanded explanations.

I would ask to slant the questions in a slightly positive tone (e.g. was it fun, what did you enjoy, what needs work etc.) Instead of a balanced​ tone that is fair (e.g. what was good, what was bad etc.)

Even though the questions can be slanted in a positive tone, it doesn't necessarily negate negative feedback, just that a game has to be quite broken for a tester to deliver very negative feedback. And at that point, hopefully even a testers negative feedback can help the designer refocus their game.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Laura Creighton
Sweden
Göteborg
flag msg tools
Avatar
mbmbmbmbmb
I am interested in setup time. How long did it take and did you make any mistakes?

How many players did you play with? Was the game good with this number? If you played with more than one playercount, how did they compare? I really want to hear things like 'Played with 3, 4 and 5 players. Best with 4, good with 5, but with 3 there was significantly less tension in the game, as people were not competing with each other for resources much'. The same game, different testers may also generate feedback like 'Played with 3, 4 and 5 players. Best with 3, good with 4, but with 5 there was too much competition for scarce resources'. Here we have a split between players who want the tension caused by competition for scarce resources in their games vs those who dislike this, so it may be that the game is perfectly positioned where it is.

You also need a section on the art. (What a section for you to leave out! ) What worked, what didn't and what improvements you might suggest. Comments about the size of the components provided can be useful -- some people greatly prefer portability in their games, while others want jumbo components at all times. I realise that lots of people will be using placeholder art -- but these are the people who may most need to hear that the testers really want 'realistic art' vs 'cartoonish'. Assuming that the final version of the game will have larger cards that will be easier to read is an easy mistake for the playtesters to make -- so they might not mention this. If they don't, the designer may never learn that the card size was a problem. If the game is fairly abstract, very detailed artwork may detract from the abstract experience, so just a note of 'don't have too much' may suffice. Somebody may want to criticise icon choices, of course, no matter what you do, but abstract gamers don't want a section about how the art made them feel, because, in general they aren't trying to feel anything.

On the other hand, if you are making a pure thematic adventure, and the art is not-placeholder then the art section needs to be extremely detailed because you need to know how the art is supposed to support the experience and if it is doing its job. These people are supposed to be feeling certain things, and it would be good to find out if the game succeeded in doing this. Here we are interested in immersive depth. Does the game have much immersive, thematic depth, and how was it generated?

I think your question about replayability needs to be expanded. I think, unless this is a light social or party game, that the replayability of a game mostly depends on its depth -- either its thematic depth or its strategic depth which are completely different things. By strategic depth, I generally mean 'the game rewards study'. As a thought experiment, you want to consider what would happen if you played this game every night for 2 hours (or whatever it's playtime is, for longer games) for a week. Or a month.

There are some games which, after the first few plays, you will have maxed out your 'room to improve'. You may become more familiar with the choices available -- and knowing the exact makeup of a deck you need to draw from will definitely give you an advantage -- but you won't be thinking up new strategies. There is no point in studying this any more, because you have already extracted all the strategic thoughts you are ever going to get out of the experience.

On the other extreme are the abstract games where after the first week -- or month -- you have barely scratched the surface of the thoughts you are going to have about the game, and where your best game will only be a challenge to other beginners like you. Despite being a whole lot better than you were after your first game, your 'room for improvement' hasn't diminished in any meaningful sense -- it is all still there, waiting for your study.

The BBG ranking system just uses one word 'depth' and this has turned out to be a problem. Some people who rank games are using depth to mean what I call 'strategic depth' and some are using it to mean 'thematic depth' and still others are using it to mean what I call complexity. All of these elements can add to a games' replayability, but they do so in different ways.

I am not sure what the best way to phrase questions about replayability to make sure you get the information about what it is about the game that makes you want to play it again. And, of course, 'this is a light filler game, you can teach in 2 minutes, and which is good to play for 20 minutes while you are waiting for the other table to finish' is perfectly useful information as well, but here the replayability is precisely because the game has hardly any depth, strategic or thematic.

So maybe you need separate sections -- fill in schedule A if this is a thematically deep game, schedule B if this is strategically deep, and C if this is a filler? I don't think that 'one size fits all' is going to work here.



 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Brendan Riley
United States
Chicago
Illinois
flag msg tools
designer
publisher
Nos operamur, te ludere
badge
"Life is more fun if you play games." - Roald Dahl
Avatar
mbmbmbmbmb
Lots of good notes to think about here. We ask what people think would be a reasonable cost for this game in a store.

If you're asking whether it was fun, you may also want to ask what kinds of games the player likes. If you have a play-tester who usually plays deep Euros testing your light filler, their response will be different than in the opposite situation.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
James Perry
United States
Spring Hill
Tennessee
flag msg tools
badge
Ex-Term-In-Ate
Avatar
mbmbmbmbmb
Please note, that my question was not "is it fun?" Rather "where did I find the fun?"

This is an important distinction.

The first question could be answered with a yes or no, which requires elaboration to be of value. By its nature should be followed up with another question or two.

The second, assumes that games are intended to be fun thus removing a question from the feedback form. So what parts of the game were fun? This helps designers and developers key in on the aspects of their games that the players enjoy. Even a very boring game will have elements of fun (in my experience) and by identifying those elements you also learn what wasn't fun.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Carel Teijgeler
Netherlands
Vlaardingen
flag msg tools
Avatar
mbmbmbmbmb
Some questions to consider:

- what do you think about the downtime?
- do you think the game mechanisms interlocked well?
- what do you think of the use of the cards/dice?
- what do you think of the text on the cards? (effects, actions)
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Carl Nyberg
Canada
flag msg tools
badge
Avatar
mbmbmbmbmb
1. On a scale of 1 to 5, 1 being “very hard to understand” and 5 being “very easy to understand”, how would you rate the rules of this game?

2. Was there any part of the rules or game play that was unclear?

3. What was unclear?

4. On a scale of 1 to 5, 1 being “not fun” and 5 being “very fun”, what would you rate this game?

5. Is there more than one strategy to win?

6. Did you want to play a second time?

7. What did you enjoy about this game?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
michael brown
United States
SANDY
Ut
flag msg tools
designer
playtesthub.firebaseapp.com
badge
patreon.com/theTrueMikeBrown
Avatar
mbmbmbmbmb
Thanks, everyone!

I have combined the responses that I got, and here is the current version of the form:

Quote:
Feelings:
What was the most fun part of the game?
What was the worst part of the game?
Would you want to play again?
If competative: Did the winners like the game more, the same, or less than the losers?
What category would you put this game in?
Are you biased toward or against that category?
List some comparable games you have played.

General:
How many players did you play with?
Did you win?
How much would this game cost in a store?
Would you buy it?
How long did it take you to learn how to play?
How long did setup take?
How long did the game take?
Discuss the length of the game (Was the game the right length?, Was there bad down time?)

Art/Theme:
Did the art feel final?
Did the art match the theme?
Did the mechanics match the theme?
How could the theme be made to fit the game better?
Discuss the art (What worked, what didn't and what improvements would you suggest?)

Rules:
Did the rules make sense? If no, which parts were confusing?
Did you have to reference the rule document during the game? If so, what rules did you reference?
Were there any changes that you would make to the rule document?
How was the text on the components?

Mechanics:
Describe the game's balance.
What was the best mechanic in the game?
What was the worst mechanic in the game?
Do you think the game mechanisms interlocked well?
Do the components help the mechanics?
Is there more than one strategy to win?

Final Thoughts
Overall raiting (1-10), using the BGG descriptors for what the values mean.
What one change would you make to the game?
Any extra feedback that you would like to give?
May the designer contact you regarding this feedback? if so, enter your contact info


I fear it might be too long now. Perhaps the next thing that I do will be to try and pare it back down.

I will update the first post.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Laura Creighton
Sweden
Göteborg
flag msg tools
Avatar
mbmbmbmbmb
I don't think:

'did the mechanics match the theme' works as a question.

If you have made an abstract game then, well, not having much of a theme is a plus. If you want to make the next 7th Continent, this is a bad idea.

If you have tried to make a deeply thematic game, then whether the mechanics support or detract from your theme is a decent question. Otherwise it is so much noise -- the mechanics are supposed to be interesting enough in themselves, and they aren't intended to match anything but the vision of the creator.

I think you need to split the art and the theme into separate bits, for thematic games, of course. If you have a game which is supposed to be thematic, but all the art is placeholder art, then you need to discuss the theme, and if it worked, independently of the art that was used.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls

Lacombe
Louisiana
msg tools
badge
Suddenly a shot rang out! A door slammed. The maid screamed. Suddenly a pirate ship appeared on the horizon! While millions of people were starving, the king lived in luxury. Meanwhile, on a small farm in Kansas, a boy was growing up.
Avatar
mbmbmbmbmb
I would want to know:

- "Describe all major strategies you could see leading to victory in as much detail as possible."

- "Did the players try to do ___*list every conceivable action, tactic, or strategic approach (especially edge cases: starvation strategies etc)*___?

Was _____ successful? / Why didn't you try _____?"


- "Which player position won the game?" (If relevant)

- "What type of strategy/tactics did the winner use?"

- "What could you have done differently to stop the winner's strategy from being as effective as it was?"

- "What other games have you played that offer a similar play experience? What does this game do worse?"

- "How many players could you see playing this game with?"

- "How far into the game did it become clear who would win?"

- "Which questions required rules look ups? Was any question unanswered despite consulting the rules?"
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
patrick mullen
United States
Washington
flag msg tools
mbmbmbmbmb
Survey design is a challenge. You want to lead them a little, in order to force at least a little bit of analysis to get useful feedback, at the same time you don't want to lead too much or overwhelm.

I really like quantitative questions that you can continue to ask new testers as you tweak and measure the results. For instance:

How many turns/how long did you play before you felt like you had an understanding of how the game works and what you were supposed to be doing?

You may also think of more open-ended questions that can prompt responses that you hadn't considered about your game. Such as:
What would you have changed about the game?
What would you not like to see change?

The first of those is sure to solicit some unusable responses, but you might see what people are missing. The second will tell you what's really working already. The more testers agree on that second question the closer I think you are to your goal.

If you can, use a survey tool which allows conditionals. For instance, "Did you win?" would then funnel into questions tailored to the winner. This can help with keeping the survey from being overwhelming. It's also more comfortable for people to answer a few questions on many pages than many questions on a few pages, at least in a digital format.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.