Recommend
9 
 Thumb up
 Hide
15 Posts

BoardGameGeek» Forums » Board Game Design » Board Game Design

Subject: 10 Insightful Playtest Questions rss

Your Tags: Add tags
Popular Tags: [View All]
Sanhueza at GAME-O-GAMI
United States
flag msg tools
designer
publisher
Avatar
mbmbmbmbmb
Here is an article posted on Gamasutra, by Wesley Rockholz. In it, he lists 10 questions he uses when playtesting in his game design and development class, and the reasoning behind why these questions are more useful than some other questions typically asked during playtests. A great read!

10 Insightful Playtest Questions:
http://www.gamasutra.com/blogs/WesleyRockholz/20140418/21581...


13 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Murray
Canada
flag msg tools
I like the first question especially: "How much time did you feel like you were playing for?"
It really tries to dig into how players perceived the game, rather than the mechanical aspects of it.

Many of the other questions, I felt, were directed for heavier games, with more focus on strategy and less focus on tactics/chance. This is not necessarily appropriate for a wide variety of games, but still interesting food for thought!

I also like this question:Can you explain why the victorious player won?
I like it for two reasons:
1) It looks at how much understanding the players have of the game's mechanics/interactions.

but more interestingly to me,
2) It looks at *how much attention* players were paying to the game experience as a whole. The variety in the answers you get from your players on this question can help tell you how seriously the player took the playtest and therefore how valid their other responses are. That is true of both aspects of this question, actually. I really like questions like this, to help me determine how much interest the player had in the game and why.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Drew NA
msg tools
Some of the questions I liked, some of the questions I did not. For instance, in question one, I think many people are going to have a tough time answering that. Also, some people are going to be looking at clocks and other people are not going to be. I don't think asking question 1 in that manner necessarily leads to someone having fun. A lot of it has to do with how many responsibilities a player has, too.

There are a few good questions here IMO, though. Question #7 is great. To me that shows whether a player can improve or not. If they can't improve they will likely keep losing, and if they keep losing they likely won't want to play. This can lead to a whole group of people not playing the game, even if others like it. It shows comprehension and critical thinking.

There is a question on whether these should be open-ended or rated to be asked. I think these types of questions are better as open-ended questions. That can lead to being harder to compare with other players, but also leave more opportunity for players to say how they feel. I like the more close-ended questions with the opportunity for notes, but I can see why people would like this route.
-Drew
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Brian Fong
United States
Los Angeles
California
flag msg tools
Avatar
mbmbmbmbmb
Quote:

1. How much time did you feel like you were playing for?

2. Did you feel like you were making friends or enemies with the other players?

3. Could you play the game again without looking at the rules?

4. What was your strategy?

5. How far in advance could you predict your opponents' moves?

6. To what extent did you react to your opponents' moves?

7. Can you explain why the victorious player won?

8. To what extent did you feel like you were in control of the outcome of the game?

9. Did anything hold you back from seeing your strategy or plans through?

10. Name the game you have played that is most similar to the game you playtested.


1. I think the question he should have been going for was "what did you enjoy/dislike about the game? Why?"

2. You shouldn't need to ask whether you made friends or enemies, but rather did the interaction with other players enhance or detract from the game play. In multiplayer games, you need to know if you are battling for a single spot and/or if you are trying to complete a task cooperatively. Player interaction should feel thematically natural. A better question would have been "What would you do to improve player interaction for this game? Why?"

3. Could you? Yes. Should you? Probably not. A better question would be "Did you understand the rules? If so, what were the rules that you enjoyed. If not, what did you not understand and how do you think it may be worded better? Or "Summarize the rules of the game that you just played."

4. An excellent question. A secondary question would be "what made it work or why do you think it did not work?" Otherwise a great question. It could be a short answer prompted as "Explain your strategy in detail and why it did or did not work."

5. "What strategy did your opponent use and was it effective against your own?" Predicting your opponents move is irrelevant.

6. "How do you feel that your opponents strategy effected your own? Was there a place where your strategy evolved based on theirs?" Compiling a player's response with the responses of the other people in the game will give you a better picture than reaction to opponents moves.

7. I can explain why player a won: they fulfilled all of tasks first or compiled the most VP. A better question would be: "What aspect of the game do you feel favors one player over another?"

8. A better question: "Do you feel that all sides had an equal opportunity to win? Why or why not?"

9. "What were the advantages and drawbacks to your strategy?"

10. Another question that I think is spot on.

- The question missing from all of these is why. Giving an answer and explaining why you got that answer are two different things.

- Never give a chance for somebody to answer yes or no to a question unless you also ask why or why not? People will usually give you the minimum of what you ask.

Just my $0.02.
6 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Drew NA
msg tools
Brian's response is right on and much better than mine. Good job Brian.
-Drew
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Steven Tu
South Africa
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
EBWonder wrote:
Quote:

1. How much time did you feel like you were playing for?

2. Did you feel like you were making friends or enemies with the other players?

3. Could you play the game again without looking at the rules?

4. What was your strategy?

5. How far in advance could you predict your opponents' moves?

6. To what extent did you react to your opponents' moves?

7. Can you explain why the victorious player won?

8. To what extent did you feel like you were in control of the outcome of the game?

9. Did anything hold you back from seeing your strategy or plans through?

10. Name the game you have played that is most similar to the game you playtested.


1. I think the question he should have been going for was "what did you enjoy/dislike about the game? Why?"

2. You shouldn't need to ask whether you made friends or enemies, but rather did the interaction with other players enhance or detract from the game play. In multiplayer games, you need to know if you are battling for a single spot and/or if you are trying to complete a task cooperatively. Player interaction should feel thematically natural. A better question would have been "What would you do to improve player interaction for this game? Why?"

3. Could you? Yes. Should you? Probably not. A better question would be "Did you understand the rules? If so, what were the rules that you enjoyed. If not, what did you not understand and how do you think it may be worded better? Or "Summarize the rules of the game that you just played."

4. An excellent question. A secondary question would be "what made it work or why do you think it did not work?" Otherwise a great question. It could be a short answer prompted as "Explain your strategy in detail and why it did or did not work."

5. "What strategy did your opponent use and was it effective against your own?" Predicting your opponents move is irrelevant.

6. "How do you feel that your opponents strategy effected your own? Was there a place where your strategy evolved based on theirs?" Compiling a player's response with the responses of the other people in the game will give you a better picture than reaction to opponents moves.

7. I can explain why player a won: they fulfilled all of tasks first or compiled the most VP. A better question would be: "What aspect of the game do you feel favors one player over another?"

8. A better question: "Do you feel that all sides had an equal opportunity to win? Why or why not?"

9. "What were the advantages and drawbacks to your strategy?"

10. Another question that I think is spot on.

- The question missing from all of these is why. Giving an answer and explaining why you got that answer are two different things.

- Never give a chance for somebody to answer yes or no to a question unless you also ask why or why not? People will usually give you the minimum of what you ask.

Just my $0.02.


The questions in the original article was designed to get to the truth without asking about the truth. If you ask anyone here "would you take a suitcase filled with cash if it just landed on your doorstep", people wouldn't say so. The real test is to actually put it there and see what happens.

Half of those questions needed you to observe the person too, so it's not just "ask and forget". It's observe, ask, assimilate.

Your versions are the sort of very blunt questions that appear in every survey that get suboptimal results.
7 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Drew NA
msg tools
I understand that Tuism but I think it's really getting too cute, and if you ask one question to get an answer on something you didn't specifically ask then there is a good chance the players will not answer the question you really want answered. Sure, with moral and ethical questions it makes sense to go more round about because people's conscience gets in the way. Conscience isn't a huge problem with these kinds of questions.
-Drew
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Brian Fong
United States
Los Angeles
California
flag msg tools
Avatar
mbmbmbmbmb
Tuism wrote:


The questions in the original article was designed to get to the truth without asking about the truth. If you ask anyone here "would you take a suitcase filled with cash if it just landed on your doorstep", people wouldn't say so. The real test is to actually put it there and see what happens.

Half of those questions needed you to observe the person too, so it's not just "ask and forget". It's observe, ask, assimilate.

Your versions are the sort of very blunt questions that appear in every survey that get suboptimal results.


A few points:
1) I just analyzed his article. A play test is not designed to discover personal strategies, but to determine which mechanisms of a game you like and dislike.

2) How do you know the suitcase is filled with cash unless you take in inside and open it? it's not like people advertise what's in the bag. It's also a Schrodinger's cat issue.

3) In gaining insight on a play test, you want blunt answers. I don't have time to try to figure out what you mean by an answer. I need you to tell me what i need to know. This is not make the person feel good, but i need answers; please tell me.

I know he's the TA in a game design class, but the games are there to help teach and clarify mechanics, not to be play tested. As a Paraprofessional educator, I can tell you that it's the wrong approach to assume that you want "to get to the truth without asking about the truth." No teacher has time for that (on top of grading, lesson planning, meetings, family, etc.).

Most people also don't learn from observing. if it were so we wouldn't have so many people doing the same damn thing that doesn't work. It should be more of a Observe, Plan, Test, React, Restest, Explain, Share. The scientific method is taught because it works.

To quote Zee Garcia, "You want people who are brutally honest."`
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jeremy Lennert
United States
California
flag msg tools
designer
Avatar
mbmbmbmbmb
EBWonder wrote:
3) In gaining insight on a play test, you want blunt answers. I don't have time to try to figure out what you mean by an answer. I need you to tell me what i need to know. This is not make the person feel good, but i need answers; please tell me.

Of course you want blunt answers. But most playtesters are not professionals and will not give them to you (even if you specifically ask for them); politeness is too deeply ingrained.

Your proposed questions are aiming to express what you want from your playtesters. The questions in the OP link are aiming to get what you want, taking into account the fact that playtesters are human beings (and generally amateurs) and therefore often don't know how to give you good information, even if you ask for it.

Otherwise this would be easy!

I could certainly quibble with some of the original questions, but I think most of them are probably better than your alternatives.
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Brian Fong
United States
Los Angeles
California
flag msg tools
Avatar
mbmbmbmbmb
I've never pull punches when I play test, but I guess that's just me.
I've learned along the way that it helps nobody. I am not a professional tester nor a developer. I am not a publisher either. I just like doing what I can to see the best game possible be made. If they want my opinion, I'll give them to it straight. But I guess that may just be me.

I'm also not saying that you have to trust one play tester over another, but how are we going to break the habit if we refuse to try to change it?

"Anything worth doing, is worth doing right." - Hunter S. Thompson
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Steven Tu
South Africa
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
Really, asking someone a direct question is much less likely to get you the results needed. It's just human nature, we should try to embrace it and learn from it instead of arguing about it.

A famous example was in a video game development in a huge, expensive team. Playtesters expressed that a weapon was not powerful enough and that they were worried about the balance. So the developers upped the damage and tested again. The testers still felt the weapon was not powerful enough. This went on for a while until the weapon was undoubtedly broken. The developers were puzzled. They began to observe the playtesters, and then started tweaking a bunch of stuff, then observed the playtesters some more.

Eventually they made the screen shake more, and bigger explosion, and made the effect more prominent, but kept the weapon's damage at its original level.

The playtesters said "Yes, now it's way more balanced".


The truth is that people don't know what they want. Another famous example is that if we improved our transport by asking people what they wanted, we'd still be riding faster horses
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Steven Tu
South Africa
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
EBWonder wrote:
I've never pull punches when I play test, but I guess that's just me.
I've learned along the way that it helps nobody. I am not a professional tester nor a developer. I am not a publisher either. I just like doing what I can to see the best game possible be made. If they want my opinion, I'll give them to it straight. But I guess that may just be me.

I'm also not saying that you have to trust one play tester over another, but how are we going to break the habit if we refuse to try to change it?

"Anything worth doing, is worth doing right." - Hunter S. Thompson


It's not about trusting this playtester or that or trusting people will be honest or not. It's that most people simply can't express what they think perfectly. The intuition and the subconscious governs what we find fun and enjoyable. Not many people can actually examine it correctly, let alone express it! It's noone's fault, it's the human condition.

If we could we'd all be happily married forever after and never have to go on second dates
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Brian Fong
United States
Los Angeles
California
flag msg tools
Avatar
mbmbmbmbmb
I agree. There will never be a perfect response nor will you ever find what you want all the time. I just think that you can ask people to be objective to the best of their ability. When I see a question that I can answer yes/no, I tend to cringe. Perhaps it's because I hope for the best but expect the worst.

I try to follow the KISS method. Do I do it every time? I try and I may fail at it sometimes as well.

People are shaped by their environment. People who say that it has no effect usually didn't have to deal with the same issues. I try to effect my environment because I want to change the way that people think. Will I get everybody? No, but it's a lofty goal.

So I can hope to meet the love of my life, but I'm not going to fall apart because I don't or I fail to realize that they are it.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Barry Figgins
United States
Woodland
California
flag msg tools
badge
http://lyrislaser.com
Avatar
mbmbmbmbmb
Many of the questions in the article are designed to get away from unquantifiable observations, and get closer to something quantifiable.

For example, the question of how much time you think has passed. A player will either be correct, too low, or too high. From one person, none of those answers indicates a problem. But if, in your playtest group, 80% of people thought that they played your light party game for an hour and a half, something might be wrong.

Likewise, asking a player to describe their strategy. Strategy may be a vague, subjective thing, but the question is specific - can you describe what you were trying to do? If you've played through a game, and yet can't explain why you took the actions you did, that's a problem, too.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Brian Fong
United States
Los Angeles
California
flag msg tools
Avatar
mbmbmbmbmb
I find that most people can say why they do a particular move, but cannot necessarily see how it effects the game in the long run. I agree that making them, at the end, remember every move and why it was done is moot, but if they take notes (which I always encourage) it makes it easier.

Strategy doesn't need to be complex or focused to be a strategy. Your strategy could be I liked the picture at the time. It doesn't make it any less of a strategy, just not a particularly optimal one (maybe, it may be optimal and you like the picture). There is no wrong way to answer the question, but rather a wrong way to interpret the question.

From a design point of view, I never start to develop a game and say it should take about X minutes to play the game. I find that the estimated time on the box will get people to buy a game, but it doesn't mean you'll actually get it done in that time. I rarely do anyways. If somebody has a warped feel of time, does it mean that they played a different game from another player? While I understand the methodology for his questions, i don't feel that it works well in a play test (for problems) scenario.

And I think that's where a big disconnect in the message occurs. His goal, I think, was go get them to think about the aspects of a game that may or may not work for something you may want to develop. Any developer that says they will never play any other type of game because they want their ideas to be pure has no idea of what mechanisms may work better than what they can think of on their own. To read about or to watch is different than to play with it. Well, depending on they way you learn best, at least.

Barry, I agree that understanding how long your game will take is important. I agree that it is one of many steps that a developer should take. however, games can take up as much time as they need. A 4-player game of Gloom should take about 1 hour (according to the box) but in reality it could go by in 15 minutes or it can go by in about 2 hours. Never assume that a games weight determines how long it will take. Sure, I'll give you that heavier games tend to take longer, but a game of Risk and a game of TI3 can both take 8+ hours to play. I don't think Risk has the same weight as TI3 by a long shot.

That being said...

If you think that your game should take 30 minutes and the play test has an aggregated average play time of 90 minutes, then yes, you have a problem.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.