Recommend
7 
 Thumb up
 Hide
38 Posts
1 , 2  Next »   | 

Wargames» Forums » General

Subject: Blindtesting? rss

Your Tags: Add tags
Popular Tags: [View All]
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mbmbmbmbmb
Is it dead?

I was communicating with a designer recently, and he asked
(after I had commented on his game not being adequately blind-tested)
if I knew anyone who could do this (presumably well).

Are there wargamers out there - willing to cast a significant amount
of effort into learning and providing detailed comments on a game
in exchange for a (early) free copy and a name in the credits? Or
are the only people willing to do this all tied up in a few projects?
7 
 Thumb up
0.05
 tip
 Hide
  • [+] Dice rolls
roger miller
United States
California
flag msg tools
publisher
They are tough to find and many of the best are already busy. When I get a volunteer, no matter how inexperienced, I start working with them and build a good relationship with them. Another way I do testing is trade favors with other designers. So I will be testing Dien Bien Phu for Kim Kanger and at a later date he will return the favor for me with one of my games.
7 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Michael Hovan
United States
Elkhart
Indiana
flag msg tools
Avatar
mbmbmbmbmb

I am very willing to do it and have friends who would do so with me. Getting plugged into the system seems to be the problem.

Imho, it is like software development. Most game designers do not leave enough time for independent testing and are then surprised when bugs appear.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mbmbmbmbmb
rmiller1093 wrote:
They are tough to find and many of the best are already busy. When I get a volunteer, no matter how inexperienced, I start working with them and build a good relationship with them. Another way I do testing is trade favors with other designers. So I will be testing Dien Bien Phu for Kim Kanger and at a later date he will return the favor for me with one of my games.
The problem is, they aren't really blindtesters if they've seen
the system in its last iteration. To really get a full effect,
you have to start with someone who is learning the rules from
scratch. One round isn't enough, if major changes are needed. :/


That's not to say dedicated testers are a bad thing (they're not - they
come with insight into the design that helps), but they can't help try
and find out what doesn't work in terms of the learning process. And this
is why blindtesters are so vital.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jesse Escobedo
United States
Rockville
MD
flag msg tools
badge
Avatar
mbmbmbmbmb
It seems that wargame playtesters are a pretty self-selecting lot. Maybe if you offer the latest Munchkin expansion as incentive you would be better able to idiot-proof the rules and game flow.

Jesse

P.S. I am casting aspersions on Munchkin players in good humor, please take it that way. I would be serious were I talking of Fluxx players...
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Darrell Hanning
United States
Jacksonville
Florida
flag msg tools
badge
We will meet at the Hour of Scampering.
Avatar
mbmbmbmbmb
I did this for Don Greenwood, in the eighties. Doing it right takes a lot of time, and a lot of communication.

Frankly, I just wouldn't have the time, anymore. It would have to be a game that was really special to me, to push so much else off my plate.
7 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mbmbmbmbmb
DarrellKH wrote:
I did this for Don Greenwood, in the eighties. Doing it right takes a lot of time, and a lot of communication.

Frankly, I just wouldn't have the time, anymore. It would have to be a game that was really special to me, to push so much else off my plate.
Yeah. There was a time in my life when I would have.

Essentially, I ended up doing it for La Grande Guerre (though after
publication). I was performing the same sort of role working on
a Mathematics text, and really had my editor's hat on. I had compiled
exhaustive question lists before with EU and EiA as well, but only for personal use - coming
up with every single issue that I could find to ask a question on, with
the intention of resolving them within our play group.

I didn't realize at the time that this was something one could
reach out and ask to do (not that I'd have that gall anyhow); in a sense,
I guess I'm hoping that some who DO have the time see this discussion
and do try and reach out.

Is there something you're interested in/heard about being developed?
If so, and you are available, it might be worth making it as good
a game as possible.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jason Maxwell
United States
Arvada
Colorado
flag msg tools
badge
Avatar
mbmbmbmbmb
As a software tester I've always been interested in playtesting but never knew how it worked. Personally I'd love to be a playtester for some of thee games that claim to play solitaire as it feels like many times the solitaire rules are tacked on without adequate testing.
1 
 Thumb up
0.02
 tip
 Hide
  • [+] Dice rolls
Mike Hoyt

Butte
Montana
msg tools
Avatar
mbmbmbmbmb
I play-tested a new edition of one of my favorite games. I was flattered that the Publisher and Designer asked me and I went all out to do a good job. They were appreciative and my suggestions/corrections made it into the final product so I know they meant it.

It was an incredible amount of work. Reading and re-reading the rulebook. Trying to mis-interpert rules the way a newbie might, or ask reasonable questions that were not covered. Trying to break the system with a perfect strategy or gamey tactic. Trying to be a rules lawyer and dispute everything. And play the game. And take notes on all of the above. And then do it all over, and over. And then write up my findings and suggestions for the Designer, and get the revised rulebook back and start over.

A free game and my name in the credits? Nice, but certainly did not pencil out economically. I gave up a lot of "hobby" time, and "hobby" time has become more precious over the years.

And that was for a game I really liked and wanted to contribute to making better. Blind testing a title I don't have much interest in? Forget about it.

I feel the same way about writing reviews. I wrote a review for Fire & Movement of game I thought I would like, but ended up hating. But I did a good job of the review, which entailed a ton of time (though less than testing). Now I sometimes see calls for reviews, and frankly the prospect of a free copy is not enough to entice me at all. Instead I bang out a review here on BGG when I feel like it, and get paid exactly the same.

People who will do Testing and Reviews for nearly nothing are heros and my hat is off to them. And if I hear about a game that interests me enough I might even rejoin their ranks. But by and large I prefer to spend my "hobby" time elsewhere.

(I should mention that my impresson is that Designers get even less per hour. This whole hobby is largely a labor of love)
7 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Carsten Bohne
Germany
Duisburg
NRW
flag msg tools
badge
Avatar
mbmbmbmbmb
Software tester as well, but I doubt that I could put in the time needed to really provide serious feedback these days. Back in college this would have been great, but now with a job and kids... cry
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Edgar Gallego
Spain
Sarriguren
Navarra
flag msg tools
badge
Avatar
Microbadge: Napoleonic Wars enthusiastMicrobadge: GrognardMicrobadge: Eagles of the Empire fanMicrobadge: WargamerMicrobadge: Enemy Action Ardennes fan
blockhead wrote:

And that was for a game I really liked and wanted to contribute to making better. Blind testing a title I don't have much interest in? Forget about it.
That sums it up
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mbmbmbmbmb
blockhead wrote:
I play-tested a new edition of one of my favorite games. I was flattered that the Publisher and Designer asked me and I went all out to do a good job. They were appreciative and my suggestions/corrections made it into the final product so I know they meant it.
Why/how did they already have your name? Seriously. Because finding people
who are truly new to a system is vital to actual blindtesting.
Most games get playtested. That's not what is at issue.

Quote:
It was an incredible amount of work. Reading and re-reading the rulebook. Trying to mis-interpert rules the way a newbie might, or ask reasonable questions that were not covered.
Ah. Sounds like you were doing normal playtesting. Projecting as a newcomer to
the system. This is the issue. Really getting these games into people
who aren't familiar with a designer's style (or worse the core system)
is what I'm looking to here, not trying to estimate how such a person
would react.

Quote:
Trying to break the system with a perfect strategy or gamey tactic. Trying to be a rules lawyer and dispute everything. And play the game. And take notes on all of the above. And then do it all over, and over. And then write up my findings and suggestions for the Designer, and get the revised rulebook back and start over.
These aren't the core to blindtesting though. Sure, once someone has
done the blindtesting, they might (if useful enough) be moved into
the regular rigors of playtesting - but the big bonus from blindtesting
is that the people doing it are just trying to learn the system. It's
something of a final step to getting the rules unmistakable, once normal
developmental playtesting is done.

Quote:
A free game and my name in the credits? Nice, but certainly did not pencil out economically. I gave up a lot of "hobby" time, and "hobby" time has become more precious over the years.

And that was for a game I really liked and wanted to contribute to making better. Blind testing a title I don't have much interest in? Forget about it.

I feel the same way about writing reviews. I wrote a review for Fire & Movement of game I thought I would like, but ended up hating. But I did a good job of the review, which entailed a ton of time (though less than testing). Now I sometimes see calls for reviews, and frankly the prospect of a free copy is not enough to entice me at all. Instead I bang out a review here on BGG when I feel like it, and get paid exactly the same.
Right. And HOW do you get people willing to put (perhaps less) effort
in when they cannot already love the game?


Edgar Gallego wrote:


It's a labour of love, so it's like if it's dead most of times. People spends a lot of time and headaches only to see a polished game/system, sometimes without the free game (it depends on publisher), and some end up preferring to enjoy the finished game hoping others making the sacrifice.

Credits are an acknowledgment, not the objective. And after all the work you have to buy the game (although that's not the usual)

Same issue comes up here. Who's going to put the effort in when
they cannot know the game well? It's less effort - but I think the
hobby badly needs blindtesters. They used to show up in the credits all
the time. I don't see them much anymore.

And all that is ignoring the likelihood that people will be familiar
with a lot of core concepts. But, the hobby is (usually - though modern
designs seem to be trying to be more variable than previously, even
within a principle design like a CDG - it seems that every game needs
to handle movement and combat in different enough ways to not seem
'derivative').
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mbmbmbmbmb
Edgar Gallego wrote:
blockhead wrote:

And that was for a game I really liked and wanted to contribute to making better. Blind testing a title I don't have much interest in? Forget about it.
That sums it up
I'm afraid so. But there MUST be people who are interested in trying
a design. Take the COIN series for example. I think it's important that
blindtesting cannot be considered the same level of commitment that
standard playtesting is. I'm not sure the developers 'get' this though.

Basically, offering pre-publication copies, during the final development stage,
to nail down that the rules are reasonably understandable.
Maybe it's just not worth the effort for them. Letting a decent
game flop - especially with the whole pre-order concept - doesn't really
risk anything. Most decent games end up having a few kinks, but those
can be worked out through living rules or a new edition. There aren't
too many that totally crap out. Better than slowing the publication
pipeline, I guess (which I think is the real cost).


1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Christopher Rush
United States
Iowa
flag msg tools
Avatar
mbmbmbmbmb
I'm interested. How does one get involved in this, other than going to the company Web sites and get on the lists? or is that the only way?
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Charles Vasey
England
Mortlake, London
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
calandale wrote:
Is it dead?

I was communicating with a designer recently, and he asked
(after I had commented on his game not being adequately blind-tested)
if I knew anyone who could do this (presumably well).

Are there wargamers out there - willing to cast a significant amount
of effort into learning and providing detailed comments on a game
in exchange for a (early) free copy and a name in the credits? Or
are the only people willing to do this all tied up in a few projects?
Yes, they exist but not many.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
roger miller
United States
California
flag msg tools
publisher
I was slightly miss understood. When I test for another designer or they for me we are blind testing late in the design. I do not have time to do normal playtesting of a game that is still in development.

As for being a tester of any sort just contact the company. Most will quickly set you up. Mike Rinella of Take Aim Designs has been begging for testers for months.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
I've done it in the past, but it's very time-consuming and a lot of work for a free copy of a game.

I stopped volunteering after a particularly unpleasant exchange with a designer who explained that my criticisms were stupid and worthless. The game went to print without my recommended changes.

The changes were incorporated in the living rules issued some months after the game had been released.
10 
 Thumb up
0.05
 tip
 Hide
  • [+] Dice rolls
Charles Vasey
England
Mortlake, London
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
DeletedUser389997 wrote:
I've done it in the past, but it's very time-consuming and a lot of work for a free copy of a game.

I stopped volunteering after a particularly unpleasant exchange with a designer who explained that my criticisms were stupid and worthless. The game went to print without my recommended changes.

The changes were incorporated in the living rules issued some months after the game had been released.
Thou hast conquered Nazarean!

The designer needs to treat his playtesters with some regard while ensuring his design survives. It is like boiling a small fish.
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Kenneth Lury
United States
Williamsburg
Virginia
flag msg tools
mbmbmbmbmb
I would be a great play tester as being relatively new to the hobby, I find most war games very difficult to learn and spend more time learning than playing.

2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
United States
Vista
California
flag msg tools
badge
Avatar
mbmbmbmbmb
Is there no longer the equivalent of SPI Friday nights?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mbmbmbmbmb
Neopeius wrote:
Is there no longer the equivalent of SPI Friday nights?
That wouldn't work for this though - people taught the games.

The big point here is to get the rules a final shakedown.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Judd Vance
United States
Wichita
Kansas
flag msg tools
Who's the master?
badge
"Just get that sucka to the designated place at the designated time and I will gladly designate his ass...for dismemberment!" - Sho Nuff.
Avatar
mbmbmbmbmb
One of my buddies did that for Dawn of the Zeds (Second edition). I think Victory Point Games will let people blind test them.

I didn't know it was called that. I thought all games were done this way. I have seen a few that look like they are not and I can tell because I ask, "How the heck did they not catch that in playtesting?"

I did it for the first time this year. When I learned about Hands in the Sea, a deck builder on the First Punic War, I jumped at the chance and wrote the designer about playtesting it. Not having a chance to get a lot of face-to-face time in town, I created a Vassal module for it and he and I played it a LOT. I posted sessions and screenshots on the "Wargames on Your Forum", provided quite a bit of feedback and suggestions. Some of those cards are directly from my suggestion (puffs out chest).

When the game gets published, I'll probably have a credit somewhere in the back of the book and a copy of the game, but if it gets Kickstarted, I'll fund it anyways, because I really believe it, and not because I came up with a couple of ideas. I really enjoy it and moved it #2 on my Top 10.

Yeah, the "reward" vs. time spent isn't there financially. That's why I have a career. If I were paid to do it, I don't think I would do as well. If you do it because you love it, you put your heart into it. Plus, I got other rewards: 1) The game was by far the hardest Vassal module I ever built, but it's a doozy. At first, I borrowed a lot of the ideas from A Few Acres of Snow. Now, there is hardly anything in common, because this one is so much better. The only module I have seen that is more automated is the one for Twilight Struggle (the king of all modules, in my book). By creating this, I made myself a much better module creator for future modules. 2) I have a copy of the game and don't have to wait!

It's also been fun to see how a game is created. How much chrome do you add? Who is your audience? Design for effect. Stuff like that.

I had a lot of fun doing it. I won't do it much, but if the topic is of great interest and I personally like the designer, I'd do it again.
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mbmbmbmbmb
airjudden wrote:

I didn't know it was called that. I thought all games were done this way. I have seen a few that look like they are not and I can tell because I ask, "How the heck did they not catch that in playtesting?"
What always shocked me was the degree to which it was not
adequately done by some companies in the past. Especially AH.
I look at things like Republic of Rome and Geronimo, and have
absolutely no idea why the blindtesting feedback didn't tell
them they had really terrible rules.

Then again, with things like The Struggle of Nations, it seems
pretty clear they had it at the printer before they bothered
looking at any playtest feedback (there are errata in the
game's Q&A section); and it never got enough rounds of playtesting
to really hammer it into a reasonable game - much less to pass it
through a blindtest routine. It made more sense (and never was so
extreme) with SPI's often maligned process (though in the 3W days,
S&T managed some doozies).
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Charles Vasey
England
Mortlake, London
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
calandale wrote:
airjudden wrote:

I didn't know it was called that. I thought all games were done this way. I have seen a few that look like they are not and I can tell because I ask, "How the heck did they not catch that in playtesting?"
What always shocked me was the degree to which it was not
adequately done by some companies in the past. Especially AH.
I look at things like Republic of Rome and Geronimo, and have
absolutely no idea why the blindtesting feedback didn't tell
them they had really terrible rules.

Then again, with things like The Struggle of Nations, it seems
pretty clear they had it at the printer before they bothered
looking at any playtest feedback (there are errata in the
game's Q&A section); and it never got enough rounds of playtesting
to really hammer it into a reasonable game - much less to pass it
through a blindtest routine. It made more sense (and never was so
extreme) with SPI's often maligned process (though in the 3W days,
S&T managed some doozies).
The problem with blind testers is they have a short life before they begin to be able to "correct" errors automatically. You need new blood.
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Enrico Viglino
United States
Eugene
OR
flag msg tools
Slowed - BGG's moderation policies have driven me partially from here
badge
http://thegamebox.byethost15.com/smf/
Avatar
mbmbmbmbmb
Charles Vasey wrote:


The problem with blind testers is they have a short life before they begin to be able to "correct" errors automatically. You need new blood.
I guess that depends on the target audience for the game.
I'm not bothered by a few little ambiguities in a game that
no one outside the hobby is likely to approach. The blindtester
is not doing anything more than what a player would do.

Still, even if a player is likely to make such correction, I'd
think a bit of discipline on the testers' part would still allow
them to notice (and note) the clarifications. The fatigue of doing
that is more likely to be a cause of their value degrading. I know
that it's a lot tougher for me to approach a game that way now.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
1 , 2  Next »   |