$20.00
$30.00
$5.00
$15.00
mattle elttam
msg tools
Introduction

Are you curious about how balance is 7 Wonders: Duel? Do you feel like some wonders are stronger and some are weaker? Is there an advantage being first player or second? Is it way too easy or hard to win with Science or Military? In this post, we will answer these questions with real data!

We have been analyzing 450+ human-against-human online game plays in mattle.online (http://sevenee.mattle.online), and here is s short summary of our findings.

Full report and detailed analysis can be found at: http://sevenee.mattle.online/lobby/forum/topic/mvxoybDcABT6Q...

1. Victories Distribution
Of the 463 games collected, 6 were tie, 237 were finished with Civilian, 107 were finished with Military and 113 were finished with Science.




2. First Player Advantage
We found that there is a slight advantage of being the first player, which won 245 times out of the 457 untied games.


3. Wonders popularity
Graph below shows the popularity of wonders. The popularity of wonders is measured in this way: As you know in each game, there are two times 4 wonders will be revealed and a player got to pick the first one. We assume that this "first pick" wonder is what players think is the best among the four, so we calculate number of times being picked first for each wonders. Apparently, "The Template of Artemis" is the most popular wonder!



4. Wonders true power
We then tried to verify whether wonders popularity agrees with the actual strength of the wonders. What we did is to calculate the winning percentage for each wonders. (i.e. chances of winning a game if a particular wonder is on players hand). Unfortunately, "The Template of Artemis" isn't as good as what people perceive.



5. Extra turn ability
We figured that the popular and strong wonders all have one thing in common - the extra turn ability. So next, we want to see how extra turns are affecting the chance of winning. What we did is to calculate the differences in the total number of extra turn symbols between two players, and plot this against the winning percentage. Obviously, the differences in extra turn is ranged from -4 to +4. As expected, there is a strong positive correlation between the two.



Conclusion
In our opinion, the Military and Science supremacies are well designed and pretty balance according to the victories distribution.

The biggest issue in the game is the "extra turn" ability in wonders. If you think about the wonders picking mechanics, one can simply start with 2 more extra turn symbols even both players put priority on getting extra turns. So with pure luck, one can start already with an unfair advantage.

Balancing game strategies is hard, so all in all, we still think 7 Wonders: Duel is well designed. We hope that the analysis can help with the design of the future expansion or rules amendment.

Thank you for reading. Full analysis can be found at the original report mentioned in the beginning.


56 
 Thumb up
6.90
 tip
 Hide
  • [+] Dice rolls
Nick Shaw
United Kingdom
Plymouth
Devon
flag msg tools
designer
mbmbmbmbmb
Very interesting analysis.

I wonder, how different would the results be if you took results just from the highest ranked players, or the lowest ranked, or just new players? I'm assuming this covers the entire spectrum of the user base.

The Wonders analysis is interesting, though did you do any analysis on what combinations of Wonders is the most powerful? e.g. getting all Wonders that give you an extra turn, for example?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jason Webster
United States
Connecticut
flag msg tools
mbmbmbmbmb
Based on the extra turn graph getting wonders that give you extra turns and being able to use the extra turns is very powerful.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Aleksander Idziak
Poland
Warsaw
flag msg tools
mbmbmbmbmb
great job. Thx!
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Kevin Duke
United States
Wynne
Arkansas
flag msg tools
mbmbmbmbmb
Nice wotk!

Given all the posts we have seen that suggest Military and Science are a waste of time, it's lovely to see them winning with a significant--and really, just about perfect--25% of the time each.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Nick Shaw
United Kingdom
Plymouth
Devon
flag msg tools
designer
mbmbmbmbmb
kduke wrote:
Given all the posts we have seen that suggest Military and Science are a waste of time, it's lovely to see them winning with a significant--and really, just about perfect--25% of the time each.


Agreed. I've always thought Military and Science victories were really just a way to prevent one player monopolising a certain strategy and cause a runaway leader, and it looks like that's true - in practice, MOST players end up playing a balance between Military, Science and everything else, to push to end-of-age-3 scoring; but drop the ball on either Military or Science and you'll regret it!
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jack Bennett
United States
Rougemont
North Carolina
flag msg tools
designer
Play. Always.
mbmbmbmbmb
To get nitpicky, 245/457 is not statistically significantly different from 50% (using Wald or Score statistics with alpha=.05). Or just using regular z-scores (this one is 1.543675), you've got an insignificant p-value of .1227 (>.05).

To say another way, 245 out of 457 is close enough to 50% that the data doesn't prove that the first player does have a statistically significant advantage. (See here for a similar discussion [on a different topic] and related math).

EDIT: I hate to be THAT GUY, but to me an actual statistically significant first player advantage would be a pretty negative thing. So it's worth it to me to point out that that the data (so far) doesn't support that! I'd be curious to look at it again with more data.
19 
 Thumb up
5.00
 tip
 Hide
  • [+] Dice rolls
Conan Meriadoc
France
flag msg tools
mbmbmbmbmb
I'd be curious, if we started mixing a bit that data, if we could see patterns emerge.

For example, comparing the percentage of science/military wins vs score victories for games where the first player won, vs. games where the second player won. Is science/military easier when you're starting the game ?

Same thing for games with a 2+ differential in extra turn abilities in favor of the winner, vs games with a 0/1/negative differential. I strongly suspect extra turns are a big factor in science/military wins, maybe not so much in civil victories.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Kevin Duke
United States
Wynne
Arkansas
flag msg tools
mbmbmbmbmb
Jack, I agree that a big first player advantage would get a bad thing.

To get nitpicky about your post, the OP did say "a slight advantage," and I don't think it has to be a lot to count as "slight."
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
tom tom
United States
Springfield
Missouri
flag msg tools
mbmb
In my experience the extra turn wonders are most important in contributing to a science or military victory. Especially in the third age.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jack Bennett
United States
Rougemont
North Carolina
flag msg tools
designer
Play. Always.
mbmbmbmbmb
kduke wrote:
Jack, I agree that a big first player advantage would get a bad thing.

To get nitpicky about your post, the OP did say "a slight advantage," and I don't think it has to be a lot to count as "slight."


Well, the point is that there's no proof of even a slight advantage. The reality of the game could be that even the second player has an advantage and it's still not unlikely to see data like this. All you can say with this data is that the people in this sample won more when they went first, and that there is not enough evidence in this data to support rejecting the idea that going first has no effect.
8 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Scott Douglass
United States
Minnesota
flag msg tools
mbmbmbmbmb
I was pretty sure that Piraes was the best wonder and the Colossus was the worst. It's interesting to see data consistent with my conclusions.

I've been thinking that Piraes would be better balanced giving 0 points, and that the Colossus would be better balanced giving 5-6 points, but I haven't tested either of those ideas. I would still take Piraes a lot of the time, but wouldn't automatically first pick it anymore, and I would take the Colossus more seriously.

I'm surprised that the Temple of Artemis is so popular. I would take any of the other extra turn wonders over it most of the time.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
mattle elttam
msg tools
njshaw2 wrote:
Very interesting analysis.

I wonder, how different would the results be if you took results just from the highest ranked players, or the lowest ranked, or just new players? I'm assuming this covers the entire spectrum of the user base.

The Wonders analysis is interesting, though did you do any analysis on what combinations of Wonders is the most powerful? e.g. getting all Wonders that give you an extra turn, for example?


Yes. It would be informative to do the analysis by isolating high rated players. However, I don't think we have enough data just yet to make it statistically significant. Hopefully, for another month and we will see. Stay tuned!
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
mattle elttam
msg tools
pusherman42 wrote:
To get nitpicky, 245/457 is not statistically significantly different from 50% (using Wald or Score statistics with alpha=.05). Or just using regular z-scores (this one is 1.543675), you've got an insignificant p-value of .1227 (>.05).

To say another way, 245 out of 457 is close enough to 50% that the data doesn't prove that the first player does have a statistically significant advantage. (See here for a similar discussion [on a different topic] and related math).

EDIT: I hate to be THAT GUY, but to me an actual statistically significant first player advantage would be a pretty negative thing. So it's worth it to me to point out that that the data (so far) doesn't support that! I'd be curious to look at it again with more data.


Yes. I also don't want to see that the first player does has advantage. I guess you are right. we'll definitely make another run a month later when we get more data, and then we'll see!
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Chris D
Italy
Muggiò
MB
flag msg tools
mbmbmbmbmb
Interesting stats!

The Piraeus is obviously very strong, but count me between the ones who like Temple of Artemis a lot - it's probably the most versatile wonder, you build it and then you are pretty much guaranteed to take any card (available) you want. Maybe not always first pick, but definitely not last.

The Colossus could probably use an increase in victory points.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
david beebee
Spain
flag msg tools
The idea is interesting but 450 games is such a small sample.
Variance can be quite high, which results in biased conclusions.

I would like to see the same analysis when >2000 games are played.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
M. B. Downey
United States
Alexandria
Virginia
flag msg tools
designer
mb
defdean wrote:
The idea is interesting but 450 games is such a small sample.
Variance can be quite high, which results in biased conclusions.

I would like to see the same analysis when >2000 games are played.


thumbsup

I'd like a run of 20,000 myself. I think you could get a lot of good data by having the AI run against itself a bunch of times and see what you find out. That would also let you improve the AI, and get you different play styles (hate drafting, pushing military/science, opportunistic, etc.)
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Kevin Duke
United States
Wynne
Arkansas
flag msg tools
mbmbmbmbmb
What AI is it that is going to run these 20k games? And what value will they be in comparison to human v human games, which the OP said is what they are measuring and reporting about here?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
M. B. Downey
United States
Alexandria
Virginia
flag msg tools
designer
mb
kduke wrote:
What AI is it that is going to run these 20k games? And what value will they be in comparison to human v human games, which the OP said is what they are measuring and reporting about here?


Their site also offers human versus computer, so they have an AI.

The value is that you can run players of equal skill against each other a statistically significant number of times and see what kind of an effect the first player has. And you can do it much more quickly than you could with humans, unless you just have a TON of people playing.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Russ Williams
Poland
Wrocław
Dolny Śląsk
flag msg tools
designer
mbmbmbmbmb
downeymb wrote:
The value is that you can run players of equal skill against each other a statistically significant number of times and see what kind of an effect the first player has. And you can do it much more quickly than you could with humans, unless you just have a TON of people playing.

The only catch is that in principle the AI might be failing to exploit (or defend against) first player advantage in ways that competent humans would successfully exploit (or defend against).

But I agree that AI testing is a cool and at least somewhat useful tool in the testing kit!
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
M. B. Downey
United States
Alexandria
Virginia
flag msg tools
designer
mb
russ wrote:
The only catch is that in principle the AI might be failing to exploit (or defend against) first player advantage in ways that competent humans would successfully exploit (or defend against).

But I agree that AI testing is a cool and at least somewhat useful tool in the testing kit!


I said the AI players were of equal skill level; that doesn't mean they are skilled.

But you could obviously enhance the AI from the results, and run it again, and then look for more improvements in the results, and run again, and so on until you get the point where it recognizes the advantage and plays to it. Or until you've created a self learning AI that hits the Singularity and destroys the human race; we're all undone by a desire for more board game data.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.