Recommend
7 
 Thumb up
 Hide
30 Posts
1 , 2  Next »   | 

The Resistance» Forums » Strategy

Subject: Using Excel to find spies rss

Your Tags: Add tags
Popular Tags: [View All]
Bob Costas
msg tools
Hey guys,

The last couple days I've been working on a spreadsheet that attempts to assign a "spy rating" to players based on the following:

Mission failed while leader
Mission succeeded while leader
Voting against a successful mission
Voting for a sabotaged mission
How often a mission has failed with their involvement
How often a mission has succeeded with their involvement
How often they've voted down a mission into disapproval

The formula I've come up with is the following:

[Leadership Fail + Mission Fail + .1(Mission Disapproval) + Vote Down Successful Mission + Vote Up Failed Mission)] / (Mission Success + .1(Leadership Success) + 1)

Do you guys have any ideas on how to get a more accurate reading out of this? I'd appreciate any help you guys can offer.

-TK
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Joe Kundlak
Slovakia
Bratislava
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
You need a cold shower and a different game
17 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Matt Vollick
Canada
St. Thomas
Ontario
flag msg tools
badge
Avatar
mbmbmbmbmb
Try additive models first and use multiple regression to come up with the weights.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
In that case I'm going to need more sample games...four games just isn't going to cut it.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jeremiah Lee
United States
Milan
MI
flag msg tools
designer
publisher
badge
Avatar
mbmbmbmbmb
Is this supposed to tell is how good they are at the game, or how likely they are to be a spy?

I really like either idea, though I'm not sure I'm the guy to offer much help.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Travis Worthington
United States
California
flag msg tools
designer
publisher
badge
2010 Releases ........................................ The Resistance, Haggis & Triumvirate ..................................... Now accepting submissions for 2011 releases ........................................ www.IndieBoardsandCards.com
Avatar
mbmbmbmbmb
I would add some contextual analysis - mathematically that might be represented as voting in the minority on a team proposal.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
Yes, that's why I have "voting against a successful mission" and "voting for a sabotaged mission" as factors to take into consideration.

Jeremiah_Lee wrote:
Is this supposed to tell is how good they are at the game, or how likely they are to be a spy?

I really like either idea, though I'm not sure I'm the guy to offer much help.
This is supposed to determine how likely a person is a spy based on their actions.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Travis Worthington
United States
California
flag msg tools
designer
publisher
badge
2010 Releases ........................................ The Resistance, Haggis & Triumvirate ..................................... Now accepting submissions for 2011 releases ........................................ www.IndieBoardsandCards.com
Avatar
mbmbmbmbmb
tommyknocker2121 wrote:
Yes, that's why I have "voting against a successful mission" and "voting for a sabotaged mission" as factors to take into consideration.

But that is only measuring votes that get approved and go on the mission.

There are many votes that don't get approved, and I would bet there is some correlation between being on the minority side of a vote, regardless of the outcome.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
T Worthington wrote:

But that is only measuring votes that get approved and go on the mission.

There are many votes that don't get approved, and I would bet there is some correlation between being on the minority side of a vote, regardless of the outcome.
But it is taking count of the minority vote. If they voted against a mission that was successful they were, by default, in the minority. And vice versa of course.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Travis Worthington
United States
California
flag msg tools
designer
publisher
badge
2010 Releases ........................................ The Resistance, Haggis & Triumvirate ..................................... Now accepting submissions for 2011 releases ........................................ www.IndieBoardsandCards.com
Avatar
mbmbmbmbmb
tommyknocker2121 wrote:
T Worthington wrote:

But that is only measuring votes that get approved and go on the mission.

There are many votes that don't get approved, and I would bet there is some correlation between being on the minority side of a vote, regardless of the outcome.
But it is taking count of the minority vote. If they voted against a mission that was successful they were, by default, in the minority. And vice versa of course.
only for those subset of votes that a majority approved.

it ignores all the votes that a majority did not approve.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
T Worthington wrote:
only for those subset of votes that a majority approved.

it ignores all the votes that a majority did not approve.
Did you mean # of times a player voted for a mission that was disapproved?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Travis Worthington
United States
California
flag msg tools
designer
publisher
badge
2010 Releases ........................................ The Resistance, Haggis & Triumvirate ..................................... Now accepting submissions for 2011 releases ........................................ www.IndieBoardsandCards.com
Avatar
mbmbmbmbmb
tommyknocker2121 wrote:
T Worthington wrote:
only for those subset of votes that a majority approved.

it ignores all the votes that a majority did not approve.
Did you mean # of times a player voted for a mission that was disapproved?
yes


ultimately I think you would need to collect information that wasn't available at the time (ie, votes against teams with no spies), and do so from many different groups to be able to do the regression analysis against the entire data set to find any correlation on factors that are only discernible with pubically available information to get meaningful results.

From experience, the game is played very differently in different groups.
4 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
I've grabbed the play by play from four of the games from the play by play forum so far. I think that there's ten or eleven complete games there. The ones that I've inputted from start to finish have anywhere between six and ten players.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Travis Worthington
United States
California
flag msg tools
designer
publisher
badge
2010 Releases ........................................ The Resistance, Haggis & Triumvirate ..................................... Now accepting submissions for 2011 releases ........................................ www.IndieBoardsandCards.com
Avatar
mbmbmbmbmb
tommyknocker2121 wrote:
I've grabbed the play by play from four of the games from the play by play forum so far. I think that there's ten or eleven complete games there. The ones that I've inputted from start to finish have anywhere between six and ten players.
Not sure that play by forum has any relationship to in-person games.

Also there would be a lot of information that was exposed via plot cards - would need to build a relationship map to understand that - just as you would need to build out the relationship with players that went on a failed mission.

For example If you and I go on a failed mission, and I am resistance then I know that you are a spy and will vote against any mission that has you on it. That information has to be included in a deep analysis.

It might be the case that a lower level analysis could be predictive in 60% of the cases - ie, it is marginally useful.

 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Agent J
United States
Coldwater
Michigan
flag msg tools
He's looking real sharp in his 1940's fedora. He's got nerves of steel, an iron will, and several other metal-themed attributes. His fur is water tight and he's always up for a fight.
badge
He's a semi-aquatic egg-laying mammal of action. He's a furry little flat-foot who'll never flinch from a fray. He's got more than just mad skills, he's got a beaver tail and a bill.
Avatar
mbmbmbmbmb
I love the fact that this thread even exists.

I moreso love the fact that I have used Excel to try and find spies...

I think you're looking at it backwards, unless you're planning to play online.

Basically, you are trying to find spies in active games based on what they do, correct? You should be trying to figure out WHAT SPIES DO and then apply that to a game you are playing in person.

I agree with Travis about online play though, it is completely different than FTF, but both have their charm. Well, most games do.

Edit:

Actually, thinking about it, I think that's what Travis was getting at too. But really, it seems like everyone wants to make up a formula or program based on what they think spies do and then apply to see if it works, when really we should be looking at what spies do consistently and then getting a formula out of that. A lot of that should end up being the talking, social aspect, because spies have information that the Resistance does not and has a hard time hiding this fact.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Clyde W
United States
Washington
Dist of Columbia
flag msg tools
Red Team
badge
#YOLO
Avatar
mbmbmbmbmb
Agree. You guys ever read the play Arcadia? In it, two characters are working on the same problem, only one is working forwards and the other is working backwards. One is trying to draw pictures from equations, while the other has a whole cabinet full of hunting "game books" (records of who shot what each year for 200 years) and is trying to deduce the mathematical equation that governs grouse mating.

Anyway, I too wrote a Python script to do something very similar to this. You should check your results via the Play by Forum data, in any case. But I agree with Jyth, we're asking the wrong questions I think.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Clyde W
United States
Washington
Dist of Columbia
flag msg tools
Red Team
badge
#YOLO
Avatar
mbmbmbmbmb
T Worthington wrote:
ultimately I think you would need to collect information that wasn't available at the time (ie, votes against teams with no spies), and do so from many different groups to be able to do the regression analysis against the entire data set to find any correlation on factors that are only discernible with pubically available information to get meaningful results.
My Python script does this, although I quite sure I wouldn't call what I do "regression analysis". I just "assume" a certain set of players are spies then score that set based on a series of tests. So if no spies were put onto the mission and all of the "spies" (under the theory) voted no, then this adds points to the guess. If they all vote yes, then that theory gets negative points, etc. If one spy was on the mission and the other spies vote yes, then this gives that theory more points, etc. It's very rudimentary, and prone to weighing people who're on a lot of missions as spies accidentally, but it works pretty well against the PBF data.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Colin Sham
Canada
Toronto
Ontario
flag msg tools
That's exactly what a Cylon would say!
badge
All is dust...
Avatar
mbmbmbmbmb
With enough data you can feed it into a neural net and train it. Since neural nets are fundamentally just weighted decision nodes, it's effectively creating an equation (though one with so many variables that you'd never be able to understand the reasoning for any of the weights). This would allow you to model closely the results of your equation... but much more easily.

It would be a worthwhile research project to see how well this works in the same gaming group, and how well it works across many different groups. That would at least prove static groupthink (though since people also evolve it might not be sufficient).

*shrug* I'm not a computer scientist by any means, and only took a course in AI back in University that was merely an introduction. Someone more knowledgeable would likely torpedo some of my assertions as inapplicable due to some overlooked detail.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
It's just that. My spreadsheet attempts to use all the criteria of the spreadsheet and aggregates it into a single number that indicates "spy-ness." The formula above has been scrapped and am still working on finding the proper coefficients.

I also thought that a person being on a failed mission would be the perfect criteria for this. Interestingly enough, preliminary data seems to indicate a very low correlation between the two. At first I thought I was doing something wrong but then I realized that a good spy will never sabotage a mission with only two operatives. Since there are usually more rebels than spies on a failed mission, it skews the data and makes it...not as useful.

Right now, with the small sample I have, voting patterns are the best indicator of "spyness." Thanks to Travis' intuition and getting me to include that.

Again, this is all very preliminary and need more data to go on.

Would anyone like to help me out by using my spreadsheet to record their games? If nothing else it's a great way to keep track of the game. I just ask that people email me a copy of the completed spreadsheet.

-TK
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
Using only six games as a sample, I can get ~70% success rate in capturing spies. In three games the success rate was 100%

I think I just need a larger sample size
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Clyde W
United States
Washington
Dist of Columbia
flag msg tools
Red Team
badge
#YOLO
Avatar
mbmbmbmbmb
Raid1280 wrote:
I have to ask:

What is the purpose of this 'spy rating'?

This seems like it would give numerical value to the statement "He's likely a spy because he went on two missions that failed."

My ears begin to bleed every time I hear that phrase in a game.
My script doesn't really do that. It values the voting record pretty much more than anything, though it does give some slight credence to the fact that that player had been on two different fails. (But it'll give nearly as much weight to the fact that that player had been on two passes as well!)
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Christopher M
United States
Goleta
California
flag msg tools
badge
Awesome Possum
Avatar
mbmbmbmbmb
I approve of this project. If you need more data, a friend of mine has been keeping track of our games and has about 60-70 games recording in detail. Geekmail me and I can send you the data I have.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
1awesomeguy wrote:
I approve of this project. If you need more data, a friend of mine has been keeping track of our games and has about 60-70 games recording in detail. Geekmail me and I can send you the data I have.
Thanks, I've sent you the info.

Raid, thanks for the encouragement

Seriously though, I hear what you're saying -- I'm not 100% convinced that the tool will be more than marginally effective at what it's meant to do. If nothing else it will be a kick-ass way to record games.

Plus I'm having fun with it and putting a log of skills to use that I haven't since grad school.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
If anyone is interested I've uploaded the recording spreadsheet here: http://boardgamegeek.com/filepage/75690/resistance-recording....

Again, I hope you'll email me a copy of your completed games and if not, I hope you get some use out of the sheet.

Thanks!
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bob Costas
msg tools
Has anyone actually used my spreadsheet? How are they working out for you all?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
1 , 2  Next »   |