Recommend
9 
 Thumb up
 Hide
20 Posts

Android: Netrunner» Forums » General

Subject: Post your data analysis requests here rss

Your Tags: Add tags
Popular Tags: [View All]
Bryan Goodwin
United States
West Linn
Oregon
flag msg tools
badge
Avatar
mbmbmbmbmb
I've created a scrubbed & validated dataset (early notes on method here) from the most recent OCTGN data dump which I intend to release to the community soon. It includes a few value-adds such as per-player Elo ratings and a few calculated fields to make analysis easier.

My intention for this dataset is to provide our community of analysts a common source that they can base their work on. Everyone's been rolling their own reports from the source file, with varying degrees of rigor; this can make it difficult to verify results or run analysis to support counter-arguments.

-----------------------------

So my question to everyone is: "What would you most like to have examined using this data?"

(even if it's been looked at in another context, there are several prior claims that could bear validation/a fresh look with player skill data)

A lot of suggestions get buried in the threads following an analysis post, and I'd like to gather them in a single spot for public reference. Post what you want to know, and why; use thumbs to indicate your interest in other people's questions. If your question's been included in an earlier analysis, post a link to the thread the original results were posted in.

I'm not saying that I'll necessarily be the one to respond to a request, but I'll probably poke into a few of the more interesting ones. Others may pick up ideas here for their own analysis.
6 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bryan Goodwin
United States
West Linn
Oregon
flag msg tools
badge
Avatar
mbmbmbmbmb
(reserved for 'pinning' requests and recording threads where they're answered)

* Average points conceded with a win: by identity; by skill level

* Relationship between player skill and flatline incidence

* Correlation between corp skill & runner skill (distinct side Elo)

* Compare player's 'overall' Elo to their per-ID Elo; examine resulting ID rankings (minimum 8 games req'd for Elo, will limit available population by narrowing to ID level). Check across skill levels.

* For flatline results: Corp APs, Runner APs, AP difference. Validate MagicDave conclusions (http://thesatelliteuplink.blogspot.co.uk/2013/09/psychograph...).

* Win rate matrix: All Corp, all Runner. Also indicate flatline results. Average APs also?
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Christopher Barnett
United Kingdom
Northwich
Cheshire
flag msg tools
Avatar
mbmbmbmbmb
Is it possible to work out average agenda points conceded by identity when said identity wins, would be interesting to know when considering matchplay.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Captain Frisk
United States
Connecticut
flag msg tools
badge
MIND|GAMES
Avatar
mbmbmbmbmb
1. Do flatline victories decrease as player skill increases?
2. If you calculate ELO seperately for a players runner and corp - how much correlation is there between runner and corp skill?
3. If you calculate ELO seperately for a player + identity, what is the effective ELO shift per identity. (ie, if my elo as gabe is 1250, and my average elo is 1200, then gabe is +50).
4. Does the identity rankings in 3 change based on player elo?
5 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
John Thornby
United Kingdom
University of Warwick
West Midlands
flag msg tools
badge
Avatar
mbmbmbmbmb
Agenda points scored when winning by flatline.
i.e. verify (or not) Magicdave's stats from last year that suggest that Jinteki do not win by pressuring the runner into mistakes at 5-6 APs.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
John Thornby
United Kingdom
University of Warwick
West Midlands
flag msg tools
badge
Avatar
mbmbmbmbmb
Runner ID vs. Corp ID matrix of winrates.
You could throw into this matrix average AP for/against and % flatline.

Edit: The formatting of my example doesn't work, so I removed it.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Zak Jarvis
United Kingdom
flag msg tools
mbmbmbmbmb
xenakis wrote:
Agenda points scored when winning by flatline.
i.e. verify (or not) Magicdave's stats from last year that suggest that Jinteki do not win by pressuring the runner into mistakes at 5-6 APs.


This data will be much more interesting if presented as:
Corp APs on flatline;
Runner APs on flatline;
AP difference on flatline.
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
John Thornby
United Kingdom
University of Warwick
West Midlands
flag msg tools
badge
Avatar
mbmbmbmbmb
popeye09 wrote:
xenakis wrote:
Agenda points scored when winning by flatline.
i.e. verify (or not) Magicdave's stats from last year that suggest that Jinteki do not win by pressuring the runner into mistakes at 5-6 APs.


This data will be much more interesting if presented as:
Corp APs on flatline;
Runner APs on flatline;
AP difference on flatline.


Yes, that's what I had in mind.
And obviously it would be nice to see that for each Corp ID (vs. each runner ID would be nice too, as a matrix, but I reckon the sample sizes will be pretty small by that point).
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Andrew Keddie
Wales
flag msg tools
Avatar
mbmbmb
Do 'strong players' who PRIMARLY play Jinteki have a better success rate than people who play a mix of Corp IDs (or 'strong players' who primarily play another Corp but occasionally dip into Jinteki)?

Basically testing the hypothesis that Jinteki requires a different skillset that doesn't translate well to other Corp playstyles.

EDIT: Sorry, to clarify this should be do they have a better success rate WITH Jinteki than players who don't focus on Jinteki as much.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Rana Puer
United States
Michigan
flag msg tools
Avatar
mbmbmbmbmb
I would just like to see data in general on which cards (not necessarily identities) get better/worse at different ELO levels.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bryan Goodwin
United States
West Linn
Oregon
flag msg tools
badge
Avatar
mbmbmbmbmb
ranapuer wrote:
I would just like to see data in general on which cards (not necessarily identities) get better/worse at different ELO levels.


OCTGN data doesn't track down to the cards-in-deck level; would be cool to have though.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
John Thornby
United Kingdom
University of Warwick
West Midlands
flag msg tools
badge
Avatar
mbmbmbmbmb
It would be useful to filter (if you haven't already) to remove certain decks.

For instance - remove all decks with an illegal amount of influence; remove any using 0 influence (most likely core starters and/or players with no clue that you don't want skewing the data). Probably also cut any decks with crazy deck sizes (since large deck = inefficient it's probably a n00b who has trouble cutting so many good cards).
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bryan Goodwin
United States
West Linn
Oregon
flag msg tools
badge
Avatar
mbmbmbmbmb
xenakis wrote:
It would be useful to filter (if you haven't already) to remove certain decks.

For instance - remove all decks with an illegal amount of influence; remove any using 0 influence (most likely core starters and/or players with no clue that you don't want skewing the data). Probably also cut any decks with crazy deck sizes (since large deck = inefficient it's probably a n00b who has trouble cutting so many good cards).


It's got all that and more! Here are most of the pre-filters (I'll write up properly when I post the data):

Remove where:

* Runner deck size < Identity min
* Runner inf > Identity max
* Corp deck size < Identity min
* Corp inf > Identity max
* Corp Flatlined = yes
* Duration < 0m
* Duration >= 180m
* Runner Score < 0
* Runner Score >= 10
* Corp Score < 0
* Corp Score >= 10 (except NBN - Beale)
* Remove all games involving a player with <= 3 games in the entire data set
* Remove all games involving a player who has only played against a single opponent in the data set
* Corp deck size >= (Identity min + 15)
* Runner deck size >= (Identity min + 15)
* Corp deck size > Identity min (also allows min + 4, +9, +14)
* Runner deck size > (Identity min + 4)
* Corp inf < (Identity min * 2/3)
* Runner inf < (Identity min * 2/3)


There's some gating based on dates as well.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Captain Frisk
United States
Connecticut
flag msg tools
badge
MIND|GAMES
Avatar
mbmbmbmbmb
Miaowara_Tomokato wrote:

* Corp deck size >= (Identity min + 15)
* Runner deck size >= (Identity min + 15)
* Corp deck size > Identity min (also allows min + 4, +9, +14)
* Runner deck size > (Identity min + 4)


These rules seem like they are redudant, the 2nd rules should always supercede the previous.

On corp side, i don't think that a 41 card new NBN deck that just refused to cut 1 last ice should be ruled out.

Also - a professor deck that intentionally includes > 50 cards to maximize its ability should probably also not be excluded.

I'm sure at the end of the day you aren't filtering all that many games with these things though.

 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bryan Goodwin
United States
West Linn
Oregon
flag msg tools
badge
Avatar
mbmbmbmbmb
Captain_Frisk wrote:
Miaowara_Tomokato wrote:

* Corp deck size >= (Identity min + 15)
* Runner deck size >= (Identity min + 15)
* Corp deck size > Identity min (also allows min + 4, +9, +14)
* Runner deck size > (Identity min + 4)


These rules seem like they are redudant, the 2nd rules should always supercede the previous.

On corp side, i don't think that a 41 card new NBN deck that just refused to cut 1 last ice should be ruled out.

Also - a professor deck that intentionally includes > 50 cards to maximize its ability should probably also not be excluded.

I'm sure at the end of the day you aren't filtering all that many games with these things though.



They are redundant; they're both listed because I perform two separate cuts: one to remove the straight-up illegal decks, and then another to remove the ones where they fall afoul of the 'competitive deck' filter, which is a judgment call. I keep the data culled by the second pass in a separate table so it can be reviewed later, e.g. to see if those results are indeed significantly different from the 'competitive' group.

There are about 100k results in the final table. Several thousand rows of the oldest game data get dropped immediately because the deck size and agenda count values weren't initially tracked by OCTGN, so there's no way to tell the good from the bad based on the above criteria. Fortunately, the oldest data is the least relevant. The 'competitive' filter knocks about 16k games out of the data set; there are a lot of games with under-utilization of influence or bloated decks.

I'm willing to make future carve-outs for things like a 41-card deck or a larger Professor deck, provided there's community support for such changes (the goal of all of this being a generally accepted/standardized way of looking at OCTGN data). Once I post a thread making the data available, I'll take recommendations for updates and maybe post some polls to gauge support for any changes.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Lysander
United States
Georgia
flag msg tools
Avatar
mb
If you have time, I'd like to request for comparative purposes:

Win% of NBN : Making News at 45 and 49 card decks
Win% of NBN : TWIY* at 40 and 44 card decks
Win% of Jinteki : PE at 45 and 49 card decks
Further breaking down the wins by flatline and AP victory would be further helpful.


Currently there's no concrete evidence whether 40 or 44 cards is statistically more ideal in TWIY*. I have a running theory that a 44-card TWIY is better by adding a 4/2 and switching to a 45 card MN deck (its ability would then turn on ice (Caduceus, Draco) and other tools (Bernice Mai) plus you'd get 3 extra influence). Following this theory, 40 cards would be a fair amount superior since TWIY is currently beating MN in win percentage. Running the numbers on Jinteki may lead to some hypotheses on Harmony Medtech, though the card pool will be drastically different at that point.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim Vaduva
United States
Michigan
flag msg tools
mbmbmbmbmb
Miaowara_Tomokato wrote:

* Runner Score >= 10


It doesn't happen very often, but I've had legitimate games where a run on archives has gotten me over 10 points. I would not filter these games out. Especially with Weyland and Jinteki dropping a bunch of agendas in archives for nefarious reasons.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Captain Frisk
United States
Connecticut
flag msg tools
badge
MIND|GAMES
Avatar
mbmbmbmbmb
tvaduva wrote:
Miaowara_Tomokato wrote:

* Runner Score >= 10


It doesn't happen very often, but I've had legitimate games where a run on archives has gotten me over 10 points. I would not filter these games out. Especially with Weyland and Jinteki dropping a bunch of agendas in archives for nefarious reasons.


In desperation the other day, I played a blind Accelerated Diagnostics. My opponent had just scored 1 agenda, was vulnerable to double punitive death, i had 2 agendas in hand, so I figured WHY NOT?

Since I'm posting in this topic, I think you'll all know what went into archives.
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bryan Goodwin
United States
West Linn
Oregon
flag msg tools
badge
Avatar
mbmbmbmbmb
tvaduva wrote:
Miaowara_Tomokato wrote:

* Runner Score >= 10


It doesn't happen very often, but I've had legitimate games where a run on archives has gotten me over 10 points. I would not filter these games out. Especially with Weyland and Jinteki dropping a bunch of agendas in archives for nefarious reasons.


Fair enough, I'll loosen the runner score rule. This one was only hitting a very small number of results anyway.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
dan dargenio
United States
Philadelphia
Pennsylvania
flag msg tools
Avatar
mbmbmbmb
Lysander1 wrote:
If you have time, I'd like to request for comparative purposes:

Win% of NBN : Making News at 45 and 49 card decks
Win% of NBN : TWIY* at 40 and 44 card decks
Win% of Jinteki : PE at 45 and 49 card decks
Further breaking down the wins by flatline and AP victory would be further helpful.


Currently there's no concrete evidence whether 40 or 44 cards is statistically more ideal in TWIY*. I have a running theory that a 44-card TWIY is better by adding a 4/2 and switching to a 45 card MN deck (its ability would then turn on ice (Caduceus, Draco) and other tools (Bernice Mai) plus you'd get 3 extra influence). Following this theory, 40 cards would be a fair amount superior since TWIY is currently beating MN in win percentage. Running the numbers on Jinteki may lead to some hypotheses on Harmony Medtech, though the card pool will be drastically different at that point.


You'd need to control for elo to make these results mean much.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.