Hello, everyone! Micah, here, back with a thrashing terrible sinking into all this draft data that everyone has submitted (I would call this a deep-dive but, that seems to more in RNGEternals wheelhouse). For those of you who have never submitted a deck, lets talk about what the data looks like when we receive it.
The Timestamp is used as the unique identifier for a draft deck, the rest is somewhat self-explanatory. We record wins, losses, current rank, and the decklist. The current rank was added after the initial announcement of the project due to some user requests, but I’m not sure it is relevant enough to continue collecting when the next set comes out.
I will admit that I had only an inkling of a plan on how to handle the data when we began collecting it. Trying to drink from a firehose is an apt description of the sensation. I knew from the beginning that I wanted to see if we could connect deck win-rates to individual cards. To do that I need a way to cleanly break out all those decklists because I was 100% not going to do it by hand. Enter Trifacta Wrangler.
Once the data is processed in Trifacta we get the wonderful data tab in the spreadsheet you all know and love. A few pivot tables and sheet mergers later we get that beautiful RESULTS tab. We now have win% for cards, but is that information useful? Let’s see.
I've got some Bad News
The biggest issue with our data is that we don’t have enough of it. Even at 1000+ submissions, there are still cards that haven’t shown up more than 1 or 2 times. The 2nd biggest issue is that it has biases.
To solve the issue of enough data requires DWD to effectively grow the game, and for us to spread the word about the project. One of those things we can control, so tell your friends! Don’t tilt and then not submit! Don’t submit just your 7-win runs! I’m as guilty as anyone else of forgetting to submit, or being so frustrated by a crushing draft, that I just close the game. But I think we can do better. Now about those biases…
That is a pretty strange looking bell curve! So we have an obvious bias toward 7-win runs. It so striking that I would assume that the majority of users are only submitting their 7-win runs. And that is a pretty big problem for the integrity of the data.
The other bias is not as apparent and that is the child__roland and mann_und_mouse effect. CR’s submissions are around 10% of the data and MuM’s is probably just below that. So ~20% of the data comes from two users (maybe this explains the 7-win skew?). Although to be fair to MuM, 1/6 of his data is chat draft.
I think the 7-win bias can be overcome with careful analysis. I’m not so sure what to do about individual users having such an impact. The only thing that can really be done is to continue to reach out to players and ask them to start submitting.
The win percentage of a given card does trend toward the mean as the number of appearances increases. However, there are outliers that rise above.
Xenan Initiation is the best uncommon in the format. It’s appeared 103 times and has a win rate of 71%. That’s almost a full 10% over the median! The best rares are probably either Waystone Infuser or Xenan Obelisk. They both have almost 30 appearances and are both well above the median. Banish would probably take that spot if it wasn’t in Shadow. The best common is Awakened Student, followed closely by Temper (~20 more submissions and we’ll have a better understanding of just how good temper is). The worst? Cripple.
We’ve still got a way to go with the analysis, but I hope you find this at least interesting. I would LOVE to hear everyone’s thoughts on the project and where to go from here. Next installment we’ll talk about faction pairs!