So You Think You Can Test?

Decision making under uncertainty is complicated business. This game aims to make decision makers more aware of the complex trade off between indecision and acting on insufficient information.


You will be presented with a series of ten independently simulated experiments. Only one of these is simulating a scenario where there is a difference in effect between base and variant. The other nine are so-called A/A experiments, with absolutely no difference between the two treatment groups. Because of random fluctuations, combined with the fact that we observe the experiments as they are simulated, all ten result will seem to change over time.

Your job is to identify which of these experiments is not an A/A.

Once you are certain you've identified the right experiment, make your choice known by clicking (or by using the numeric keys "0" through "9"). With every subsequent level, the size of the simulated effect will diminish, making your task progressively more difficult. A point is awarded for correct decisions; mistakes will cost a point. When visitors reaches zero, the game is over.

This is a game of chance, but not of chance alone. Good luck!

Switch to simulation-mode
realistic turn-based simulation

Created by Lukas Vermeer — Make this more awesome on GitHub

Paradox Of Choice, hmm? Worse Than Random You Win Some, You Lose Some Okay, I Guess Great Job, No Really That's Awesome! Can You Do Better? Easy Does It Best Probable Score. Seriously. You Can See The Matrix!

You made {{level-1}} decisions and scored a total of {{score}} points. That's awesome!

Just remember that not making any decisions is also a choice. You might not have made any mistakes, but you've also not really added any value to the company. Perhaps you were sleeping? Sure, you were decisive, but sometimes it is better to wait a bit before you make a decision. Maybe try playing another round and this time stick to making good decisions only, hmkay? Keep in mind that this game is all about making trade offs. You cannot win, but you can probably do better than this. You can probably make a few more good decisions, or a few less mistakes. This is not an easy game. You should be proud you managed to get a positive score. While it is possible to do better than this, it will not be easy. Are you up for the challenge? This is a very good score! Looks like you made all the right choices, but perhaps you can do even better than this if you make all the right choices just a little bit faster? This is a very good score! Looks like you made a few costly mistakes along the way. Perhaps you can do better if you wait just a little bit longer when you\'re not quite confident you're making the right decision? We've done the math, and the odds of getting a score even higher than this are absolutely astronomically small. This is as close as you'll ever be to beating this game, but perhaps you'd like to try just one more time? What you've achieved is statistically very improbable. You probably cheated, but you might also simply be The One. If you manage to do this a second time, we'd be very impressed.

Created by Lukas Vermeer — Make this more awesome on GitHub

Effect {{(1/Math.pow(2,level-1)*100).toFixed(2) + '%'}}

Score {{score}}

Visitors {{time.toLocaleString()}}

Experiment {{exp.experiment_id}}