Regular readers of this blog know I’m a bit obsessive about testing and measuring small, iterative improvements to my website. Previously I used Google Website Optimizer but I found it had some annoying limitations. In particular, it was too much work to start tests, too much work to code tests, and too much work to maintain expired URLs after tests were over.
So I went away to the code salt mines for the last week, and coded my own A/B testing framework in Ruby on Rails. It is called A/Bingo . (The name sort of popped into my head and wouldn’t leave. It is a pun, it relates to my business and personal brand, and it gives the impression of Hitting the Target that successful testing should give you. My designer took that and ran with it for the logo. I’m very impressed with the results.
A/Bingo is, without question, the most technically impressive code I’ve ever written.
- Setting up a test takes one line of code. It is so easy you’ll find yourself doing it automatically when writing copy or making new features.
- It tracks absolutely any event as a conversion, in one line of code.
- It does statistical significance testing for you, producing human-readable output which tells you exactly how confident you can be that the test results are accurate.
- It is fast. No, it is fast. Like, “A/Bingo could handle being linked from the front page of a social news site. Heck, it could be used on the front page of many social news sites.”
If you want to read any more about it, I encourage you to take a look at the website. A/Bingo is already powering all the Bingo Card Creator A/B tests. I expect that the ease of use and copious features will mean I run a lot more tests. I’ll continue sharing what I learn through them.
P.S. Incidentally, I scored a 16.9% improvement to funnel completion thanks to Mixpanel.
Patrick, very easy implementation of A/B testing for rails – where’s the .NET version? ;-)
You mention that you are A/B testing the “Get the code” link using bold or italics. It changed on me from italics to bold when I changed pages. Shouldn’t it always stay the same for me?
Never mind. You’ve fixed it…
Thanks for sharing your hard work! Unfortunately your git server seems to be down. Do you have an alternate download link for A/Bingo?
Hiya Pius. I got the git server back up — it went down when this server got crushed by an avalanche of traffic yesterday, and it seems that it didn’t restart automatically when the server did. Try again now.
Patrick, A/Bingo looks amazing. I have a JS-heavy app and I’ve been struggling with Google Analytics trying to do funnels and A/B testing.
I figured I would do a quick google search to see if there was anything better, A/Bingo + mixpanel look awesome together. I have to try them out!
Thanks for developing the plugin and all the great documentation!
Thanks Remi, I’m glad you liked it. I’m going to be integrating Mixpanel into the default A/Bingo distribution later.