A/Bingo 1.0.0 Official Release

Back in August I released A/Bingo, an MIT-licensed OSS Rails A/B testing framework.  I have been using it continuously on Bingo Card Creator, and judging from the support requests I’ve been getting it has gotten some traction in the Rails world.  The 5,000 or so people seeing A/B tests on my site on Valentine’s Day are almost certainly less than 1% of the beneficiaries of the software now. Yay.

As A/Bingo has grown in popularity, I have begun to get requests for features that I did not need urgently for my own development, as well as the usual support requests, patches, and the like.  I want to make your use of the software as pleasant as possible to further evangelize the cause of A/B testing, so here you go:

New features:

A/Bingo now ships with a default dashboard.  Previously, I assumed that everyone would be writing their own dashboard code, so I just included the absolute minimum to show you what you’d need to do to get data out of A/Bingo.  Many people have remarked that they would really appreciate a “works out of the box” solution.  Your wish is my command — you can now enable a default dashboard in about ~30 seconds. It would work totally out of the box, but there are security implications, so I wanted you to have to think for a moment prior to enabling it.

#Create a new controller.  The name is up to you -- this example uses abingo_dashboard_controller.rb
class AbingoDashboardController  :abingo_dashboard

You can customize the dashboard code yourself. Nota bene: it uses your application layout, and has CSS classes applied to most of the elements, so you can style it quickly with CSS if you desire to. By default, it probably looks terrible. If you want to send me a patch to make it pretty, be my guest.

Experiments can now be stopped: Using either the built-in links on the above controller or, if you prefer programmatically scripting things, experiment.end_experiment!(alternative_content), you can now stop an experiment without touching the code.  Stopping an experiment causes all users to get the specified alternative rather than what they would have gotten randomly.  It also ceases stats collection.  Stopping an experiment is irreversible (currently — that might change later).  I tried to make this feature not affect the performance of A/Bingo for larger sites — it makes each test require one extra cache access.  (*cough* Rounding error, hopefully.)

A/Bingo internals are now fairly thoroughly tested: Unit tests are not exactly my cup of tea (“Argh, it works in production, what else do you want from me?!”), but Rails developers look askance at software that does not include them.  So I knuckled down and wrote a test suite.  (Hat tip to Nathaniel Talbott for mentioning A/Bingo in a conference presentation.  The constructive criticism regarding testing drove this change.)

I have not written thorough integration tests for the syntax sugar that you get via the included helper methods, but I’ll fix that eventually.

Named conversions: Previously, all A/Bingo tests required one line to add the test and one line somewhere else to track conversions.  Typically, since businesses have very many tests and fairly few conversion events, this resulted in code like:

#A controller method
def purchase
#Business logic goes here.
  bingo!("new_button_test")
  bingo!("email_copy_test_january")
  bingo!("microcopy_test")
  bingo!("button_colors")
  bingo!("login_button_alignment")
end

That isn’t very DRY at all.

Now, A/Bingo will take an optional parameter :conversion (or :conversion_name) when you’re defining a test, telling it to listen to a particular named conversion. This way, you can reuse the same conversion for as many tests as you want, decreasing the lines of code needed to create most new tests from two to one.

def some_method_with_a_test
  @alternative = ab_test("some_test_name", %w{altA altB}, :conversion => "purchase")
end

def some_other_method_with_a_test
  @foo = ab_test("bar_test", %w{coke water}, :conversion => "purchase")
end

def purchase
  #Business logic goes here!
  bingo!("purchase")  #Calls conversions for both of the above tests.
end

A/Bingo handles tests with spaces in them more gracefully: Although I still don’t recommend doing it, A/Bingo has been improving its handling of test names which have a space in them.  (The reason I don’t recommend it is because some cache stores — particularly memcached — do not support this well.)

Official support for Redis: Assaf Arkin picked Redis for his awesome Vanity project (which also does A/B testing for Rails, among other things), which inspired me to take a look into it.  It appears to be a much, much better alternative for a key/value store than Memcachedb, which is what I use for persistence.  A/Bingo has always accepted any cache store that Rails does, but I want to make it explicitly clear that I run tests against Redis, Memcached, and MemcacheDB. Just add the following to your environment:

#Goes in environment.rb
config.gem  'ezmobius-redis-rb',
  :source => 'http://gems.github.com',
  :lib => false

config.gem  'jodosha-redis-store',
  :source => 'http://gems.github.com',
  :lib => 'redis-store'

#Goes in whatever environment you're using:
require 'redis-store'
Abingo.cache = ActiveSupport::Cache::RedisStore.new

I intend to migration my own deployment to Redis when it becomes reasonably convenient for doing so.

Versioning: Previously I’ve just released patches to the A/Bingo git repository when I got done coding them, but I feel that is suboptimal now that there are substantial deployments which I could potentially break with changes.  So, here’s the skinny: A/Bingo is now, as of this blog post, 1.0.0.  I’ll communicate breaking changes by bumping that number up.  If it goes up by a tenth or more, expect that you need to re-run the migrations and that you will probably lose data on any tests in-progress, so plan ahead for that.  Version increases in that last number should be safe to apply directly.

I do not anticipate breaking the published A/Bingo API (i.e. methods mentioned in the docs) until at least v2.0.0, if ever, so upgrading A/Bingo should almost never cause you to need to update your own code.

How To Contribute

I would like to thank everyone who has submitted bug reports and patches. As usual, I’m always happy to get bug reports or feature requests. If you’d like to contribute code, make it available via git anywhere you please, and then send me an email telling me about it.

How Do I…

If the question isn’t answered in the (copious) documentation, feel free to ask me over email. If your business has particular needs for A/Bingo or you just want to talk A/B testing strategy with somebody who breathes it, I’m available for consulting engagements starting April 1st.

You Should Be Doing A/B Testing

I really can’t stress this enough: A/B testing is an easy, reproducible process that you can use to improve your marketing, website copy, product, user experience, etc. If you haven’t started yet, take A/Bingo, Vanity, or your other framework of choice for a spin. It won’t take you five minutes until you’re getting actionable data which you can use to make money.

No Responses to “A/Bingo 1.0.0 Official Release”

  1. P.J. Hinton February 14, 2010 at 6:19 am #

    (pauses for a moment and recalls your post from 12/31/2009)

    Would this release announcement have been more effective had it been expressed in terms of benefits to the user rather than new features?

    Regardless, congrats on the 1.0.0 release!

  2. Patrick February 14, 2010 at 6:36 am #

    Probably! However, the intended audience here is mostly folks who are already on A/Bingo, and wondering “What does this new release mean for me?” Answer: it breaks things now (potentially — most users won’t see breakage) so I don’t have to break things later.

    The A/Bingo site itself is focused on the needs of a person new to A/B testing or evaluating whether to use A/Bingo or not for their Rails testing needs. You’ll note I hit the benefits early and often there.

  3. Peter Harkins February 14, 2010 at 8:39 am #

    Would you consider creating or blessing an official repo on GitHub? It’s the go-to spot for Rails coders and I didn’t think twice before I looked there first for a copy of the code to start testing. There are four copies of A/Bingo there for taking, none up-to-date and most with changes.

    The git-multipush script ( http://code.google.com/p/git-multipush/ ) makes it convenient to push to both your existing repository and a new one.

  4. Patrick February 14, 2010 at 8:57 am #

    The fact that you’d look for my code on Github in preference to my own site is exactly why I will not ever release A/Bingo on Github.

    We’re all businessmen here, right? In addition to an abiding desire to spread the light of A/B testing throughout the world, I kind of like getting the links, attention, and job offers associated with publishing a piece of OSS in an underserved niche which has high commercial value. Putting my software on Github does not help me get links, attention, and job offers.

    If you would like to see it on Github, you’re welcome to fork it and put it in your account.

  5. Cullen February 14, 2010 at 1:18 pm #

    Thanks for your contributions to OSS! I use A/Bingo on http://ridewithgps.com with decent success. So far it’s been little tests showing good results, but I am excited to take it further.

    I hadn’t thought too much on posting OSS tools I create on github, figuring it was just a good default place to post them. However, I see the benefits of bringing eyes to your site in order to use your software. If you are not charging for use of the software, it’s a decent way to at least get exposure and climb in pagerank in lieu of payment!

    Anyway, thanks for making this handy tool available.

  6. Jim Jones February 14, 2010 at 2:31 pm #

    Patrick,

    Thanks for this great contribution. I plan on trying it out. This is a bit off topic, but I’m curious: what is your Rails dev environment setup like (OS, editor, tools, etc). I’d really enjoy a post on the topic. I’m actually a desktop mISV/developer that’s been learning Rails.

    Thanks again.

  7. Peter Harkins February 14, 2010 at 10:46 pm #

    I had thought of pushing to GitHub as making your code and you easier to find and as adding to the number of links to you, but I understand your perspective.

    Thanks for sharing the code.

  8. Martin Jones February 15, 2010 at 2:49 am #

    This makes me very jealous… I wish there were a similar plugin for grails (my chosen web framework)! Have you written a super-high-level overview of how the plugin works that might help someone to “port” it? I’m not sure how tricky this would be in practice – depends on how similar grails/rails are under the hood. Guess I should probably just read the code….

  9. tighten vagina June 22, 2011 at 1:08 am #

    Cool, book marked sit down and can share with my own freinds who also have a tabata mp3 that they use.

  10. bingo blog October 20, 2011 at 3:40 pm #

    I do trust all of the ideas you’ve presented in your post. They are really convincing and will certainly work. Still, the posts are very quick for beginners. May you please lengthen them a little from next time? Thank you for the post.

Trackbacks/Pingbacks

  1. Listen to A/Bingo 1.0.0 Official Release - MicroISV on a Shoestring - Hear a Blog - February 16, 2010

    [...] http://www.kalzumeus.com/2010/02/14/abingo-1-0-0-official-release/ [...]

Loading...
Grow your software business:
(1~2 emails a week.)