In my recent post about Paypal’s new icons I mentioned that I was going to use CrazyEgg to check whether the icons were more loved or not than the old ones, and I am, but I decided to go the extra mile and use Google’s Website Optimizer to do a split test. A split test is when you randomly send half of all prospects to one version of a page and half to another to determine, in a rigorously scientific and statistical manner, which of two alternatives is better. They’re typically a pain in the hindquarters to accomplish, but this one wasn’t so bad, thanks mainly to a new feature that Website Optimizer includes.
Previously, you had to markup the bejeezus out of your web pages to Optimize them, which harms some user experience (if the bracketed portion of the page loaded slowly, congrats, you lose) and took far too long. Now, while thats still an option for multivariate tests, Google has a simpler option — make two pages, put two bits of Javascript in the first (front and end of the file) and one in the second, and then put a tracking code on your conversion page, and Google takes care of the rest via transparently redirecting folks who hit the first page into the second with 50% probability.
They recommend that you leave the second page up indefinitely, because folks could conceivably link or bookmark it. I first thought that was good advice. Then I realized DANGER WILL ROBINSON having two pages on your site with 95% similar HTML is an excellent way to get smacked down by the duplicate content penalty, and that would hurt me oh-so-much more than getting a modest bump in conversions from rigorous testing.
Happily, there was a simple one line fix to my robots.txt file that I could make to ward off any possibility of that:
#Somewhere above here we have the “User-agent: *” or “User-agent: Googlebot” line
Disallow: /name-of-my-file.htm
After I’m done with the test, I’m going to use .htaccess to 302 redirect the alternate page to the main page rather than leaving the alternate up forever, which will keep any links or bookmarks good without forcing me to keep an outdated page around and consistent with the rest of my site for eternity. Why do more work?
Anyhow, if you want to see the site, take a gander at my purchasing page which has a fifty-fifty shot of actually showing you the old purchasing page.
Interesting stuff, and a good read.
A safety point though, from a guy who has to pick up the pieces for people who try this sort of thing:
Don’t EVER use a 302 if you can achieve your objective any other way.
Mr G just hates them; and when he finds one in your htaccess it means “We’re gonna watch this guy”, or sometimes “Bosh! Minus 30 penalty for you pal.”
Thanks. I had meant to say 301 redirect, which to my knowledge are safe. The numbers, they all run together some days…
Patrick, I know this is an old post, but keeping testing is ongoing.
I was wondering how you could improve your purchase page. Some ideas to test (of course I do not know what you have already tested)
a, currently your purchase page is great for the competitive and methodical buyers. Test how many of your visitors tend to be more in the humanistic group – need faces to see (have more testimonials – maybe rotate, make them more quote like, add thumbnail photos to testimonials, show classroom pic of happy bingo players, etc.)
spontaneous: add a short video, some pics of topic bingo cards (baby shower, Xmas, etc.)
For these changes you may need a wider page – it would be needed anyway, right? A wider left hand column could accommodate pics, etc.
Limited offer – I would test the CD combined version with a 1-3 day limited offer (on and off), e.g. minus $2 if you buy it today!
Basically, also because I’m a more visual type, I’d expect more visual impact from the whole website. :) But that’s me, so a test would tell the truth.
Anna