Archive | web design RSS feed for this section

Minor Lessons from A/B Testing

I am going hog wild with A/B testing now that I have an easy way to do it.  I thought I’d share some of the results:

  • Login as guest link vs. No login as guest link:  There is no significant difference in the number of trial signups I get if I put a discrete login as guest link on the registration form.  There is, however, a massive decrease in the number of guests (obviously, they have to decline by at least 50%, right?).  While I don’t get nearly any economic value out of guests, I reverted to allowing them.
  • Asking for permission to email versus not : There is no significant difference in the number of trial signups I get if I show or hide the two checkboxes for signing up for the mailing list.  Note that I still ask people for their email address (which I use for a username, figuring this is easiest for non-technical people to remember) and that I, stupidly, included the “We do not spam you” verbiage in the block excised for the test.  That should have been left in both alternatives.  Ahh well, I’ll do it again properly later.
  • Reordering buttons:   Reordering the bingo card pages (see example) to include the Make Your Own Cards (which leads to a trial registration dialogue) over the Download These Cards buttons increases the conversion to the trial from 11.90% to 19.82%, which was significant at the 90% confidence level.  Yay, a positive result!  I ended the test and adopted that behavior as the default.

What I’m testing currently:

  • A new version of the purchasing page, which is specific to the online version only, for people who are logged into the online version (versus the current purchasing page).  Obviously I’m looking for actual sales as a conversion here.
  • The effect of the text “It is quick and easy to get started!” versus “There is nothing to download and no credit card required.” on the landing page.  (Quick and easy are two concepts that have worked very well in my AdWords before.)

Introducing A/Bingo: Rails Split Testing

Regular readers of this blog know I’m a bit obsessive about testing and measuring small, iterative improvements to my website.  Previously I used Google Website Optimizer but I found it had some annoying limitations.  In particular, it was too much work to start tests, too much work to code tests, and too much work to maintain expired URLs after tests were over. 

So I went away to the code salt mines for the last week, and coded my own A/B testing framework in Ruby on Rails.  It is called A/Bingo .  (The name sort of popped into my head and wouldn’t leave.  It is a pun, it relates to my business and personal brand, and it gives the impression of Hitting the Target that successful testing should give you.  My designer took that and ran with it for the logo.  I’m very impressed with the results.

A/Bingo is, without question, the most technically impressive code I’ve ever written.

  • Setting up a test takes one line of code.  It is so easy you’ll find yourself doing it automatically when writing copy or making new features.
  • It tracks absolutely any event as a conversion, in one line of code.
  • It does statistical significance testing for you, producing human-readable output which tells you exactly how confident you can be that the test results are accurate.
  • It is fast.  No, it is fast.  Like, “A/Bingo could handle being linked from the front page of a social news site.  Heck, it could be used on the front page of many social news sites.”

If you want to read any more about it, I encourage you to take a look at the website.  A/Bingo is already powering all the Bingo Card Creator A/B tests.  I expect that the ease of use and copious features will mean I run a lot more tests.  I’ll continue sharing what I learn through them. 

P.S. Incidentally, I scored a 16.9% improvement to funnel completion thanks to Mixpanel.

Landing Page Design Tips

Recently, I finally got around to making a proper landing page for Bingo Card Creator.  It has been fairly successful for me so far, increasing conversions markedly (my historical average for AdWords is about 18%, that page converts at about 28%) and decreasing my cost per conversion.

Here’s what I think I did right:

Focus On A Single Goal

The stock Bingo Card Creator homepage, which has been the landing page for all of my advertising since time immemorial, has to serve a lot of purposes at once.  It signs people up for the trial.  It drives people to the download.  It sells the benefits of Bingo Card Creator.  It provides an entry-way into support.  It lets people log in.  It funnels around link juice for SEO purposes.  It, like the prototypical uISV, wears fourty-seven hats.

Landing pages should have one goal.  My new one aims to capture a sign-up to the trial of the online version of the software.  That is it.

To accomplish this:

  1. Strip out all navigation which conflicts with The One True Goal.
  2. Eliminate visual distractions from The One True Goal.
  3. Make it absolutely, blazingly visually obvious what The One True Goal is.

If you guessed that the goal for the page is clicking either the giant purple button or the single visible hyperlink, you win!

Include Trust Markers

Bob Walsh has gone over this a million times referring to his MicroISV Sites that Sell book: testimonials work.  I have never really used them to their full potential, but I have diligently collected a little notebook (with permission, naturally) from my 1,800-odd customers. This gives me a bit of a palette to work with.  While normally I personally prefer hitting customers with a few testimonials at a time, like this wonderful example, I didn’t have too much space to work with.  So I picked a single testimonial, lightly edited for length and content, and bolded the important bits.

I cannot stress enough how much people do not read on the Internet.  Accordingly, if you want to maximize your sales, you should adapt to users that:

  • skim more than they read
  • feel more than they think
  • enjoy bad self-referential humor  (OK, this one is optional).

Seriously speaking: bold the bits of the testimonials which sell your software.  Cut the bits that don’t, as long as you can do so without making the testimonial say something the customer did not.  They are not professional marketers and should not be expected to write like them.  You are, so give them a minor hand.  (Polishing it until it is too good is another danger.  Testimonials should remain authentic.)

It should go without saying but in addition to reflecting on you, a testimonial reflects on your customer.  Even if you are making them largely anonymous, extend them the courtesy of copy edits to make their words suitable for public distribution.  If I never see [sic] in a testimonial again it will be too soon.

Don’t Forget the Guarantee

I don’t feel like elaborating on this, other than to say I once walked out the door without pants on.  That is a pretty serious oversight, but it still isn’t as bad as forgetting your trusty money-back guarantee.

Address Your Shadow Audience

Google likes to see certain things on landing pages and hates to see others.  You really don’t want to tick them off.  Most of my readers are honest businessmen, which automatically saves you a lot of the headaches, but there are a few subtleties:

  1. Trust markers like a privacy policy, return policy, etc go a long way.
  2. Don’t look like an affiliate.  Google is very ambivalent about affiliates — they make billions off of them, but to the maximum extent possible Google would rather disintermediate the affiliate, because they feel that a) it provides a better customer experience and b) Google should be getting the money the affiliate does.
  3. Landing pages have a tendency to repeat large swathes of text, which could cause duplicate content problems for your site.  This is particularly acute if you build them on an industrial scale (about which I will have more to say later).  Save yourself some pain and either exclude your landing pages from the index using robots.txt or include the following code in your header somewhere:
<meta name="googlebot" content="noindex, noarchive, nosnippet">

Technically speaking there are more search engines than Google to worry about.  Hah.  Sorry, where was I.

Provide Continuity

Marketers sometimes say “information scent”, but I like continuity: seeing your ad, clicking on your ad, reading your landing page, and then interaction with The One True Goal should be one continuous flow, free of jarring transitions and mental static.  For example, you should have your ad call to action anticipate the call to action text on your landing page.  Thus:

Notice how I’m focused on the proposition of “quickly”, with the call to action “Try now, no download required!”

That matches my call to action on the landing page:

This means that for a large portion of my visitors, I get the click to the purple “for free” : they’re already mentally invested in trying now, no download required.  That is why they came here.  So I don’t have to convince them so much with the other text on the page.

Of course, that doesn’t help much if I lose them at the signup form.  Candidly, my signup form could use some work — it performs much, much better than my download/install rigamarole ever did, but it severely needs a graphical upgrade and some interaction design TLC.  However, there is one trick to it that I’m fairly happy about.  I’ll share that with you guys this weekend, as it is a little subtle and I want to collect a few more days of data to show you that it actually works.

Preview of coming attractions: I have often said that paying a freelancer to write content for my website was probably my best ROI ever.  This resulted in over 700 pages which are very responsive to exactly what searchers want.  What if it had also resulted in 700 ads?

Keeping The User Moving Towards Conversion

Recently on the Business of Software forums I gave someone advice regarding web design.  (I seriously, seriously envy his design prowess — the website for his VST host software is one of the best I have ever seen for a microISV.)  One issue I identified with his site was that at some points it lacked clear direction as to what to do next.

On reflection, there is a gaping hole in my own site design for this: the big screenshot on the front page.  It is ginormous and consistently picks up over 10% of the clicks on the page.  Clicking on it currently brings up a maximized version of the screenshot in a lightbox.  The lightbox is a wonderful effect, but what is the user supposed to do with the maximized version?

Well, ideally, they’re supposed to close that screenshot, then find the Free Trial or Try Online buttons and download.  Blech!  That is two more clicks, bringing to three the total of clicks they have made to my page!  Clearly, there is an opportunity to eliminate a click. All I need to do is bring the buttons to them, in the constrained environment of the lightbox.  (I’ve always wanted to do this but didn’t have any convenient way to do it with my previous lightbox technology, and never got around to doing it after moving to a less constrained lightbox script.)

I have pushed the new and improved version of the page live.   If this post was published recently, you can see the old version here.  For obvious usability wins like this one, I’m not going to bother A/B testing prior to pushing it live.  (Plus I have a lot on my plate tomorrow in terms of non-obvious things, and running multivariate testing scares me.)

Practical Conversion Tips For Selling Software

Today after reading a thread or three and watching a Google-produced movie about landing page optimization (over an hour long and worth your time) I was inspired to make some changes to my site.  Recently I have been doing far more development than A/B testing, and truth be known I don’t really, um, know how my funnel is working these days.  So I went back into the archives of my CrazyEgg tests and looked around to see if there was anything that jumped out at me. The fresh look caused me to pick up on a few things:

Plug Your Leaking Funnel

If you’ve ever read anything about conversion optimization you’ve heard the funnel metaphor : the steps towards buying can be thought of as a funnel, where a lot of people come in at the top and as they move forward you see less and less people until a mere trickle actually succeed in giving you money.  You want your funnel to avoid leakage, to the maximum extent possible.

Bingo Card Creator has a fairly typical funnel:

Prospect sees me in search engine or ad -> Prospect clicks to homepage -> Prospect downloads trial (or signs up for online trial) -> Prospect plays around with trial -> Prospect likes what they see -> Prospect clicks on purchasing page -> Prospect interacts with shopping cart -> Prospect gets redirected to Paypal or Google Checkout -> Prospect fills out form -> I get money.

Yikes, saying it like that makes it sound even longer than it actually is.  Anyhow, every step is crucial because the effects are multiplicative: a 5% improvement anywhere essentially results in 5% straight to the bottom line.  (Hat tip to Steve Pavlina for this insight, which is probably one of the three most important things I know about selling software.)

Anyhow, I picked a step in the funnel where I see a lot of dieoff, the purchasing page, and took a gander to see if there were any obvious ways to improve it.  Looking at several experiments in CrazyEgg, which I collected over the course of the last few years, something jumped out at me.  See if you can spot it in these heat maps:

Early 2007:

Early 2008:

Early 2009:

Did you spot it?

Far too many users interact with the main navigation bar to go backwards in the funnel.  Despite the fact that the overwhelming majority of people on this page are here because they are satisfied users of the free trial and are seriously considering giving me money, the Free Trial and Printable Bingo Cards buttons capture almost as many clicks as any other element on the page.

I love giving users free things that make them happy, but there is a time and place for it, and that time and place is not right in the middle of me asking them for money.  So I eliminated the distracting free options (and, while I was at it, the redundant purchasing option):

(You’ll note, by the way, that a lot of online retailers take a weedwacker to their entire navigation system when you’re getting into the purchase funnel.  Check out how much cruft Amazon cuts out next time you checkout there.)

Don’t Violate User Expectations

As an engineer I love elegance… probably to a fault.  For example, I always wanted to have exactly the same top-level navigation on all my pages.  This resulted in having a button to the home page actually on the home page.  OK, perhaps that wouldn’t be the worst thing in the world — except in a fit of stupidity way back in 2006 I titled that button “Information”, and despite four people telling me “You know, people seeing a button titled Information will click it thinking that it offers Information and will get upset when it just causes their screen to blink”, I never got around to changing that.

Sell What You Sell

Recently I released an online version of Bingo Card Creator, which I strongly hope will increase my sales substantially by decreasing the funnel dropout associated with downloading and installing BCC.  However, my site is still largely written to promote the downloadable version… to an almost painful degree sometimes.  For example, if you were to click on the purchasing link from within the upsell in the free trial of the online version, you’d find:

(Incidentally, that phrasing used to be “Purchase a single copy for only $24.95″.  I changed it at the suggestion of Aaron Wall, to emphasize the benefit — you get what you want instantly — over the action.  I think that sentence has been worth over a thousand bucks.)

However, when you’re selling something which is specifically designed to avoid the hassles associated with being downloaded, offering instant download is perhaps not the best idea!  So I rewrote it again to be a bit more neutral among my products:

If I had an absolutely infinite development budget I’d code up one version of the site to only sell the online version, one to sell both, and one to sell only the downloadable version, and then start split testing like a madman.  However, that is just not in the cards in the near future.  If I have some time later I’ll consider rewriting this page for people who are logged into the online version, though.

Optimizing Offsite Payment Processors

If you’re a member of SEOBook and have not yet read leeds’ post “Great conversion tactic for you”, do so now and implement the suggestion it makes immediately. (He asked me not to repeat the advice publicly.  I’m going to respect his wishes on that one.)

However, it did get the creative juices flowing:

Usual Disclaimer

I have confidence that the above examples are good ideas but confidence loses to evidence every day and twice on Sunday.  Conversion optimization is a science, not black magic: test, test, and test again to see what works in your particular market and circumstances.

Disclaimer: I am a moderator at SEOBook, but am not compensated for plugging it.  That said, it costs about $100 a month and I made that back on the first bit of advice I got.  It gets my two thumbs up.  CrazyEgg would require minimally an extra seven thumbs to express how awesome it is for visualizing user behavior — see the above heat maps.

If you found this post interesting, you may like my other posts about web design, conversion optimization, and the like.  (I don’t mean design in the sense of pretty things: I have all the artistic skill of a blind mole rat.)  Also, try searching the blog for “conversion rate” and the like, there have probably been a few posts which escaped my uneven categorization efforts.

A/B Testing Lightbox Screenshot vs. Screenshot + Download Link

Everybody paying attention to this blog recently knows I have an unhealthy fascination with Lightboxes, particularly my new favorite iBox.  I like them because they work really well at increasing conversions, most recently in my shopping cart.  (Incidentally, guys, I went back and reset the test.  It really is about 100% improved over the old version.)

So this week I removed the old Lightbox lightbox from my home page and replaced it with an iBox.  I haven’t looked in any rigorous fashion whether that improved things or not — I just did it to cut a few kilobytes off all the Javascript people have to download to use my site, and also because I think the iBox looks cooler out of the box.

Then I noticed that the iBox introduces a loading screen, which the old lightbox did as well.  I hate load screens.  When I see them I think “This is me, losing money”.  Because if two seconds to load the shopping cart cost me half of my potential sales then 2 seconds to look at a screenshot must likewise be suboptimal, and MANY people look at the screenshot (something like 20% of the people who view the page, if CrazyEgg is counting right).

Also, unlike Lightbox, iBox can cause you to get dumped direct to image if you click before the script loads.  Bad news, bears, that means a bounce for most of my users.  I got around this by putting onclick=”false” on the relevant links.

Many, many moons ago (in late 2007!) I experimented with adding blue instructory text to the lightboxed image, exhorting people to download the trial.  That was a crashing failure — it looked like a link but if you clicked it, wham, the image vanished.  But I sort of like the general idea, so it is time for a new split test — take a gander at the screenshot behavior here and tell me if you like it.

You could see the old version of the behavior on my homepage but, well, only if you flip heads the first time you load it.  It is basically the stock lightbox effect — brings up an image, click anywhere to dismiss the image.

I’m hoping for a lift in trial downloads, particularly from clicks coming from advertising.  My worry is that the people are going to get stuck in the lightbox and bounce, since it is much harder to dismiss now.  Oh well, that is what testing is for.  Bring on the visitors!

New Mini-Development Project For Me

I have a deep, abiding affection for e-junkie, which is my payment processor of choice.  So much so that I’ve been called their Local Sales Rep on the Business of Software forums.  Probably true — have I mentioned they’re the best $5 a month you will EVER spend in your life?  Sorry, had to do some shillin’ like a villain to make up for what I’m about to say.

I’m thinking of not using their shopping cart anymore.  Not because it isn’t an amazing piece of technology which manages to work for thousands of people without writing code, because it is all of that and more.  The lift I got in conversions from using it has made me thousands, which for about 5 minutes of integration work means the hourly effective wage is sweet indeed.  Its just that my web programming abilities are improving these days, and I’m starting to chafe at a few of the necessary restrictions from using a third party solution.

My big issues:

1)  Forcing people who buy a CD to put in their zip code twice — once in the cart, once at the actual checkout screen.  I’m working on the assumption that that probably kills CD conversions, and I’m not sure those conversions come back

2)  It is slow.  Clicking on the cart gives you a loading message for two or three seconds.  I still think it beats the pants off of a pageload when updating a shopping cart, I just want the perceived speed to be “bugs in my teeth fast”.  (This is where people ask me “Well, why do you need a shopping cart at all when you can just have a Buy Now button?” and where I say “I’ve tested both alternatives and the cart makes me oodles of money by simplifying the customer experience.”)

I was partially inspired to do this by my buddy Yakob at Mixed In Key, who a) has something really cool coming down the pipe which if you are reading this blog will interest you intensely and b) has a really sweet shopping cart implementation.  Try it, you’ll like it.  That’s what I want mine to work like.

I think I can repurpose some of the Widget Factory Javascript code to achieve it fairly easily.  The actual transaction will still be handled by e-junkie, but I want the cart constructed locally.  If I can do it fairly cleanly, I’ll open source this so that anybody else can apply it to their sites — otherwise, I’ll just do the usual kludging and split test it against the genuine e-junkie cart on mine.  I’m thinking the lift in conversions is going to be sizeable enough to justify a few hours of fighting with Javascript (once).

Sidenote: My dashboard informs me that I might hit $3,000 in sales this month.  Whee for new records!  Recession, what recession, 73% year over year growth.  (Watch me jinx myself.)  

What’s doing it?  Same old same old:

  • AdWords doing well as it always does near secularized holidays (Valentine’s Day bingo is always hot, and while I’m not winning that SERP the people who are thankfully slap AdWords all over the place)
  • organic SEO efforts on website paying off (~1,200 visitors from Google on average weekday these days)
  • version 2.51 converts pretty decently (subtle changes, big difference in effects, tell you about it later)
  • a few tweaks to the website are paying nice dividends

CrazyEgg Makes Me Serious Money… Again

It has been a really long time since I did any testing of webpage elements and I was thinking “Hmm, what am I paying $20 a month for CrazyEgg if I haven’t logged in since summer?”  So I sat down for a moment and thought of something I could add.  Ahh, I know — maybe I’ll try adding another download button after my benefits spiel and before the ginormous 566×545 pixel sample bingo card, whose scroll-inducing enormity comes before the one visual download button in my main body.

Four days later I come back to CrazyEgg and see the result of 4,200 visitors:

86 clicks, or 10.4% of clicks on the page, on that one stinking extra button.  This increases the conversion of my main page by about 20% (some clicks are presumably “stolen” from other ways to reach the same goal).  Since my main page is also my AdWords landing page this started decreasing the amount I was paying for conversions instantly, and of course increasing the number of trials downloaded will hit the bottom line pretty darn directly.  The naive assumption is that 20% more trials is 20% more sales.

Sadly, my main page does not account for 100% of conversions, but my quick back-of-the-envelope calculation is that the main page causes about 40% of trial downloads directly, which implies that it drives something like $800 of revenue a month, so a 20% increase in that is $160 added to the bottom line (plus assorted cost savings from AdWords which I will not get into because the math gets heady).

Putting a button there is kind of jawdroppingly obvious… I’m not sure why I didn’t do it before.  Sure, there is one button visible to the left when you open the page, and the first text link in the content area is to the download (I have previously tested this and highly recommend it, because the first text link in the content area gets a huge portion of user interaction).  But if someone scrolls down to actually read the benefits list, then they’re left high and dry with nowhere to click unless they scroll past the huge image or scroll upwards.  Neither of those activities “flows” well after you’ve just read a benefits list, not nearly as well as “click here” does.

Well, lesson learned. 

I don’t think I’ve said this in a while: I love you, CrazyEgg.

Speeding Up Web Page Rendering

I have the good fortune to be on a fast, fast, fast Japanese connection and as a result don’t typically wait too long to view webpages.  This makes me forget sometimes that the rest of the world doesn’t get nigh-instantaneous page loads, which means I don’t often design with this factor in mind.  However, somebody pointed out that the new sidebar buttons were loading last for them, and as my front page can potentially take 4 seconds to load (longer than a significant portion of visitors stay!) I didn’t want that.

Why is a bit of a long story.  First off, the sidebar is literally the last thing in the HTML body element (with the exception of various bits of Javascript).  This is to push up the main content area in the document, because search engines treat stuff near the top as more important and I want pages to rank for their own content not the content of pages they happen to link to on the sidebar.  This means that, if you parse the webpage starting at the top and going down, you come across those IMG tags pretty much dead last after everything else on the page.

Now, how do browsers download assets (images, Javascript, CSS, and whatnot)?  Basically, there is a FIFO (first-in, first-out) queue maintained per host name.  For example, if I have the web page reference 10 images on, then those are listed from first occuring to last occuring in the queue.  The browser then, following HTTP specifications, starts downloading two at a time per queue.  This means that if you have, say, 25 objects to grab from your own domain and 1 offsite Javascript the offsite Javascript will start downloading about as soon as its discovered while the last object from your domain waits in line.

So that is the technical explanation.  What was happening was my front page loads 2 fairly sizeable screenshots totaling 200kb of the total 350kb required to load the page, and as these are both ABOVE the sidebar when you’re reading HTML linearly, they were blocking the download of the buttons.  That is certifiably ungood, particularly as one of the screenshots isn’t even visible when you open the page (it is far below the fold, most people won’t even see it!)

However, since the queue is maintained per hostname, if I could only host on a hostname other than, I could have things download in parallel.  Happily, my webserver (Nginx) makes it really simple to set up extra names like, say, which are essentially the same as the main domain in every way except they are, well, named differently.  Thus tricking your browser to download from them in parallel.  I set up 4 domains this way and used that to do a bit of manual queue optimization to ensure that the images with the highest payoff (big beautiful buttons!) load fastest. 

And to think I was wondering why so few folks tracked in CrazyEgg were hitting the images compared to before.  Now I know — they were waiting so long they had already found a text link before engaging the image.  The results are visibly different even on my monster Japanese connection.  (Its like the Godzilla of latency… in a good way?)

Rails offers a way to do this automatically, but I wouldn’t recommend it if you have only a few pages to hand optimize.  The reason is simple — randomizing access to domains results in good average case load orders but if you can hand-optimize things you blow it out of the water.  (For example, there is about a 40% chance that the large image would block one of my conversion buttons with random host names.)

Pressing My Customers' Buttons

In my continuing desire to split test the heck out of everything, and my continuing total inability to graphically design to save my life, I went back to the LogoSamurai folks and got some buttons designed for the website as drop-in replacements of my current ones.  I’m also getting some made by my web designer Gursimran, and will be testing tNew buttons to testhe two against each other (and, of course, against the current set) and keeping the ones which are most successful at driving the conversions. 

Gursimran isn’t done with her set yet, but since the Logo Samurai guys are, I thought I’d give you a sneak peek.

As you can see, these are much more Web 2.0 than most web sites in my niche, and quite different from the previous look&feel for the Bingo Card Creator site.  That’s why we test them, of course — I don’t want my opinion on what my customers want to calcify and blind me to the actual facts of what they act upon.  Gursimran has a very different visual style so I’m intensely interested in seeing how things work out.  (Sidenote: I love the pencil!)

Cost for these, incidentally: $60, $15 per button.  I had to convince Gursimran to take money for the buttons, as she wanted to just throw them in with the price I already paid for the web design.  As great a deal that is, I would feel really terrible imposing on her like that — professionals should not be afraid to charge money for services, in my opinion.

Plus, if the winning set of button increases my trial conversion rate by 1% and/or my purchase conversion rate by 1%, that would work out to be quite a bit of money.  I also like keeping all of my freelancers happy with me, as I feel it will tend to get me work done in a timely and high quality manner, and thus prevent me from having to go play Russian Roulette with the freelancer sites to find a replacement source of talent.