Comments on: CrazyEgg To The Rescue Again http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/?utm_source=rss&utm_medium=rss&utm_campaign=crazyegg-to-the-rescue-again Patrick McKenzie (patio11) blogs on software development, marketing, and general business topics Sat, 11 Feb 2012 04:17:24 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Patrick http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2575 Patrick Mon, 19 Apr 2010 03:59:07 +0000 http://www.kalzumeus.com/?p=884#comment-2575 Zack: see the A/Bingo source code. The math behind A/B testing is also covered here: http://elem.com/~btilly/effective-ab-testing/ Zack: see the A/Bingo source code. The math behind A/B testing is also covered here: http://elem.com/~btilly/effective-ab-testing/

]]> By: zackattack http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2574 zackattack Mon, 19 Apr 2010 03:52:48 +0000 http://www.kalzumeus.com/?p=884#comment-2574 Hi Patrick, Can you link us to the formula for your one-tailed T tests? Do you have an app for that, or what? Thanks, Zack Hi Patrick,

Can you link us to the formula for your one-tailed T tests? Do you have an app for that, or what?

Thanks,
Zack

]]> By: Patrick McKenzie http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2573 Patrick McKenzie Wed, 07 Apr 2010 13:39:17 +0000 http://www.kalzumeus.com/?p=884#comment-2573 Jeff: Users are randomly assigned to one of the two treatments and consistently see that for. For details, see the A/Bingo docs. I don't do usability tests with users prior to deploying things live. Statistical significance. I use, if I recall correctly, one-tailed T tests. Jeff: Users are randomly assigned to one of the two treatments and consistently see that for. For details, see the A/Bingo docs.

I don’t do usability tests with users prior to deploying things live.

Statistical significance. I use, if I recall correctly, one-tailed T tests.

]]> By: Jeff http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2572 Jeff Wed, 07 Apr 2010 12:25:37 +0000 http://www.kalzumeus.com/?p=884#comment-2572 In your A/B approach, are you randomly using one or the other for different users or did you go a period of time with A and then B (Hopefully for an equal period.)? Did you do any usability tests first with a few users? And how many more clicks does it take to justify making any changes? In your A/B approach, are you randomly using one or the other for different users or did you go a period of time with A and then B (Hopefully for an equal period.)?

Did you do any usability tests first with a few users?

And how many more clicks does it take to justify making any changes?

]]> By: Anne Gunn http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2571 Anne Gunn Thu, 01 Apr 2010 14:52:56 +0000 http://www.kalzumeus.com/?p=884#comment-2571 You've probably seen this but I think it is a great post. It relates to your point that you can't just rely on numbers/stats, you have to actually think about what is going on. http://blog.asmartbear.com/data-interpretation-mistakes.html p.s. Happy first day of going full time. You’ve probably seen this but I think it is a great post. It relates to your point that you can’t just rely on numbers/stats, you have to actually think about what is going on.

http://blog.asmartbear.com/data-interpretation-mistakes.html

p.s. Happy first day of going full time.

]]> By: Patrick http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2570 Patrick Thu, 01 Apr 2010 02:58:59 +0000 http://www.kalzumeus.com/?p=884#comment-2570 Now there is an idea... Now there is an idea…

]]> By: Stuart http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2569 Stuart Sun, 28 Mar 2010 21:34:04 +0000 http://www.kalzumeus.com/?p=884#comment-2569 Have you tested the LP without a sidebar at all? Have you tested the LP without a sidebar at all?

]]> By: Sylvain http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2568 Sylvain Sun, 28 Mar 2010 07:25:11 +0000 http://www.kalzumeus.com/?p=884#comment-2568 Very interesting. Thanks for sharing. One thing that's getting my attention is the vertical "hit" line on the right hand side of the side links. I wouldn't have guessed that so many people would try and click away from the supposed link text. Probably worth considering to extend the link or even put a small clickable icon at the right of every item. Cheers, Sylvain Very interesting. Thanks for sharing.

One thing that’s getting my attention is the vertical “hit” line on the right hand side of the side links. I wouldn’t have guessed that so many people would try and click away from the supposed link text. Probably worth considering to extend the link or even put a small clickable icon at the right of every item.

Cheers,
Sylvain

]]> By: Listen to CrazyEgg To The Rescue Again - MicroISV on a Shoestring - Hear a Blog http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2567 Listen to CrazyEgg To The Rescue Again - MicroISV on a Shoestring - Hear a Blog Tue, 23 Mar 2010 06:13:47 +0000 http://www.kalzumeus.com/?p=884#comment-2567 [...] CrazyEgg is really one of my favorite secrets of being a metrics junkie, simply because it makes problems look so freaking obvious, when they could be buried if you relied on quantified analysis.  For example, there is nothing I can drill into in Google Analytics, A/Bingo, or my homegrown stats tracking which would have told [...] Original post [...] [...] CrazyEgg is really one of my favorite secrets of being a metrics junkie, simply because it makes problems look so freaking obvious, when they could be buried if you relied on quantified analysis.  For example, there is nothing I can drill into in Google Analytics, A/Bingo, or my homegrown stats tracking which would have told [...] Original post [...]

]]> By: Paul http://www.kalzumeus.com/2010/03/21/crazyegg-to-the-rescue-again/#comment-2566 Paul Mon, 22 Mar 2010 08:41:13 +0000 http://www.kalzumeus.com/?p=884#comment-2566 >> If adding the links decreases registration in a statistically significant way [...] it gets yanked >> Interesting - the image implies that the links are desperately needed. I suppose I never considered that they could possibly _decrease_ registration rates; that seems so counter-intuitive. But I suppose that it is possible, and thus will probably happen sooner or later. In order to prevent it, would you say the best approach is simply "test everything"? >>
If adding the links decreases registration in a statistically significant way [...] it gets yanked
>>

Interesting – the image implies that the links are desperately needed. I suppose I never considered that they could possibly _decrease_ registration rates; that seems so counter-intuitive.

But I suppose that it is possible, and thus will probably happen sooner or later. In order to prevent it, would you say the best approach is simply “test everything”?

]]>