The Dirty Little Secret Nobody Tells You About Website Split Testing...

Nearly all "online marketing gurus" promote the benefits of conducting split tests and multi-variable tests on your website.  But, few (if any) tell you the WHOLE truth about how to test, what to test and what results to realistically expect from your testing.

The consequences?  Many online markers become confused by and disappointed with their first fledgling attempts at conducting testing campaigns on their websites.

Because their results often don’t live up to the "hype", they become discourage and simply assume that "testing doesn’t work" or "it’s not worth the effort".

I hear this from my clients all the time.  They will tell me, "oh... I tried testing on my site and it didn’t help."

Well... in an effort to clear up a few of the misconceptions about testing and help folks overcome some of the unrealistic expectations that simply set them up for disappointment, I’m about to tell you the "dirty little secret" that the gurus usually fail to mention.

Every variable you test will NOT automatically improve sales.  In fact MOST of the variables you test will either reduce response or make no measurable difference!

In fact, after conducting over 6,000 split and multivariable tests over the past six years I’ve generated some pretty good data on what you can expect to achieve in a typical testing campaign.

In most multivariable tests only 1/4 to 1/3 of your variables will boost response.  Another 1/4 to 1/3 of your test variables will actually reduce response.  And 1/3 to 1/2 of your variables won’t make a measurable difference. 

(Well... that’s not entirely true.  If you run any test long enough a winner will usually emerge, but it is not worth the trouble to let a campaign run for months on end just for a 1/10th of a percent improvement in conversion.)

Of course, the more experience you gain in testing, the better your odds of finding a variable that will be a winner... but you still have no guarantees.

Often variables that you are sure will boost sales, wind up having no effect or worse, decrease response. 

I’ve seen tests where a 1 year guarantee was tested against NO guarantee and the no guarantee version won.  I’ve conducted price tests in which we tested $17 vs. $97 and $97 converted better!  (Not just higher visitor value, but actually higher conversion!)

So it really is impossible to know in advance what version or variable is going to win.  Experience and prior testing data can help you make an educated guess, but you still have to test it and see.

I view selecting test variables, kind of like picking stocks.  You can do all of your homework, use a variable that has boosted response on every other site you tested and still not have a winner. 

And the opposite can happen.  You can test a variable that you are almost positive will be a dog, (or even make an error when setting up your variables) and it can boost response.  I’ve often seen headlines that I hated or graphic images that were down right UGLY convert better than the more appealing versions.

So just like picking a stock, past performance doesn’t guarantee future results!  Every single variable you test will not be a winner.  In fact when it comes to the stock market, if you can pick a winning trade just 51% of the time, you’ll become rich.

The same principle applies to test variable selection.  If you’re able to consistently pick winning variables 50% or the time or more, you’re doing an outstanding job!  (Or you’re simply not running enough tests...)

Now, am I telling you all of this to discourage you from testing?  ABSOLUTELY NOT!

Like I said at the beginning of the article, I’m a testing fanatic.

I’ve seen small businesses transformed into "big" businesses almost overnight via proper testing.

I’ve seen a laid off, single father go from struggling to pay the bills selling an ebook to making a solid six figure income by only tweaking and testing 4 items on his sales letter.

Done right (and with proper expectations) testing can indeed change your life and transform your business.

I just want to make sure that you go into the process with the right, long term mindset and that you don’t become discouraged and quit if your first attempts at testing fail to yield earth shattering improvements in response.

The good news for testing is, because you can flush the variables that don’t boost response and only keep the "winners", even if you only pick a winner 10% of the time you’ll still make huge gains in conversion and response over the long haul.

Happy testing!