Tech Marketing Zone is brought to you in partnership with:

Founder of Collectorz.com and Manula.com, I am a programmer who has learned that software marketing can be effective *and* fun. I write about software marketing on my blog at AlwinHoogerdijk.com. Recurring topics are: Adwords, SEO, A/B split testing, website design, mobile apps and web-applications. Alwin is a DZone MVB and is not an employee of DZone and has posted 3 posts at DZone. You can read more from them at their website. View Full User Profile

A/B Split Test: Cartoons VS Photos

02.27.2013
| 1962 views |
  • submit to reddit


For the past 2 years, our Movie Collector product page has shown a picture of me standing in front of my own DVD collection. Similarly, the Book Collector home had a picture of my wife Sytske and her book collection.

But then we had nice cartoon characters created, mainly for use in advertising material. Being happy with the results, we thought: let’s use these characters on our product home pages too. It would improve the “after-ad-click” recognition of the visuals and of course, it just looked nice.

So we replaced the photos with the cartoon characters, without split-testing.

Recently, while staring at our website, I started wondering: was this really a good idea? The cartoons may look nice but do they really work better than the photos?

So I finally did the A/B split-test: Cartoons (A) vs Photos (B).

Version A – Cartoon characters:

Here’s what our Movie- and Book Collector pages look like with the “CLZ Babes”:
Pros: it looks nice and creates a strong connection between our banner visuals, the landing page and the software itself (where the CLZ babes are also used, e.g. in the splash screen).

Version B – Photos of me and Sytske:

These are the same product pages with the collection photos of my wife and me:
Pros: recognizable image for collector-types, builds trust because it shows the company owners are collectors too.

The results

Well, I was glad I finally came to my senses and did run the A/B split test after all.
Because these are the results after 7 days (about 1,500 sign-ups and 300 sales):

Compared to version A (cartoons), version B (photos) generated:

  • Sign Ups: 5.1% more
  • Sales: 35.3% more
  • Average Purchase: 15.6% higher
  • Total Profits: 53.9% higher

Is that scary or what? Needless to say, I am back to using photos on my landing pages.
But the more important lesson here is:

never make big changes to your website without split testing.

 

Published at DZone with permission of Alwin Hoogerdijk, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)