VeggieTales Case Study

Optimize
Attract
Build
C O N V E R S I O N O P T I M I Z AT I O N
VeggieTales
Case Study
RE SULTS
→→ 17.4% increase in revenue per category
page visitor
→→ 13.9% increase in revenue per product
page visitor
→→ 14.3% increase in revenue per checkout
page visitor
→→ 36.8% increase in revenue per homepage
visitor with social media changes
→→ 28.1% increase in revenue per mobile
visitor
SITUATION
→→ VeggieTales wanted to systematically
optimize the design of their entire site to
enable it to perform better for visitors.
HOW W E DID IT
→→ By first examining their analytics,
we were able to determine where to
start testing and what to test using
Optimizely’s optimization testing
platform.
38+62+Y 42+58+Y
100+T
38%
increase YOY in sitewide
revenue per visitor
42%
50,000+
Visitors tested for
statistical significance
Increase YOY in
sitewide conversion
rate
VEG G I ETAL ES
CASE ST UDY
VeggieTales
VeggieTales is a leading animated home video series among moms with
preschoolers in the United States. Since 1993, VeggieTales has sold over
70 million videos, 15 million books, and more than 7 million CDs. They
had their site designed and built by another agency, and they suspected
its design was causing their revenue and conversion rates to suffer. They
came to the Optimization Team at Blue Acorn for direction and results.
Commitment To Data
At Blue Acorn, we take a scientific
approach to our services that relies on
proven results to guide our decisions,
which is why we are the only agency in
the world to hold certified partnerships
with Magento, Optimizely, and Google
Analytics. Our eCommerce expertise
allows our team to form hypotheses, but
it is actual data that enables us to provide
our clients with the best results possible.
At Blue Acorn, we believe in redesigning with Intelligence. This means
learning our clients’ goals, studying their analytics, examining their
current designs, and forming Intelligent hypotheses before we begin
testing. Our initial research drove us to start with some subtle design
changes that, after reaching statistical significance through A/B testing,
now receive 100% of the traffic as permanent design changes.
Conversion Rate vs. RPV
Optimizely is the world’s leading optimization testing platform. It’s not just
that they have a better product. They also focus on a better metric. Like our
partners at Optimizely, we feel the truest metric for measuring the success
of optimization testing for an eCommerce site is Revenue Per Visitor (RPV).
Increasing our clients’ conversion rates is easy if they agree to start to sell
everything at 90% off, but that would kill their revenue, which - in terms of
purpose - is what sets an eCommerce site apart from a website, where
conversion rates typically refer to sign-up forms. Now, this is a dramatic
example, and constantly attempting to improving your conversion rate is
still a smart idea. However, eCommerce merchants should focus on RPV as
their primary Key Performance Indicator (KPI).
It’s also important to note the difference between visits and visitors.
Whereas Google Analytics - by default - determines a site’s eCommerce
conversion rate using the number of visits to transactions, Optimizely
measures RPV as the average revenue per visitor, not per visit, which we
believe to be a more accurate look at how your site is performing.
VeggieTales
store.veggietales.com
l
Officia
VEG G I ETAL ES
CAT EG O RY PAG E
VeggieTales
store.veggietales.com
l
Original Page
Official Store
VeggieTales’ category pages
had featured very large, bright,
distracting banners. Customers
landing on the page didn’t have
general guidance on where to go,
and they were getting distracted
from finding products they wanted
and continuing to the product page.
→→
Hypothesis: Dropping the
banners would result in a higher RPV.
Variation
VeggieTales
store.veggietales.com
l
Variation
Official Store
→→
Results: Without the banner, VeggieTales’ category pages
produced a 17.4% increase in
RPV.
→→
Statistical Confidence Level: 95.3%
+17.4%
revenue per visitor
VEG G I ETAL ES
P RO D UCT PAG E
Original Page
Pistachio: The Little Boy That Woodn’t VeggieTales DVD
store.veggietales.com/pistachio-the-little-boy-that-woodn-t-veggietales-dvd.html
Using custom heat mapping tools,
we confirmed that VeggieTales’
current product pages had very
poor spacing, distracting attributes,
and lacked organization. They
weren’t awful by any means, but
they weren’t optimal either.
→→
Hypothesis: Improving the spacing and increasing the prominence of the primary call-to-action will result in a higher RPV.
Variation
Pistachio: The Little Boy That Woodn’t VeggieTales DVD
store.veggietales.com/pistachio-the-little-boy-that-woodn-t-veggietales-dvd.html
→→
Results: The product page
variation produced a 13.9%
increase in RPV.
→→
Statistical Confidence Level: 96.1%
+13.9%
revenue per visitor
VEG G I ETAL ES
CHECKO UT PAG E
Original Page
Checkout
store.veggietales.com/checkout-onepage/
This is where eCommerce
conversions happen. However,
VeggieTales’ checkout page
included all of the same header and
footer elements that were found on
every other page of the site. They’re
good elements, but they serve as
unnecessary distractions to a visitor
trying to convert to a customer.
→→
Hypothesis: Simplifying the checkout by removing all of the distractions in the header
and footer - except those designed to act as confidence builders - will result in a higher RPV.
Variation
Checkout
store.veggietales.com/checkout-onepage/
→→
Results: The more streamlined checkout page variation produced a 14.3% increase in RPV.
→→
Statistical Confidence Level: 98.1%
+14.3%
revenue per visitor
VEG G I ETAL ES
HO MEPAG E
Original Page
Veggie Tales l Official Store
store.veggietales.com
The original homepage had a
strong callout for free shipping,
which is never a bad idea
for converting visitors, but it
overshadowed the site’s social
icons.
However, we didn’t just want
to move the social icons to the
header, we wanted visitors to be
aware of the immense size of the
VeggieTales community, which
serves as a confidence builder.
→→
Hypothesis: By adding social icons to the header and making visitors aware of the size of the VeggieTales community, visitors will be more likely to have social interactions within the community and consumer confidence will increase.
Variation
Veggie Tales l Official Store
store.veggietales.com
→→
Results: Not only has VeggieTales
gained more fans and followers, but the variation homepage produced an astounding 36.8% increase in RPV.
→→
Statistical Confidence Level: 95.7%
+36.8%
revenue per visitor
VEG G I ETAL ES
M O BI L E
Original Mobile UI
iPhone 5 and iPhone 5s Cases and Covers l Speck Products
speckproducts.com/iphone-case/iphone-5-cases.html
There’s actually nothing mobile
about this UI. It is simply the
desktop UI viewed on a mobile
device. It’s nice in that it doesn’re
rely on Flash, which is still not
supported on iOS, but it requires
a tremendous amount of effort to
navigate the site on a smartphone.
iPhone 5 and iPhone 5s Cases and Covers l Speck Products
speckproducts.com/iphone-case/iphone-5-cases.html
→→
Hypothesis: Sending moible traffic to a mobile-themed UI will increase the mobile conversion rate and revenue per visitor.
Variation
→→
iPhone 5 and iPhone 5s Cases and Covers l Speck Products
speckproducts.com/iphone-case/iphone-5-cases.html
iPhone 5 and iPhone 5s Cases and Covers l Speck Products
speckproducts.com/iphone-case/iphone-5-cases.html
Results: Thanks to testing, VeggieTales now knows they can be generating a lot more revenue from their mobile visitors, as this quick, non-customized, mobile-themed UI saw a 28.1% increase in revenue per visitor.
→→
Statistical Confidence Level: 99.1%
+37.5%
conversion rate
Understanding Statistical Significance
The “statistical confidence level” references in this case study allude to the statistical significance of each test, which is the
likelihood that the variation will beat the baseline. Given the average revenue per visitor data collected in this experiment,
what is the likelihood that the actual average revenue per visitor for the variation is higher than the actual average revenue
per visitor for the baseline? In other words, what is the likelihood that the observed average revenue per visitor increases are
not a fluke? Statistical significance is needed to prove this. Once a statistical significance of 95% or higher is reached, most
optimization professionals will agree that the results are not a fluke. More information on the science behind this can be
found in Optimizely’s Knowledge Base on their company website.
VeggieTales’ Newfound Success
It’s far from over. The improvements VeggieTales has experienced thus far, which have resulted in 38% sitewide increase
YOY in RPV, are just the beginning. Once the changes to the homepage variation are applied to the rest of the site, we
expect that number to grow even higher. By doing our due diligence before we began testing, Blue Acorn was able to
identify the lowest of the low-hanging fruit, in order to produce the quickest gains for our client. These initial tests were
simply the first on a roadmap of tests that we have planned. Using Optimizely dramatically cuts down on the time these
tests take to set up, which means the amount our clients are billed pales in comparison to the returns they see on such
investments. This is important because you should always be testing. Every aspect of your site that make sense to test
should be tested. Every change you make to your site should be tested. It’s what’s best for your customers, and it’s what’s
best for your business.
Blue Acorn
Optimizely
VeggieTales
Blue Acorn is a premium eCommerce
agency helping retailers and brands achieve
revenue growth through a comprehensive,
data-driven approach. It is the only agency in
the world to hold certified partnerships with
Magento, Optimizely, and Google Analytics.
They are committed to delivering Intelligent
eCommerce solutions rooted in data
and driven by their best-in- class design,
development, marketing, and optimization
capabilities.
Optimizely is the layer on top of the web
that enables businesses to deliver better
experiences and drive top line improvements
to their websites. Using Optimizely as a
split testing tool, marketers create different
versions of their web pages and can
measure how changes affect performance.
The results can be shocking and unlock
dramatic revenue gains. The company
closed a $28 million Series A financing round
led by Benchmark Capital in April 2013.
VeggieTales is a wholly owned subsidiary
of DreamWorks Animation’s division,
Big Idea Entertainment, a leading studio
and producer of children and family
programming, characters and brands. Big
Idea’s best-selling series, VeggieTales, is a
one of the most recognized brands in the
US among mothers of preschoolers and has
enjoyed four seasons as a top-rated series
on NBC Saturday mornings, Telemundo, and
ION TV.
blueacorn.com - 635 Rutledge Ave, Suite 101 Charleston, SC 29403 - Tel 877.944.2583 x227