Part 2 - MarketingSherpa

Transcription

MarketingSherpa Selling Online Subscriptions Summit Transcript 2008
Take a Look Inside: Building Your Subscription Base Through
Segmentation and Usability
Session Description
Building a loyal, relationship-focused experience for
both new and returning visitors starts with increasing
the usability of the site. In the process of testing
features like ‘Look inside the Magazine,’ guest checkout,
and a variety of site search results pages, Magazines.
com has developed experiences tailored to many
audiences in order to maximize the functionality of
their site and appeal to and deliver relevant content
to all visitors. Hear lessons learned around acquiring
online subscriptions, establishing long term consumer
education goals, and site optimization based on
customer behavior.
About the Presenter
Kate O’Neill
Director of Customer Experience &
Product Development,
Magazines.com
Kate O’Neill leads the Customer Service Experience group
at Magazines.com in which she has begun to establish
a culture of experimentation to build the business by
serving the customer. She formerly managed content
at Netflix during the company’s early years, developing
Presentation
MODERATOR: All right. Thank you very much. All right, so
we’ve got one session left before lunch. In this session
we’ll be exploring - well, let me take a step back. I
think all of us have gone and looked at a hypothesis
and said to ourselves, “This is just implicitly true. This
is self-evident, we don’t even need to test it.” In our
next session, Kate O’Neill from Magazines.com is going
to explain why that is never – that you never are sure.
With that, Kate, why don’t you come on up. Thanks.
Kate O’Neill: All right. A little bit about Magazines.
com. I realized after I looked at this slide today, to
say largest online subscription agent is a little vague
in this room. Online magazine subscription agent is
really much more accurate. We are an aggregator. We
carry about 1,800 magazines, both B-to-B and B-to-C
titles. We carry about a million subscriptions annually.
its content strategy to support the subscriber base
and inform the customer experience. More recently,
O’Neill held a core contributor role throughout a legacy
intranet replacement project with Hospital Corporation
of America in which she principally guided search and
content strategy for 200,000 employees across more
than 300 hospitals and business units. In marked
contrast to her modern professional life, O’Neill resides
in an antebellum brick cottage near downtown Nashville,
Tenn. with her husband, their six cats and their neverending renovation plans.
About magazines.com
Magazines.com is the oldest, the largest and the
most dependable magazine subscription service on
the Internet. We are a privately-held company based
in Franklin, Tenn., a suburb of Nashville. Our major
investors include Anderson Media (one of the country’s
largest wholesale-distributors of magazines, music,
books and video) and Time Inc. (publisher of leading
titles such as People, Sports Illustrated and Time).
In 2003, Mazagines.com acquired the leading online
subscriptions service for free business magazines.
Magazines.com was recently ranked by comScore as
one of the Top 100 U.S. Internet Properties and has
appeared on the TODAY show as one of the best gift
ideas for Christmas, birthdays and other occasions.
That’s kind of an old number. It’s probably higher than
that. I haven’t got the actual numbers up to date. We’re
actively trying to switch our focus from the subscription
orientation to a subscriber orientation, so that was a
really great session for me to hear, the last one.
About myself, my title here, as you can see, is probably
the longest title ever. Director of Customer Experience
and Product Development. What that basically means is
I have too much to do. I kind of own the site, you could
say, so from a new feature development perspective,
analytics, optimization, all of that sort of stuff, it all
falls into my small group, my A-team.
My background educationally is in linguistics, so I come
at this from a little bit of a different perspective than
folks who have a purely marketing and purely technology
perspective. To me it’s a lot about the semantic and
syntactic structures of how you communicate online. So
if I go off on a tangent with that, that’s probably why.
© Copyright 2008 MarketingSherpa LLC. It is forbidden to copy this transcript in any manner
| 83
Take a Look Inside: Building Your Subscription Base Through Segmentation and Usability
When I first joined Magazines.com in January of last
year, there were a lot of features and functionality that
people talked about wanting to add to the site that
customers had been asking for in the call center for a
long time, that email requests had come in, and some
of those were that people wanted to be able to check
out without registering. They wanted to be able to
take a peek at the content of magazines, similar to the
Amazon ‘Look Inside The Book’ feature, or Barnes and
Noble just launched the Zinio feature, if you guys have
seen that.
More interactive site experience. They wanted to have
more of a Web 2.0 feel and more personalized content.
So little by little over the last year and a half we have
tested an awful lot of these things. I actually have case
study data on all of these, and I tried to work it into
this presentation, and it’s just too much. Instead, I’m
just going to focus on this look-inside piece, which as
mentioned, is kind of the great moral case study in why
you should always test.
Of course, the best way to find out what users want is
going one on one, talking with users and asking input,
right? You’ll hear directly from them what they want.
The problem – interview a series of people and watch
what they do, look where they go, see where they click.
Kind of get a real intimate sense of how the user actually
uses the site, then ask them, “Why are you doing what
you’re doing? Why did you click where you clicked? Give
me some insight on that.” Well, the problem with that
is users don’t know why they do what they do, or click
where they click, and this sort of thing. They just do
it. When you ask them why they do what they do, they
make stuff up.
This is where we found ourselves, with an awful lot of
our usability studies, is asking users in usability studies,
“Why are you clicking on that?” And them saying, “I
don’t know, it’s just – it was red.” OK, now we have a
lot of red on our site. It’s too much red now. It can be
really, really difficult to know the difference between
actual good user feedback that’s useable data, and
made-up stories.
The problem with made-up data, of course, is that
people will often claim a bias that they don’t really
have, or they’ll say that they need something that they
don’t really need. I know this is a skill that you acquire
through doing user research, knowing the different
sort of – the wheat and the chaff sort of separation
– knowing when people are telling you something that’s
fundamental versus something that’s fluff, right? So
84 |
the best way to get the unbiased data is probably to
do onsite testing and more specifically, onsite multivariable testing. Then you can look to your analytics
platform for opportunities to test, and to find those
hidden gems of opportunity that you can exploit.
My last point here, formulate hypotheses and then test.
Don’t just test random stuff. It’s very tempting when
you first get – we signed with Offermatica a year ago,
and it was tempting to just throw things out on the site
and test them. Red buttons, let’s try blue buttons, see
if those work. That’s great, but without a hypothesis
behind the test it doesn’t lead to a whole lot of insight.
It just leads to noise, and sometimes lift and sometimes
not, then you have to kind of go back and figure out
what you were testing.
For us, with the look-inside test, after hearing repeatedly
that people wanted to be able to have this preview
feature, we started formulating what the hypothesis
really was. For us, it was we think that visitors would
want to utilize a tool that would allow them to preview
a sample issue of the magazine. Our success metric
there would be the click rate. If that was true, people
would be clicking on that feature.
We hypothesized, also, that the presence of a preview
tool may boost visitor confidence in the site. They may
think we’re a more rich, robust site, that we have more
technology experience, that we have – that sort of
thing – more to offer in expertise. In that sense maybe
the success factor for us would be just the product
conversion rate for those test participants, and that the
people who were actually already visiting the product
where the feature was appearing would convert at a
higher rate.
Another hypothesis may be that the presence of that
tool may simply enhance the user experience and it may
make the experience more rich. We thought maybe a
success factor there would be that visitor conversion
rates overall on the site would increase, not necessarily
associated with any given product.
The problem with onsite testing is that people behave
differently, and that random groups of people behaving
differently in different contexts can cancel each other
out and make themselves look invisible. What we were
trying to prepare for was what we’d seen in other tests
occur, that if you have a group of people that are very
happily accepting one variation, and then another
group of people that are very happily accepting another
variation, but overall it’s about the same acceptance
rate in those two groups, and they’re about equal
groups, it looks like nothing’s happening unless you’re
segmenting properly, and so on. We started also trying
to think about segments that made a lot of sense for
which kinds of users would be most interested in a
preview functionality. What kind of users would most
respond to this sort of functionality?
Here’s the setup of our look-inside test. You can see,
sort of in the background here is the standard product
detail page, in this case for Entertainment Weekly, and
what we have shown in the blue here is two variations
of the look-inside bug, the button that would take you
to the preview feature.
We deliberated over this for a long time. You’ve
probably seen this feature on a number of websites in
various forms. A lot of the time when you see this kind
of feature, like on Amazon, for example, you’ll see an
overlay over top of a cover image, or a book image. We
thought about that for a long time, and we thought what
we were testing, in most cases, were weekly magazines,
and it was going to be very difficult to ensure that the
cover image that was on the site, and the issue that
was actually shown in the preview tool were one and
the same. If it were not the same, it would produce a
negative user experience, we thought.
So we thought we wanted to keep it away from the
cover image during the test period, just so that we
would have less difficulty, less interference of that
element. We kept it away. We talked about verbiage of
how to say not explore this issue, or preview this issue,
or preview this magazine, but explore an issue. It was
more sort of a generic sense that you’re going to get to
see inside a magazine. It isn’t necessarily this issue of
this magazine, but you’ll get to see it. Whether or not
that’s actually what users were wanting, who can say?
We tested it, we talked anecdotally with friends and
family to kind of get a feel for that. This was the way
we decided to set it up.
Some other considerations with that were that this
was a feature that was hosted by Time, Inc. They’re
an investor in Magazines.com. I don’t know if there’s
any Time, Inc. people here today. They were prototyping
this feature on their own, and we were able to include
it in our testing. It was for 14 Time, Inc. titles, and
these were all fairly well known titles. That was one
consideration. It also meant that the feature itself was
making an external call off of the Magazines.com site
over to the Time, Inc. servers. That was one factor in
the whole setup.
Also, the test ran during the holiday season. It was
about November, early-December, timeframe. For us,
that’s an extremely busy season. It’s an extremely high
converting season. Our conversion rate about triples or
quadruples during that time, so it’s a little bit skewed.
We know that for sure. It’s also a heavily gifter-oriented
time period, so we don’t know whether that has some
noise factor in the results, but there it is. The preview
tool, as I said, appeared with 14 Time, Inc. titles, and
it appeared on product detail pages. So implicitly, the
visitor has already expressed an interest in that product
before they’ve even encountered the feature. There’s
an element that’s kind of a confusing or a conflicting
element as well, and we’ve definitely tried to think
about that in retrospect.
There for your consideration are some of the elements
that went with the tests. The results were, to us, a
little surprising. After such confidence that this was
going to be a conversion boosting tool, it actually
suppressed conversion in almost every case. One of the
main features we think that was, or one of the main
attributes, that was negative was that it was very slow
to load.
I don’t know if any of you had an opportunity to
experience the feature on our site, or have experienced
it on other sites, where you have a pop-up window, and
then it just sits there, and it turns in cycles, and you’re
kind of going, “Hmm,” especially during the holiday
season when you’re trying to buy gifts for somebody. It’s
probably not a very good experience. So we recognize
that may have been a complicating factor.
In any case, visitors who did click on the feature were
more likely to click on the preview verbiage than the
explorer verbiage, for whatever that’s worth. Again,
variations did result in a drop in conversions, but
that drop in conversion was only slightly greater with
preview than it was with Explorer, so they clicked more
on preview than they did with Explorer, and the drop in
conversion was only slightly greater. Overall, the more
established titles showed fewer clicks than the less
established titles did. That was kind of interesting to
us, too. Much less interest from people in clicking on
those. Presumably people are already familiar with those
titles; they don’t necessarily need to see a preview of
it.
The lesser known titles – somewhat lesser known titles,
they’re still fairly well known, Sports Illustrated Kids,
Health, Style Watch, Money – these had a higher click
%age. Then in both cases, kind of reversed. The greater
the click %age was, the higher the drop in conversion.
| 85
It’s a little bit confusing, counter-intuitive. I hope that
makes sense. It’s a lot of text on this slide.
That’s the gist of it. I’ll leave some time for questions,
I think.
The point of this for me is to take a step back and to
say, “Wow, that’s a very counter- intuitive result, so
what am I supposed to do with that information?” For
me, the answer is to take a step back, look at the results
and try to find what you’re actually being told by those
results. I like to say find the story within the story. To
figure out – in this case we had subjective information
that told us that people wanted this feature – maybe
they really didn’t – and objective information that told
us that however the feature was presented, it didn’t
work. Trying to figure out exactly why is the science, or
the art and science of it all. Then for me to look for the
unobvious as well as the obvious answer.
MODERATOR: Oh, absolutely. OK, thank you very much.
Perhaps it really made more sense to have this feature
appear on lesser known titles, or long-tail titles. With
them appearing on very popular Time, Inc. titles, maybe
it’s not as useful of a tool. So when we have another
opportunity to run this test again, we’ll definitely be
trying to run it with some longer-tail titles.
Again, the test ran during the holidays, so perhaps
there’s some sort of conflict of having this feature in
place where gifters are involved, and where the motive
is to find something that makes sense for a gift for your
daughter, or son, or husband, or wife, or whatever, and
maybe you’re not as familiar with Golf Digest, or Golf,
I think, was one of the titles, as your husband would
be, or whatever. You can imagine the scenario. Running
the test at another time of year would certainly be a
variation that I would be interested in running.
The next conclusion is that it feels to me like the viewer
holds the title to a higher standard of quality than not
having the viewer does. And certainly holds the viewer
content to a higher standard of quality, if that makes
sense. What we were thinking is that the option to have
the viewer only makes sense if you’re able to show a more
unilateral approach across the site to that tool, and to
be able to say, “Here’s the inside of that magazine. If
you don’t like it, here are some other options that you
can explore.” So more a discovery model.
I think it’s a really good piece of a discovery model,
but we don’t have a very good discovery model that
supports it. One interesting conclusion for me is that
it maybe it’s a part of a suite of discovery tools, and
because the rest of those tools were lacking, it couldn’t
succeed. That may have implications for you all in your
work too. Again, essentially having that viewer poses a
risk unless we’re able to back it up.
86 |
Participant: One question that I have is the site
capable of dynamically delivering content based on
whether – let’s say you knew somebody was a gifter
based on their behavior?
Kate O’Neill: Yes. Yes, it is. In fact, one of the case
studies that I was going to include in this presentation
today was about some dynamic testing that we ran with
our home page where we found that visitors to the
women’s category expressed more affinity for a given
layout of the home page. We could certainly try gifting
and non-gifting variations – or any number of other
segments that we would try. Yeah, certainly.
Participant: Hello. I was wondering how long of a
delay there was in bringing up the content when you
clicked through? Do you believe that was a significant
impact?
Kate O’Neill: Yeah, I think it was a significant impact.
Thank you for the question. I think it was significant. It
was a long delay, in my experience, depending on your
internet connection, of course, depending on whether
you had things cached. All these different factors came
into play, but when we were – one of my staff and I were
visiting our Summit, New Jersey office, and they had an
internet outage earlier in the morning, and so things
were a little shaky on their Internet connection that
day, and one of them tried to pull up the viewer, and it
felt like it just took five minutes. Clearly that’s not an
experience that a customer is going to stick around for.
We felt like there was good feedback there, and kind of
a good learning. Definitely I feel like that contributed.
MODERATOR: OK, we’ve got another question over here.
Participant: Along those same lines, how can you tell
when it’s an issue with your product versus an issue
with your presentation? So is it that they didn’t like the
viewer at all, or was it that they didn’t like that viewer,
or the way that it was implemented? How can you tell
the difference?
Kate O’Neill: Yeah, and that’s a really good question.
I feel like the answer comes back into balancing the
subjective and the objective testing. I think going back
with some paper prototypes and sitting with actual
users, and showing different mockups of a viewer tool,
and talking through if this feature is placed on top of
this image will that make more sense, and so on. We
have done that. We have talked to some users. We just
haven’t gotten a solid set of responses that we feel
really nails it for us.
Then balancing that against let’s try maybe having –
laying it over top of the image and seeing if that’s a
better experience. So I feel like it just means iterations
upon iterations of testing, in this case, where there’s
such confusing results.
Obviously, not every test is going to have such
conflicting and confusing results, and you’ll hope that
you can just go yeah, scored one. Just push that aside
and make that the benchmark for future studies. Yeah,
it’s complicated I think.
Participant: Yeah, Kate I’m interested to know – you
mentioned that there was a technical problem with the
viewer, and it was hosted on a different server. Will you
be retesting once that technical fault is corrected, and
maybe you’d have a cleaner test? I’m also interested to
know if were you able to track how many people opened
the viewer, and then exited because it was taking too
long, and never went past that point? How many people
sat through the four-minute load and got past that point
and saw how those two different audiences differed in
their ultimate subscriptions?
Kate O’Neill: That’s a great question. The second part
of that question – I always like to start with the end.
The second part of that question, honestly we had no
ability to do that since we weren’t hosting the tool, and
it was a very complex kind of arrangement, so we lost
that visibility.
But, if you were all trying something where you had
any kind of complex feature that you’re testing, I do
encourage you to use that metric, as you so cleverly
pointed out, knowing the difference between someone
who actually sat there and waited, and was that
persistent. How do they convert versus someone who’s
frustrated and obviously shows very little patience for
your site, your brand, and then wants to move on?
Then the first part of that, yes, we absolutely do intend
to retest with a more – I don’t mean to say more
stable environment – but maybe a more predictable
environment for us.
Participant: Have you been able to track conversion by
segment, looking at ISPs and what type of connection
those people had? There might have been some lift
on broadband users, and just more results on dial-up,
and then it goes to your overall kind of bad result, but
maybe it’s a better experience just for people with very
fast connections.
Kate O’Neill: That’s a really good question, and we did
actually look at that. Surprisingly, there wasn’t a whole
lot of difference between the dial-up visitors versus
the broadband visitors. I think it – a combination
of that visibility and with the visibility the previous
gentleman was asking about – would probably have
been very informative to us, but as it was, just the
speed of the connection didn’t seem to make any
discernible difference in whether people converted or
didn’t convert, or clicked or didn’t click.
MODERATOR: We had a another question up there –
Participant: Hi Kate. In one of the first slides, you
mentioned that you should know what your customers
want. How long does it take to incorporate all of those
wants in your product, especially when you’re competing
with start-up companies that are just making changes
within like half an hour, twenty minutes, or so? With a
larger corporation, especially, it takes longer. What did
you guys do?
Kate O’Neill: I lost some of the question, but I think
the gist of it was how do we make those changes that
people were asking for, or how quickly can we make
them?
Participant: Yeah, how quickly.
Kate O’Neill: I think it’s been – we’ve been able to
do sort of a first iteration of most of those features,
most of the feature requests that I was aware of when
I joined the company a year and a quarter ago. We’ve
tried a little of everything.
Some of the things have – some of the tests, for
example, we completely rewrote our checkout process
and we did allow users to complete a checkout without
the perception of having registered for checkout. They,
in fact, do create an account as they go through the
process, they just aren’t really aware that they’re creating
an account because there’s no barrier before they do so.
That has produced wonderful lift and conversion for us.
That’s funding a lot of the testing that we do.
I think that, for me, that feels like the answer to the
question is that we’ve looked for the winners, and
the winners have really given us the momentum to
keep going through these other tests and trying to
slowly make our way into greatness – test our way to
greatness.
| 87
I think our site, if you look at it today, it’s not a very
pretty site. We’re certainly looking at a redesign, and a
grand relaunch and this sort of thing, but what we’ve
been mostly concerned with has been trying to make
sure that the experience, from end to end, from prepurchase through post-purchase, and through renewal,
makes sense for the user, and the type of thing where
we look for the opportunity and clean it up and exploit
it, make sure that users are happy with that process. It
gives us the momentum to keep going with these other
tests.
Participant: Can you actually track what they’ve
purchased and go back to respond to them with a more
personalized email in the future that would say, “Based
on what you’ve purchased in the past …”
Kate O’Neill: Yes.
MODERATOR: OK, I think that’s it for questions. Kate,
thank you very much. It’s been very helpful.
Participant: Hi, how are you, Kate?
Kate O’Neill: Hi.
Participant: Question for you. In the beginning of
your presentation you mentioned the ability to check
out – people wanted the ability to check out without
registering. How have you guys gathered customer data
to provide a more personalized content experience, and
have you guys, in some of your segmentation testing,
have you addressed that question?
Kate O’Neill: That’s a great question. As I mentioned,
people actually are creating an account as they go
through checkout, they just aren’t really aware of
creating the account. They are cookied as they go
through our site and through the checkout. They do
have a user account that they can come back in. When
a user comes back to the site, as we prompt them to
do, they are able to log in by their email address and
having a password mailed to them, just like the forgotyour-password process.
So they have an account, they go into their account,
and now we’re able to provide them with a logged-in
user experience even though they didn’t feel like they
had to create an account. That was the one piece that
really stood out to us from some of the early usability
testing that we did, is people kept coming across the
barrier of providing their email address to create an
account and going like, “Oh, wait a minute. Am I going
to get a bunch of spam now?”
It was just interesting to us. We kind of took a step
back from that and said, “All right. Well, if that’s such a
big barrier for people, let’s make them feel like they’re
just proceeding, and the email address is just one
piece of information that they’re providing that’s just a
natural piece of data gathering in order to get through
the process.”
88 |
Take a Look Inside:
Building Your Subscription Base
through Segmentation & Usability
Kate O’Neill, Director, Customer Experience &
Product Development
Magazines.com
Monday, May 12, 2008
More data on this topic
available from::
About Us

About Magazines.com:



About Kate O’Neill

available from::
Largest online subscription agent
Over 1 million subscriptions annually


Director, Customer Experience & Product
Development
Technologist-cum-marketer
Joined Magazines.com in January 2007
2
| 89
Getting Real-World Results

Some of the many things we “knew”
customers wanted:




Ability to check out without registering
Ability to look inside magazines
More interactive site experience
More personalized content
available from::
3
Best way to get real user input?




available from::
Go one-on-one: do usability testing
Interview a series of different people
Watch
what they do
where they click
what they notice
Ask them why they
do what they do
click where they click
notice what they notice
4
90 |
The problem with 1:1 testing?



available from::
People don’t know why they
do what they do
click where they click
notice what they notice
And when you ask them,
they make stuff up
It can be very hard to tell the difference
between usable data and made-up
explanations
5
The problem with made-up “data”

People will often claim a bias that they
don’t really have
available from::
6
| 91
Best way to get unbiased data?




Onsite multi-variable testing (MVT)
Look to analytics for opportunities
Analyze high-traffic areas as well as
hidden gems
Formulate hypotheses and then test

(Don’t just test random stuff)
available from::
7
Look Inside: Hypotheses

Visitors will want to utilize a tool that allows
them to preview a sample issue of a
magazine


The presence of a preview tool may boost
visitor confidence in the title


available from::
Success factor: Click rate
Success factor: Product conversion rate for test
participants
The presence of a preview tool may enhance
user experience and raise conversion rates

Success factor: Visit conversion rate for test
participants
8
92 |
The problem with onsite testing?

People behave differently

Different groups of people behaving
differently can hide each other by
cancelling each others’ actions out

By itself, onsite testing only shows you
average data
available from::
9
Look Inside: Setup
available from::
10
| 93
Look Inside: Other Considerations



available from::

Feature was hosted by Time Inc. on their
servers, so it meant an external call from our
product page
Test ran during the holiday season, when
conversion rate is highest and purchasers are
overwhelmingly gifters
Preview tool appeared with 14 Time Inc. titles
only
Appeared on product detail pages, so visitors
implicitly had already expressed an interest in
the title by visit to the page
11
Look Inside: Results





available from::

Received several comments from staff and customers
that the tool was slow to load; presumably due to
external hosting
Visitors more likely to click on Preview offer (4.89%
average) vs. Explore (3.85%)
Both variations resulted in a drop in conversion
Drop in conversion only slightly greater with Preview
(-.87%) than with Explore (-.76%)
Overall, more established titles (Time, Sports
Illustrated, People) had lowest Click % as well as
below average drop in conversion of all viewerenabled titles
Somewhat lesser-known titles (SI Kids, Health,
StyleWatch, Money) had highest Click % and higher
than average drop in conversion of all viewer-enabled
titles
12
94 |
What’s an online marketer to do?

Balance subjective & objective testing


Find the story behind the story



(But know that you may get it wrong)
Look for a narrative in onsite testing

available from::
(And know that you may get it wrong)
(But know that you may get it wrong)
Look for the unobvious
AND
the obvious

(And know that you may get it wrong)
13
Look Inside: Conclusions



available from::

Perhaps people visiting the product pages of well-known
titles are familiar enough with the title that they’re less moved
by the option to preview
 More relevant option with longer-tail titles?
Since the test was conducted during the holidays, perhaps
titles with highest Click % were gifters who wanted to verify
that the content matched the person they were gifting the
title to
While this holds the title to a higher standard to sample
quality content in the viewer (and may present a drop in
conversion when the content isn’t impressive), it also allows
us the opportunity to aid in discovery in the future – when we
can use the product a visitor is considering as the reference
point for additional search criteria (i.e. – I want more
pictures, or content for a younger child), then we have the
opportunity to direct them to a title that is better suited for
them (or their gift recipient) and increase overall lifetime
value.
Essentially – having the viewer poses a risk unless we are
able to facilitate a discovery path to present alternatives to a
visitor that is not satisfied with a particular title’s content.
14
| 95
Credits/Thank You
Kate O’Neill, Magazines.com
615-778-2129
[email protected]
available from::
A special thank you to:
Heather Weaver, Magazines.com
Brian Hawkins, Omniture
Stephanie Yang, Omniture
15
96 |
Keep Them Wanting More: Maximizing Customer Lifetime Value Through
Strategic Offer Development and Delivery
Session Description
The presenters discuss the techniques and tactics that
they have successfully implemented to extend LTV
(lifetime value) of their paid members. With effective
site optimization to creative CRM email marketing
strategies, TheLadders.com dramatically increased
their revenues and LTV of their customer base. These
implementations, along with their successful execution,
allowed TheLadders.com to experience package mix
shifts to the longer-term subscriptions at initial sign-up
along with increased subscription conversion to longerterm subscriptions post sign-up. Learn new approaches
on how messaging, design and timing create longer
subscriptions for your customer base and increase the
bottom line in revenue.
About the PresenterS
worked at ADP, Volvo Finance North America and TIG
Insurance (formerly Transamerica). McCurdy earned his
MBA from the University of Dallas and completed his
undergraduate work at Dallas Baptist University where
he received a Bachelor of Business Administration.
Leslie Semegran
Director Online Marketing,
The Ladders.com
Leslie Semegran is the Director of Online Marketing at
TheLadders.com. Since joining the company in 2004,
she has worked in acquisition, conversion and product
marketing helping to grow the site to over 1.7million
users. Prior to TheLadders.com, Semegram worked at
TheStreet.com as a Marketing Manager. She attended
the University of Michigan where she graduated with
a BBA.
Michael McCurdy
Director CRM, The Ladders.com
Michael McCurdy currently serves as the CRM Director
at TheLadders.com located in New York City where he’s
responsible for email marketing communications. Prior
to working for TheLadders.com, McCurdy worked in
Dallas, Texas for Match.com as Director of CRM Product
Marketing where he was responsible for all customer
touch point communications for both Match.com and
their newest brand Chemistry.com. Prior to Match.com,
McCurdy worked for Sabre Holdings (parent company for
Travelocity) holding a variety of management positions
in strategic marketing, product marketing, product
management and portfolio planning. Prior to Sabre he
PRESENTATION
About TheLadders.com
TheLadders.com is the world’s largest online resource
catering exclusively to the $100k+ talent market.
Located in New York and London, TheLadders.com
makes the global search for senior talent quick and
effective. By allowing recruiters to directly connect
with executive professionals who search qualified
leads by function and industry, TheLadders.com
has grown into the largest specialty employment
website. Founded in 2003, we now have over 1.8
million members, 35,000 recruiters and $10 billion in
available job listings. Through TheLadders.com and
our UK site, TheLadders.co.uk, we give our members
the tools, contacts and opportunities they need to
take their next career step.
to expand that customer lifetime value. What we’re
going to be learning about today is how they’ve done
that…Leslie and Michael, come on up.
MODERATOR: Let me introduce to you Michael McCurdy
and Leslie Semegran. They are here from The Ladders.
com. You’re probably familiar with TheLadders. It’s sort
of a premier site that connects executives with executive
recruiters. I think the cutoff is, what, $100,000 is sort
of the line of demarcation there, and they’ve grown
quite astonishingly in a very short period of time.
Leslie Semegran: Hi, everyone. I’m the Director of
Online Marketing and Michael’s the Director of CRM. So,
we work together doing all sorts of marketing initiatives
on the site and through email and we’re here today to
share that with you.
One of the ways in which they’ve done that is to identify
the key factors in customer lifetime value and to work
TheLadders started about four-and-a-half years ago and
we’ve really grown the business on direct marketing. But
| 97
Keep Them Wanting More: Maximizing Customer Lifetime Value Through Strategic Offer Development and Delivery
starting this January, we had our first brand campaign
offline and we launched our first national TV commercial
so we thought that we’d show it to you.
Television Commercial Runs
Great. So TheLadders.com, as you can tell, is a job
site for $100k+ professionals looking for $100k+ jobs
and for those companies and recruiters looking to hire
for $100k+ positions. We really help the $100k+ job
seekers stand apart from the crowd. We were started
by former HotJobs executives who realized there really
wasn’t anywhere online where $100k+ or higher paid
people could go to stand apart from everyone else that
was applying to every position on HotJobs and Monster.
So they created the site to help put $100k+ people in
an elite crowd where they’re still competing for job
positions, but they’re competing against other people
that are actually qualified for those positions.
We put the job seeker more in control of their job
search, so we have a basic and a premium product. Our
basic product allows the users to get a taste of what
the site is like. They can see some of the listings, they
can kind of look around and see what they would get
if they pay, and then when they pay they have access
to all 80,000 job listings and they have access to a
database of recruiters, which is actually the first of its
kind. There’s nowhere else that you can go and look for
recruiters that can help you with your job search. They
can apply to these positions, read content, things of
that nature.
So we have the subscription fee for the job seeker, which
is also one of the first of its kind, where we actually
charge the job seeker to get access to this content and
it really serves as a velvet rope to put the right people
in the right place for these jobs. We have a few different
segments; there’s MarketingLadder, SalesLadder,
OpsLadder. So the job seekers put themselves in the
ladder that’s right for them so they can search for
the type of jobs that are appropriate for them. And
we actually offer three different types of subscription
lengths: We have our $30 subscription, which is for one
month, we have $120, which is for six months, and we
have $180 which is for an annual subscription. I’m sure
a lot of you have the same type of setups with your
subscriptions. There’s different lengths and packages
that people can choose from.
We’ve always had these three types of subscription
lengths and when we started looking at the lifetime
value of the customer, we said: “Is there a difference
98 |
between people who choose the monthly option versus
chooses the six and twelve-month option?” And we got
our answer: 29 % increase in lifetime value for the sixmonth subs versus the monthly and a 39 % increase in
lifetime value for annual subs versus the monthly. So it
was really advantageous for us to get our job seekers
on the longer term subscriptions, but actually more of
them were choosing the monthly as their first choice
when they came to our site.
So Michael and I took a look at this. It started about a
year ago. We do a lot of testing at TheLadders, so we’re
going to take you through a bunch of the stuff that we
did on our site and through email.
First, we looked at what job seekers were choosing when
they first joined TheLadders and, like I said, they were
mostly choosing the monthly. They were trying it out.
It was the least expensive option. They weren’t used to
paying for a job service, so it seemed likely that most of
them would join the monthly. And we wanted to look at
how can we change that. How can we put people on the
six and twelve-month subscription without affecting
the overall conversion rate?
So while it might have been great to get 100% of the
people on the annual subscription, we didn’t want to
lose our total number of people coming in. So, one of our
goals was to keep our conversion rate at either neutral
or better than we were seeing before and just look at
moving the package mix. And we also said to ourselves:
“Well, we have all these people on a monthly. Is there
any way that we can get them to move from that to the
six and twelve month and extend their lifetime value?”
This is a little fuzzy, but I’ll take you through what this
page is. So like I said, we have a basic subscription and
a premium. When you’re trying to access jobs or apply
to certain jobs, you come to an upgrade page. I’m sure
most of you have a page like this, which is basically
where people put in their credit card.
Right from the beginning, we tell them that we want
them to upgrade to the premium subscription. We do
show a benefits chart here. I know that we discussed
that earlier today. So we put it off to the side. If people
are interested in more information, they can look over
to the left. It shows the difference between basic and
premium.
We have our different subscription options here and
this says: Select your membership. It’s $30 for monthly,
$120 for six months, which saves you 33%, and $120
for a twelve month, which saves you 50%. Then we ask
for the billing information and then we have our red
upgrade button, which our red upgrade button will take
on the grey button from earlier today any day of the
week. But I do suggest everyone try different types of
buttons in copy. You never know which one is going to
work for you. But for us, it’s always red.
So we looked at this page, and we said we have a
lot of traffic coming to this page, so we could test
relatively quickly how different things might work. And
we looked specifically at the area where we offered the
memberships and we said, “Is there anything we can do
when people come to this page to get them to choose
the six month versus the $30 – versus the one month?”
So we tried a bunch of different things. First, we tried
defaulting to the bottom twelve-month option. We had
originally defaulted to the $30 option. It didn’t work. It
just didn’t do anything. It didn’t move the results.
Then we tried different copy. We spelled out, “This is a
twelve-month subscription, this is a six month, this is a
one month.” We moved the twelve to the top, we kept
the default button here. It also didn’t work.
So then we tried something new. We thought: “What’s
the real difference between the different subscriptions?”
And it really is the discount that you’re getting by
paying for the longer period of time. So, we put all of
our subscription options in terms of monthly payments,
so we really explained the discount that you were
getting. We showed that for the annual, you are only
paying $15 a month; for the six month, you are only
paying $20 a month, and then for the regular monthly,
it was $30 a month.
We tried this out. We saw a neutral upgrade rate, which
is what we wanted. Remember, we didn’t want to lose
people in the funnel. We wanted to at least have the
same rate if not better. We had a 15% decrease in the
percentage of people who were taking the monthly,
which was good because we wanted less people on the
monthly. And then we had an increase in the amount
of people that were taking annual subscriptions, which
just increased the lifetime value and the amount of
people on the longer lifetime value subscriptions
tremendously.
We saw a neutral for the six-month subscriptions, so
most people were actually just flip-flopping between
the monthly and the annual subscription. They were
kind of skipping over the middle. But we had kind of a
problem. Can anybody guess what it was?
Participant: They were canceling throughout the
year.
Leslie Semegran: Close. Any other guesses?
Participant: They thought you were overcharging
them.
Leslie Semegran: That’s right. So we did too good of
a job. People were really confused so they thought that
they were signing up for a monthly subscription for
$15 and they actually didn’t want to be on the annual
subscription, they only wanted to be on the monthly
subscription. So we had a lot of people writing in saying
things like, “I did not want to sign up for a year, only a
month-to-month for $30. Please adjust my billing info
and credit my account”. “I signed up for one month of
membership and the congratulations page said that I
had signed up for one year. I do not yet – so we’ll upsell
you later – want to sign up for one year. Please resolve
this immediately. Do not charge my card for one year.
Only charge $30. I will dispute any bill greater than
$30. Thank you very much”.
So we did have a lot of people writing into customer
service that they were confused. They didn’t actually
cancel the service, but they would go and ask for a
prorated refund. So they would ask for $150 back, stay
on the monthly subscription for $30, and continue
their life with TheLadders, but they weren’t staying on
the annual subscription as much as we would like. So
our refund rate as a whole went up a bit and we just
– we focused a lot on customer service and customer
satisfaction and so we weren’t totally happy with these
results even though we set out to do what we initially
wanted.
So we tried it again and we kept the same options where
we offered $15 a month, $20 a month and $30 a month,
but we added more of a description. So this year it says,
“You’ll be billed $180 yearly, $120 for six months and
$30 month-to-month”. We also added this area which
said, “Total charge today: $180.” So, that they knew,
even though they were choosing an option that said
$15 a month, that they would be charged $180. And
when you move from – when you toggle between the
options, this amount would change based on which
option you chose.
So we saw great results with that. We did see a decrease
in the refund rate and the people asking for prorated
refunds, but it still wasn’t back down to our former level
so we’re actually still testing this … On the current page,
| 99
we actually offer the subscription options horizontally,
something new that we’re trying. We recommend the
twelve-month option and we moved the ‘Total Charge
Today” to above the button, so on their way down there’s
no way that they could miss it. And we’ve maintained
the more optimal subscription mix, but we’ve actually
moved even more people from the one month to the
six-month subscription and we also reduced the refunds
requested by 20 %. So, we’re getting a lot closer to
where we want to be as far as customer service and
extending the lifetime value through the subscription
mix. We’re going to continue to test this with different
copy and design changes as well.
Something else that we did in conjunction with this
– every business is different, but at TheLadders, we
actually have a lot of people who come to our site,
they sign up as premium and then they leave. They get
a job or they decide they don’t want to look anymore.
They downgrade and then a little while later, their boss
pisses them off and they decide, “I’m going to pay for
TheLadders again.” So, we thought, for people who’d
been on the service before, two things: One, we have
your credit card information, so you really don’t have
to put it in again and two, you know what the service
is like so you’re more likely to take this longer-term
subscription because you’ve had exposure to it in the
past.
So, we started a different upgrade page for these people,
for returning visitors, and we have the same type of
options - $15, $20 and $30 a month – but we’ve prepopulated all of their credit card information so that we
didn’t have to ask them to dig into their purse or their
pocket to find their credit card again and type it in. We
also show them the total charge amount as it was and
we saw a 379% increase in conversion rate over the
regular page.
So, this was a huge win for us as far as conversion
rate and we were able to put these people at the same
rate on a longer-term subscription. So, if you do have
people that are coming back to the site a few times,
paying, leaving, paying, leaving, we definitely suggest
looking at pages that are different for the people who
have used your service before than for new people who
haven’t had exposure yet.
So, I’m going to hand it over to Michael, who’s going
to talk a little bit about what we do after this point
through email and other onsite communication.
Michael McCurdy: Thank you, Leslie. What I’m going
to take you through are a few of our efforts that we
100 |
use to convert our basic members to premium and also
extending our premium membership to a longer term
subscription.
What I’m showing here is an offer that we tested late
last year. It was the August, September time frame of
last year, 2007, and what we did was we tested six
different offers over an eight-week period. And as you
can see here, we have $15 off. The one on the left is the
actual email and then the landing page is $15 off.
Now, if you come to TheLadders.com as a basic member
and then you hit an upgrade page that Leslie was
showing you earlier and then you abandon that page,
you will get this email on the left the following day
indicating, “Hey, get $15 off.” And then you’ll go to the
landing page and upgrade. So, what we’ve done here is
actually we saw a really good success with this email
campaign so what we determined we needed to do from
a testing perspective is to figure out what is the optimal
offer that we should give to people in the process, not
only to increase conversion, but also extend the lifetime
value. So we were pushing people to the six month and
year packaging.
So, the offers we tested were a $5 off, 10% off, $15 off,
$10 off, 33% off and then 25% off. So I’d like to have
a show of hands. Who thinks $5 won in conversion and
then also shifting to the longer term mix? $10 off? 33%
off? $15 off? 10 % off and 25 % off? OK. It looks like
most people picked 33% off or $15 off. And the winner
is – let’s see. We’ll find out in November, the winner on
that one. Unfortunately, we don’t know that one today.
So the winner is actually 25% off.
So, what we did here was we monitored the results over
an eight-week period, did very controlled testing to make
sure the test was accurate. And, if you go back here to
the actual creative—so when you’re A/B testing, you
want to make sure you don’t have too variables in your
A/B test because then the more variables you put into
the creative and also the landing page, the longer you
have to run the test to see which variable is actually
impacting. So the only variable that we changed in each
of these creatives was on the email. We just said $15 off
or 25% off, and the same on the landing page. That was
the only variable that changed. The subject line was the
same. The copy was the same in each of these emails.
So it was a pure, accurate A/B test.
So it was the 25% off. And here are the results of the
offer testing: We had the control as the $10 off and then
as you can see here, we had conversion to open, the
one-month package, which we saw an increase either
in the one-month package, the six-month package, the
one-year package and then the average cash inflow per
week. We can’t give you the actual numbers here, but
if there’s a smiley face, that means it is statistically
significant.
So as you can see here, 25% off and 33% off were
winners across the board. But what we noticed here
was in the six-month to the one-year package mix, we
actually saw a higher increase in the shift in the 25 %
off against the 33% off, which was a real interesting
finding for us. So we actually – for the cash inflow
– we were actually getting more cash inflow for the
25% off versus the 33% off, which kind of goes against
conventional wisdom. But that’s why we test to make
sure that our test results are actually accurate and we
have to go with what the actual test results say.
And then you can see here the $5 off did not win anything
against the $10 off, the 10 % off won in the package
mix shift and then the average cash inflow, and the $15
off did win with the package mix shift, which actually
made sense because people were actually getting more
money off the longer term packages.
The next thing I’d like to talk about is what we call
search and rescue. Actually, on our site, compared to
Zagat, you can actually turn off auto-renew and cancel
online. You don’t have to actually call. So one of the
things that we have on our site is where we give people
the ability to turn off their auto-renew. So if I’m a
premium member today, I sign up today. Tomorrow I
can actually turn off auto-renew and then I’ll have 29
days left on my subscription. So we call those people
in limbo. That’s what we call them. We do search and
rescue efforts, as I like to call it.
So what we did here was we actually did an A/B test,
and we send this email out a few days prior to the
termination of their subscription, kind of our last-ditch
effort to get people more involved into the site to see,
hey, can we actually get people involved at the site,
give them a discount. We actually have a green button
here because it went better with the creative on this
instance – and then actually get them more involved.
So what we did here was this is an A/B test running on
the creative. These are the actual emails. We have the A
email on the left and the B email on the right and this
is what we call LadderStats email. So, what we did here
was we actually have all the copies identical in each
of these emails, and then we actually tell the member
their stats in each of these emails.
Well, here on the left, we can say, “Completed your bio?”
What your stat is versus our successful members and
then we give the value proposition. Ninety-one % of
the people who have landed a job through TheLadders
completed their bio. So, actually here what we did was
we don’t give the person the stats. Here, we actually
give the user the stats. So, we’re actually doing a
comparison on a A/B test. What would actually win?
Would it be more enticing for a member to know their
stats on the email itself or actually have them click
through?
Convention wisdom would probably tell you the
clickthrough would actually be more enticing. So, what
we did was this was the actual landing page here that
the user would land on. So, no matter if we showed the
person the stats or not, they’d actually land on this
page with the stats incorporated into their landing page
so these will be personalized to you as the consumer or
the premium member.
And then we have the actual memberships. We actually
tell the member here, “Reactivate your account and get
25% off.” It’s hard to see here, but we actually tell the
member how much additional discount they’ll get. And
then like what Leslie was referring to earlier, we have
the one-click here, as well, where people can actually
reactivate their account and get the total charge today.
Then as you can see here, we show the stats.
So, we’ll go back to A and B on this page. Who thinks A
won as far as converting people? Who thinks B won? I
kind of gave it away earlier, so – actually the winner is
– actually, B is the winner. So telling the person their
stats in the creative actually helped us not only through
the conversion effort, but also helped us reactivate
members. We saw a 34 % increase in conversion to
open, we saw an increase of 50% in conversion delivered
and a decrease of 133% in clickthrough, which made
sense because we didn’t show the stats in the other
email. So even with a 133% decrease in clickthrough,
this creative actually won, which kind of goes against
conventional wisdom because you would think the
higher the clickthrough, the higher the conversion.
Participant: That’s net conversion after this?
Michael McCurdy: Correct.
Participant: So the conversion rate?
Michael McCurdy: The overall conversion rate was
higher for this creative, yes. OK. One thing that we also
do is subscription extension emails with one click. So,
if you are a premium member today and you sign up
for a month membership or a six-month membership, a
| 101
few days after you sign up, we’ll actually send you this
email saying, “Hey, get two months free, 33% savings,”
to push you to a longer-term subscription. So, in this
case, we’re actually pushing someone on a one-month
subscription to a six-month subscription. And then if
you’re on a six-month, you also get one that has a little
different messaging as well to push you to the longerterm subscription.
And then we also send this email after someone actually
applies to a job. We actually send this based upon user
experience where it makes sense where someone might
be more enticed to actually extend their subscription
once they have actually done something on the site. So
we have two months here free and then as you can see,
the email is on the left and the landing page is on the
right. It’s very important as you’re creating the emails
to the landing page to have a seamless experience to
make sure the messaging, the content, the copy, the
design all flow into one because once you have emails
that are disjointed with the landing page, you lose a lot
of drop-off at that point.
So, two months free, and then here actually we have the
messaging as well. It’s hard to see here. Actually what it
says is you’ve already paid $30 a month for four months
and get two months free and then we’re actually going
to charge you an additional $90. So, it’s very clear to
the consumer what we’re actually going to be charging
them because they’ve already paid us $30 a month and
then they get transferred to the six-month subscription,
we’re actually going to charge an additional $90.
And again, this goes back to the one-click upgrade
or re-upgrade, so we actually embed the credit card
information there for the consumer so they don’t have
to re-enter their credit card. So, all they have to do
is click and they actually upgrade immediately at that
point to a longer-term subscription.
And on this, we call these transitions, so anyone who
transitions from a one-month to a six-month or to a
year or from a six-month to a year package, we saw a
65 % increase in transitions just due to this email effort
which was great, great results.
The takeaways for today? The first one is make sure you
optimize the billing and upgrade pages while keeping
a close eye on refunds. One of the things that we did
notice, like Leslie was talking about earlier, we saw
great results, but we also saw an increase in refunds,
and to really balance the equation and to constantly
tweak that to make sure we bring the refunds down to
a lower degree and to make sure that we’re optimizing
102 |
the page, so we test and re-test.
Incorporate the one-click re-upgrade and appropriate
discounting. So one of the things that we have learned
is the one-click re-upgrade really does work if you have
the customer’s credit card information. Make sure that
you keep that information in a secure file, of course,
but when the customer comes back to re-purchase, you
have that information immediately available so they
don’t have to re-enter the credit card.
And then also appropriate discounting. So, in your
business, it may be a little different as far as the
discounting and what you actually need to discount, but
make sure that you’re offering appropriate discounting
when it actually makes sense. We had a really good
success with that email campaign where people are
abandoning the billing page and then we send that
email, because in the consumer’s mind subconsciously
they actually know why they’re getting the email. “Oh,
I went to the page yesterday. Therefore, they’re offering
me a discount today.” And it actually made sense in the
user experience.
And don’t be afraid to ask your paying customers to
extend their membership when it makes sense in their
experience. So, hitting people right when they’ve
purchased, a few days after when they’re getting more
involved into your site, when their site activity has
increased. So, we really want to make sure that we’re
asking the members to extend their membership to
also extend the lifetime value, and test and test some
more.
So one of the things, like Leslie was talking about
earlier, is we test many, many things at TheLadders and
the test results don’t lie. A lot of times, even though I
want one creative to win, if the statistics say the other
creative wins, I’ll have to go with that. So with that,
that’s it. So, we’d like to open up the floor to some
questions.
Participant: You said that you contact people quickly
after they sign up initially. In other words, you’re going
to try to upgrade them as quickly as you can. What
about the longer-term people? You didn’t say – at least
I don’t recall you saying – exactly how you decide how
much time to allow before the expiration, let’s say, of
a six-month or a one-year term member. Do you notify
them? Do you send the upgrade three months out, five
months out, what?
Michael McCurdy: Yeah. We also have another
campaign when somebody cancels. Are you talking
about canceling their subscription or just like notifying
them?
Participant: No. Trying to get at them to extend their
subscription.
Michael McCurdy: Yeah. We usually do it just at the
beginning. We haven’t really tested doing it further into
the subscription process at this point.
Participant: Even the one-year people? They sign up
initially for a one-year subscription.
Michael McCurdy: Yeah. We don’t do anything to
extend that at this point.
Participant: OK. What do you do when you’re – and
what if they have gone to turn off their auto-renew and
they’re one -ear subscribers? When do you come after
them?
So, if you haven’t been back on our site in about four
weeks, you actually won’t be in the search results for
recruiters. Then we’ll market to the users to get them
to come back in and use the site and be more engaged
with the product.
Participant: That last piece was the part that I was
curious about. What sort of marketing have you done
of that type and what sort of quantitative impact have
you seen from it.
Leslie Semegran: I couldn’t, at this point, although I
could get back to you about the quantitative impact,
but we do mostly email campaigns that trigger off of
site activities, so either job applies or on the reverse,
trigger off of inactivity. But as far as quantifying that
off the top of my head, I’ll have to get back to you.
Leslie Semegran: We send an email to the longer-term
subscribers, to the 6 and 12, telling them that their
subscription is going to be finishing and that they’re
not going to be auto-renewed, so they’re going to miss
our service. I think it’s about 7 to 14 days before the
actual service is shut off at this point.
Participant: You talked about in the reactivation
campaign going from the email to the landing page and
the landing page was pre-populated or their credit card
was already on file. The email that you – the one-click
email where you’re trying to get them to come back to
the site, to me – we send them back to the site, but we
have them log in first. It didn’t look like there was any
kind of log in.
Participant: OK. Is that something you’ve tested? In
other words, why 7 to 14? Why not 30 days? Why not
5 days?
Michael McCurdy: Yeah. If they’re auto-logged in, it’ll
automatically go. If they’re not, they’ll get a sign-in
screen. I just didn’t show it on here.
Leslie Semegran: We haven’t tested for the longer
subscriptions at this point, but it’s a good point and
something that we’ll be looking at.
Participant: OK. So you’re not afraid of if you send the
email to the wrong person having access to credit card
information.
Participant: You mentioned at the beginning of your
presentation that the purpose of this was to increase
your retention rates overall. You showed some examples
of offers that you had used to do so. Have you done
– this is a two-part question. Have you done any
development of engagement programs to increase your
retention rate and, if so, can you quantify the type of
impact they had on the retention rate?
Leslie Semegran: No. If they’re at a computer that
they’ve been logged in at before, then it’ll take you
right there. But if not, it forces you to log in to get into
your account.
Leslie Semegran: Well, with this campaign, we were
looking to increase the lifetime value by putting them
on longer-term subscriptions. But as far as engagement,
we do do a lot of email and site activity-based programs
that trigger off when people come to the site, click on
jobs and then apply to jobs. So, when we know that
they’re kind of like back in our web, we’ll send them
certain offers like Michael alluded to or we actually
activate and deactivate people into the search results
for recruiters.
Participant: The examples we’ve seen are all using
radio buttons with the monthly, six-month, yearly. Have
you considered testing with a scrolling menu defaulting
on the yearly option so you would have to scroll down
to see the monthly that you kind of do not want people
to subscribe, by default?
Michael McCurdy: Yeah, and we also don’t show the
full credit card information as well. We just show the
last four digits.
Leslie Semegran: We looked at – we were doing some
homework and we looked at one on Blockbuster that
showed – it only showed the subscription they wanted
you to take and then you had to expand to see the other
options and we haven’t tried that yet mostly because
| 103
they actually had more offers than we did. They had
a whole bunch of different types of packages and ours
was pretty straightforward, so we haven’t tested that
yet, but we’ve seen it on other sites.
Michael McCurdy: Yeah. And currently also in our
subscription mix, the only difference between the onemonth, six-month and year package is time. We don’t
have a better offer if you get a longer-term package
except the discount currently.
Participant: When you have people who are in a sixmonth or an annual subscription and they’re going
to expire and you want it to renew, do you offer any
incentives for them when you communicate with them
to try and get them to renew or do you just ask them
to renew again?
Leslie Semegran: We have everyone on auto-renew.
Michael McCurdy: If you’re on a six-month or annual
package, you’ll get a notice saying, “Hey, we’re going
to charge your credit card.” And then if you don’t do
anything, we’ll auto-renew.
Participant: You addressed this a little bit earlier when
you were talking about sort of behavioral segmentation.
I’m also curious, do you do any sort of education about
features that people maybe aren’t using, like if they’re
applying for jobs but they’re not using the recruiters or
some of the other features? Have you tested anything
like that?
Leslie Semegran: Yeah. Well, we send a good amount
of email, as Michael can attest to, so we have weekly
newsletters and weekly career advice. We have a weekly
comic and also a weekly tip and most of those will be
geared toward using certain activities on our site. So,
when we came out with our career advice section, we
talked to people about going and checking that out.
We also do newsletters that talk about jobs that were
recently filled, so we tell people about other people’s
success on the site to get them to come back and reengage and apply to jobs themselves. So, we do talk
quite a bit about the features and things that they can
take advantage of.
Participant: Do you ever target based on maybe things
they haven’t done?
Michael McCurdy: We do some campaigns where if
people haven’t signed up for an email job alert, we’ll
send them notifications, “Hey, this is the best way to
find out what’s going on in the marketplace through our
email job alerts.”
104 |
Leslie Semegran: We also have some onsite targeted
messaging. If you’re in our job search and you go in and
search for a director in New York, it’ll ask you to take
steps behaviorally that’ll lead to you getting more email
and more job leads that will result in you hopefully
upgrading, or if you’re already premium, applying to the
jobs. So, if you haven’t saved the search you ran, we
say, “Hey, save this search.” If you’ve saved and you’re
not getting an email, we say, “Hey, get this email.”
And for our premium service, we also have a resume
critique service. We give everyone a free critique and
then they can look into buying professional resumes, so
we’ll promote that a lot as well, based on whether or
not you’ve done it before.
Michael McCurdy: Also, if people haven’t uploaded
their resume – you’re not successful on our site unless
you actually are active on the site and actually have
your information out there – we really try to get people
to upload their resume because we know that’s another
way to conversion as well, to guide people through the
process. They sign up, upload their resume and then
they actually upgrade. So we know there’s kind of a
process that goes through the consumer mindset.
Participant: In your example, you showed three pricepoint options. Have you also tested with just two versus
the three and which one worked better?
Leslie Semegran: The only place that we’ve done that
is for the free trial. We’ll offer people a free trial based
on certain other behavior or time that they’ve been on
our site and we’ve tested between offering the options
versus just automatically rolling them to a monthly so
they don’t have to choose or see what the different
longer terms are. We just roll them right onto the
monthly and I think you have the results on it.
Michael McCurdy: Yeah. Initially, I’ve seen more people
actually taking the offer. But monetarily speaking, it’s
better to have multiple options to select from, especially
on a free trial. You want to push people to the longerterm subscription so when they roll to an upgrade, we
benefit from that from a business perspective.
Participant: Two questions please. One, do you notice
anything different about the patterns, the usage
patterns, of month-to-monthers versus six-monthers
versus yearly people? And the second question is have
you considered adjusting price to push people towards
desired programs, so making the monthly fee a little
higher or decreasing the yearly fee to be a little lower?
Leslie Semegran: On the price one, we’ve done a lot
of price testing of different groups as a whole, so a
whole group for 25 and other options that go with it.
We haven’t actually just pushed one without pushing
the other one, so we’ve always kept kind of the same
relationship with discounting between the monthly,
six-month and twelve-month subscription. And on the
usage patterns –
Michael McCurdy: Yeah. You have to keep a close eye
on – to see what acquisition channel that you receive
a paying customer by and if they came in through a
discount, it’s something that we really look at and
analyze daily.
Participant: Since you’ve just started running TV last
year – which, by the way, being a tennis fan, I see it on
the Tennis Channel all the time and love it …
Michael McCurdy: Our CMO is two rows behind you and
he can thank you.
Participant: I’m wondering if you were able to measure
the impact of TV to your business? Are you looking at
your site conversion metrics and how much of a lift that
you see and are you running your TV all year long? Are
you able to see the differences in when it’s in market
versus not?
Leslie Semegran: Yeah. We just started in January
and that was just kind of a whole new world for us
because we are extremely numbers based, looking at
every single person on the site, seeing when they’re
converting, how they’re converting between an online
ad that you can see, an SEO. So we were pretty nervous
when we launched the TV because it’s not as measurable
or easy to see, but it was pretty obvious. We saw a huge
lift in the traffic that came in specifically through paid
search, SEO, and then also the people who came direct
to site.
The problem with measurement we have is that we can’t
weed those people out of the regular people we would’ve
gotten without the TV commercial. So we’re really
spending a lot of time looking at how the categories
are acting this year as opposed to last year at this time
to see different types of conversion rates and we’ve
seen some categories go up and some categories go
down. We also look at renewal rates, guest-to-reader
conversion, reader to page, all of those metrics. And it’s
definitely tougher than it was before, but net/net, we
just have seen so much more traffic. We’ve had so much
more visibility and so many more people paying us as a
result of the commercials.
Participant: Have you tested three months?
Michael McCurdy: That’s a great question. Yes, we
have tested three months. We’re actually getting ready
to launch another test with three months as the bottom
option.
Leslie Semegran: Now, our U.K. site that launched
last summer has a three-month option as well. That’s
apparently a very typical subscription length over there,
as well, and they’re seeing a lot of success with it, more
than we have.
Michael McCurdy: Yeah. It’s something that we’re
going to re-address because we need to get real with
our customers, as well. It’s going to take you longer
than a month to find a $100k+ job in most cases.
Participant: So your one month is not an auto renew
then?
Michael McCurdy: Yes. Our one month is auto-renew.
Participant: Everything is an auto-renew.
Michael McCurdy: Everything is auto-renew.
Participant: And you still follow – you don’t just want
to just keep them going. You follow up with them?
Michael McCurdy: Well, we see different behaviors
with different types of package mix, so we really
monitor the behaviors and what the lifetime value is of
a one-month versus the six-month versus the annual.
So it’s definitely beneficial to the company to push to
the longer term subscriptions.
Participant: I just had a question on your TV ad. Did
you guys have the discussion about doing a unique
URL and just redirecting to the site and tracking it that
way?
Leslie Semegran: We didn’t act – or I wasn’t – neither
of us were a part of this.
Michael McCurdy: Yeah. Our CMO is – talk to Robert
Turtledove right there. He’s our CMO and he can answer
that question offline. Or do you want to answer it now,
Robert?
Robert Turtledove: I think the simple equation is if
you have a URL, use it.
Leslie Semegran: But we did approach it very much
that this is a brand campaign. We weren’t looking for it
to be a direct marketing commercial.
| 105
Participant: I was just going to comment on what he
said. At Weight Watchers, we have the same issue of
wanting to track our TV, and we’ve used vanity URLs in
our ads and people don’t go to them, so it doesn’t help.
They just go to WeightWatchers.com.
Moderator: All right. Well, I think we’re going to take
a break here. I want to – everybody give a round of
applause to our panel. Thanks.
106 |
Keep Them Wanting More:
Maximizing Customer Lifetime Value
Through Development & Delivery
Michael McCurdy, Director, CRM
Leslie Semegran, Director, Online Marketing
TheLadders.com
available from::
What We Do…




available from::
Helping $100k+ Job Seekers stand
apart from the crowd and get hired!
We put the jobseeker in control of
their own success by using our
service.
We have the most $100k+ Jobs.
Jobseeker pays subscription fee to
apply to jobs and get full access to our
website.
2
| 107
Extending Subscription Length to
Extend LTV




available from::

$30 Monthly
$120 6 Months
$180 Annual
29% increase in LTV for 6 month subs
vs. Monthly
39% increase in LTV for Annual subs
vs. Monthly
3
Tackling The Issue

What are the JobSeekers most likely to
choose when they first upgrade?


available from::
How can we change that and not drop our
overall conversion rate?
If the JobSeekers choose Monthly, can
we move them to 6/12 month subs
after the fact?
4
108 |
Upgrading at TheLadders.com
available from::
5
Upgrading at TheLadders.com
available from::
6
| 109
Getting There…
available from::
7
Results




Neutral Upgrade Rate
15% decrease in the % of people
taking the Monthly
250% INCREASE in the % of people
taking the Annual
Neutral for the 6 month
available from::
8
110 |
Too Good of a Job?

It turns out, many of our users were
confused…
“I did not want to sign up for a year only month to month
for $30. Please adjust my billing info and credit my
account. Thanks”
“I signed up for one month of membership, and the
available from::
congratulations page states that I have signed up for
one year. I do not, yet, want to sign up for one year,
please resolve this immediately! Do not charge my
card for 1 year, only charge $30. I will dispute any bill
greater than $30. Thank you very much for your
prompt attention in this matter.”
9
Play it Again, Sam
available from::
10
| 111
Current Test
Results
Maintained
more
optimal
subscription
mix (moving
even more
monthlies
into the 6
month
package)
and reduced
the rate at
which
refunds
were
requested by
11
20%.
available from::
Returning Visitor Version
available from::
379% increase in conversion rate over the regular upgrade pag
12
112 |
Offer Testing - % off vs. Dollars Off
available from::
• Tested 6 different offers over the course of 8 weeks
• Looked to see how each offer affected:
• Conversion from the email and subscription package mix of the upgrades
13
Offers Tested
25% Off
$5 Off
33% Off
10% Off
available from::
$10 Off
$15 Off
14
| 113
And the Winner Is……
available from::
15
Results of Offer Testing
Offer
Conversion (to
opened)
1
Month
$10 off
6
Months
1
Year
Average Cash
inflow / week
Control
10% off
25% off
33% off
$5 off
$15 off
available from::
= statistically significant
16
114 |
Search and Rescue
available from::
17
Rescue Me!
available from::
A
B
18
| 115
Just The Facts Ma’m
available from::
19
And the Winner Is….
Increase of
34% In
Conversion
To Open
Increase of
50% In
Conversion
To Delivered
available from::
Decrease of
133% In
Click-Through
20
116 |
Subscription Extension
With One-click
Increase of
65% In
Transitions
available from::
21
LTV Takeaways



available from::

Optimize billing/upgrade pages while
keeping a close eye on refunds
Incorporate one-click re-upgrade and
appropriate discounting
Don’t be afraid to ask your paying
members to extend their membership
when it makes sense in their
experience
Test and test some more!
22
| 117
Credits/Thank You
Michael McCurdy, TheLadders.com
646-453-1949
[email protected]
available from::
Leslie Semegran, TheLadders.com
646-391-4370
[email protected]
23
118 |
The Economy of Paid Search: When and How to Use It for Maximum Effect
Session Description
This case study will detail recent paid search campaigns
for AG Interactive properties. Get practical information
on the relationship between SEO & Paid Search/Media,
the impact of other media on branded search terms, and
the impact of the high costs of search terms.
About the Presenter
roles in the ecommerce divisions of national brands such
as OfficeMax and the Home Shopping Network (HSN). He
is currently the Vice President of Acquisition Marketing
for AG Interactive, the digital division of American
Greetings Corporation. Eggleton also serves as President
of The Web Association, a northeast Ohio-based trade
organization, and on the Advisory Board of Cuyahoga
Community College’s Interactive Media program. A
frequent speaker at various local and national industry
events, he holds an MBA in Marketing from Cleveland
State University, with a BSC in Communications from
Ohio University. He resides in Bay Village, Ohio, with
his wife Sharon.
About Ag interactive
John Eggleton
VP Marketing, AmericanGreetings.com
Jon Eggleton is an experienced ecommerce and direct
marketing executive. He has worked in the interactive
space since early 1998, having held various management
PRESENTATION
MODERATOR: If you guys were here first thing this
morning and saw Stefan’s talk, he presented some
charts that looked at - we asked some questions about
the sources of traffic, the different channels you’re
getting traffic from and the quality and quantity of
those visitors. If you remember that bubble chart,
SEO is sort of the big standout bubble in both charts,
and then paid search was doing pretty well for both
consumer and business-to-business in terms of quality
and quantity, but it definitely was nowhere near as wellrepresented as SEO.
I think with paid search we’re probably - most of us
are using it in some way, but there’s always challenges
with it. The cost per clicks can sometimes go up. The
competition for the most important keywords that
you’re targeting can sometimes get pretty intense, and
it makes paid search one of those areas that it seems
like really hard to get your hands around, and it’s always
shifting around on you.
What we’ve done is asked Jon Eggleton to
come up and talk to us a little bit about their
paid search efforts at AG Interactive. Jon.
With well over 3.4 million subscribers to date, AG
Interactive is one of the largest subscription networks
on the web. Properties include AmericanGreetings.
com, Blue Mountain, Egreetings, Kiwee, Webshots,
PhotoWorks and co-branded eCard sites operated in
conjuction with MSN, Yahoo!, and AOL.
Jon Eggleton: All right everybody … First off, before
I get started, with a show of hands, who here in the
room is involved directly or indirectly with your paid
search marketing efforts? Wow, OK. Quite a few of you.
So the pressure is on to deliver a good presentation
here, I guess.
I’ll just give you a brief introduction about who I am, as
well as who my company is. We are American Greetings
Interactive. We are part of a 100-year-old greeting
card company, so we have that history in the socialexpressions space behind us.
We are one of the largest subscription networks on the
web. We’re at least in the upper tier. We have currently
3.7 million e-card subscribers to date. As an interactive
unit, we’ve been around for about 14 years, half of that
in the subscription space, so we’ve seen kind of the
highs and lows of the dot com boom, and then the bust,
and sort of rode the wave since then. We’ve definitely
seen a lot of that good history going on.
We are primarily known – and I’m going to speak today
about primarily our e-card sites, and our paid search
efforts against that. We have since – in the last couple
of years branched out to a couple different areas, namely
Kiwi, which is actually an organic site that we started
| 119
about two years ago. We purchased that domain from
a French mobile company and turned it into a social
networking content site. We actually offer content for
MSN Messenger, AOL AIM, as well as content for MySpace,
Facebook, BiBo, what have you. If you’re in the market
going after high school and college-aged students,
you’re very familiar with those different networks.
More recently, about seven months ago, we actually
purchased a couple of photo sites. PhotoWorks and
Webshots are actually two companies that we bought
about seven months ago, and both of those kind of
represent and play in different spaces in the digital
photography world. Webshots is really more about Web
sharing, or a photo sharing Web community. It has a
lot of traffic. PhotoWorks is really more the e-commerce
engine. Things like photo books, putting your picture on
mugs, and mouse pads, and things of that nature. That’s
really what PhotoWorks is all about. We’re in the process
of kind of integrating those two businesses together.
Overall, in terms of our business model, subscription is
still very near and dear to our heart. It’s still our number
one primary business model on the Web. Secondly, we
also sell advertising. It sounds like many of you are also
subscription first, advertising second. We’re very similar
to that.
Third, we have a couple other business models as
well, that are a little bit smaller for us. I mentioned
PhotoWorks and e-commerce, it’s part of that, as well
as we have some smaller businesses in search and micro
payments. We have a toolbar product where we actually
make money through monetization of different search
marketing efforts, or search term efforts. So that’s a
little bit about us.
Personally, I’m responsible for acquisition, conversion,
and retention marketing. So, basically getting them to
the door, getting them to buy once they get to the
door, and then obviously retaining them on the back
end as well. In terms of our paid search efforts, we
actually buy paid search for two of our e-card brands,
American Greetings and Blue Mountain. We’re going to
talk specifically about those.
We are sort of a hybrid model in the sense that we manage
our paid search program in- house. We do actually get
some outside help with that. We use a firm called Clear
Sailing out of Columbus, Ohio. It’s sort of a hybrid in
the sense that we have people who actually manage
the programs and really are kind of monitoring what’s
happening on an internal basis, but also working with a
third party who really provides the technology and a lot
120 |
of the day-to-day kind of management and optimization
of those different bids. As you can imagine, managing
a large paid-search program, there’s a lot of focus we
have to put on kind of managing some of those details.
So, the agency helps us with that.
We’re also a very seasonal business. As you can imagine,
we’re American Greetings, so holidays are obviously very
large for us, everything from Christmas and your winter
holidays, to obviously Mother’s Day. Valentine’s Day is
really the – that’s the big holiday for us. If we don’t
hit our subscription numbers on Valentine’s Day, there’s
probably a pretty good chance we’re not going to that
year. That’s definitely a big focus for us. What I’m going
to talk about specifically in the case study portion is
really what we did around Valentine’s Day this year and
how that compares to what we’ve done in the past.
As far as our specific challenges, maintaining growth
is difficult for us. We’re in a space where e-cards have
been around almost as long as the Web so, in terms of
growth, we see numbers around 25 to 30% is how fast
the search space is growing. Our category, we believe, is
growing at about 9%, so definitely we’re growing slower
than the Web is overall.
We have a relatively low lifetime value. I heard
TheLadders, which gave a great presentation, and they
shared with us some of the monthly subscription rates
they have. We would kill for a $30-a-month subscription.
Instead, we primarily sell through annual subscriptions,
which for e-cards is $13.99. We also offer a Create and
Print product, which retails for $19.99. These are per
year, by the way. We also offer a bundle at $29.99. You
can see that obviously the lifetime value is somewhat
capped. We have these other brands we can up-sell,
and we’re looking at doing that with some of our other
companies, but obviously we are constrained in what
we can afford to pay for an actual customer.
We’re also seeing a lot of increased competition, not
only from people getting into our space, but the fact
that – and I’ll talk about this later – but the e-card
space in search is actually fairly inexpensive in terms
of an acquisition for traffic capability. Really what’s
happening from all this is we’re seeing higher CPCs, our
clickthrough rates are lowering, which we kind of see
as a double-edged sword. Really, we’re needing to focus
more on optimization.
I wanted to share with you this is kind of a generic
overview that our search agency did for us on the
greetings category for paid search. As I mentioned,
CPCs are fairly low in this category. Some of you in
the audience are probably salivating over 23-cent CPCs.
Obviously that’s partially driven on the monetization
models for folks like ourselves who are in this space.
You can see the average clickthrough rate impressions.
This basically is taking into account American Greetings
as well as the other competitors that we have in our
direct space; just kind of give you a snapshot of what
our category looks like as we kind of dive deeper into
this.
Getting specific to American Greetings: When I say
American Greetings, by the way, I’m referring to
AmericanGreetings.com and Blue Mountain combined,
since they’re both of our e-card sites that are both
subscription based, to kind of give you an idea of what
we’re talking about here.
In terms of our own: What we’re seeing year-over-year
trends, we have three different major buckets. We have
obviously our branded terms, so American Greetings,
Blue Mountain, and variations thereof. We have our ecard bucket, which is really our largest bucket of terms
that we go after, as well as Create and Print which,
as I mentioned, is our print-at-home product that we
offer as well. You can kind of see across the board,
and obviously there’s some of these numbers that really
stick out.
Clickthrough rate for our branded terms has gone down
about 9%. We’re seeing broad matches for people using
the word “greetings,” which obviously is part of our
corporate name. Average CPCs, as a result, have gone up
almost 300%. So, basically, whereas about a year, year
and a half ago, no one was bidding on these terms but
us, and now we’re seeing some increased competition
come into that space. Obviously, that’s changed the
dynamic of what the CPCs look like.
E-cards, not quite as pronounced, but a very similar
trend. Clickthrough rates are actually going down by
almost a third. One of the main reasons for this is
we have more people who are actually playing in the
space, and that’s driving down our rankings on certain
key words. We’re having to become very strategic about
where we place our bet. Average CPCs, again, up 33%.
It’s become a much more expensive game than it used
to be.
Create and Print, similar story. Clickthrough rates
are down. CPC is actually up. That’s a slightly less
competitive category than e-cards is. We’re not seeing
quite the extent of some of the happenings there.
This is actually a year-over-year chart showing you our
clickthrough rate over time. I know this is probably
hard to see. I actually got rid of the numbers on the
left so you can just kind of focus on the trend itself.
You can see here, this is actually starting in March
of ’06, and then this builds up to about November of
2006. During that time we were being very aggressive.
We were monetizing to our clickthrough rate. We were
monitoring our conversion rates very closely, and really
trying to build our campaigns based on a lot of those
front-end analytics. That was all happening during this
curve.
Around November of ’06 is when we really started to
see a lot of those new competitors, and I’ll talk about
who some of them are and how it’s impacted us. A lot
of those competitors then came into the space around
that holiday timeframe in 2006, so it’s been about a
year and a half or so that there’s sort of been a new set
of rules of engagement, if you will, in our industry.
Starting with November of ’06, we saw some more
competition. We’ve become more aggressive on certain
terms, but generally speaking, we have seen our overall
rankings go down as we can’t afford to pay certain
levels on certain keyword terms. We did see a little bit
of an increase here. This is actually October of ’07, so
this is going back to late last year. We did become more
aggressive in some of the holiday-specific terms where
we decided that from a profit standpoint those were
terms that we needed to go after, and we could go after,
based on our own profitability metrics. By this point we
had enough knowledge of what was happening that we
could really become more aggressive there.
Traditional competition. This is just kind of a quick
snapshot of who we’re actually competing with out there.
Obviously Hallmark is out there as our virtual as well
as our offline competitor. MyFunCards, which actually
is part of Interactive Corps advertising and search
group. They’re actually monetized through a toolbar
model primarily. It’s a downloadable application that
you have in your machine, and they make money, then,
on searches that are taking place. You see Rattlebox,
WishCards, IncrediMail, Smilebox.
Sort of the interesting thing here is really a number
of these companies were coming at this industry from
different places. I mentioned MyFunCards monetizes
through a toolbar. Hallmark, in fact, gives away e-cards
for free. E-cards is really – I don’t want to say a lossleader – but it’s really a mechanism to drive traffic to
their site, which they intend to convert with e-commerce
and retail products, and also driving offline traffic to
| 121
their Gold Crown stores. We know that online is a big
part of what they’re doing in the offline world, which is
much different than what we’re doing today.
Where that becomes applicable to somebody like us is
as we’re kind of driving towards maximizing the CPA,
looking at a certain acquisition rate that we’re looking
at. They’re kind of playing by a slightly different set of
rules, but at least it’s a little bit closer to what we’re
doing.
Now the non-traditional. This has really been, I would
say, the scarier set of competitors for us, just because,
unlike the previous slide where there’s some differences
in business model, these guys are really going at this
for totally different reasons. I mentioned that the
average CPC in our category is about 23 cents. That’s,
for the most part, very inexpensive compared to other
high-traffic categories like ours. What we’re seeing is a
number of different competitors coming up in different
areas. The first one will be retail.
You see Sears is up here, you’ve got Target, Wal-Mart.
These companies are actually developing, in some
cases, very basic e-card products or offerings – where
it’s really not something they’re focusing on. They
might be having an agency do it for them. It may not
even be Flash. It might just be static graphics, or basic
animation.
Really, they’re using e-cards, again, as kind of that
loss leader. Using the convenience store model, that’s
sort of the gallon of milk that they get you to walk
to the back of the store and hopefully buy some other
things on your way back to the front. That’s obviously
impacting our metrics in terms of what they’re able to
pay, because they’re really looking at it from a different
model.
Similarly, we have a couple of what I would call CPG
companies. We have folks like Coca-Cola, Clos du Bois,
which actually is a – I don’t know if I’m pronouncing
that right. That’s actually a large winery in Sonoma.
They’ve run a couple of very clever viral marketing
campaigns, and they’ve used e-card terms on the search
engines to really drive traffic to that. They ran one called
kissandtell.com for Valentine’s Day, so that was going
on against us during – I mentioned our big holiday of
the year – where they were bidding on traffic.
I would venture to guess that – they probably didn’t
care if – the difference between 23 cents and 27 cents
was probably not too much to them since they’re really
looking at driving traffic to the site, and really just
122 |
trying to build a relationship with that consumer. The
way the site worked is you came in, you actually shared
your stories about a date or a loved one. You could send
an e-card to them, so it was very viral. You were entered
into a sweepstakes. There were some other components
to it. It was very different than, obviously, monetizing
e-cards on the Web like we are.
Similarly, they just actually launched one for – just in
the past for – Mother’s Day called Toast a Mom, which
is very similar. They actually monetized it by some ecommerce and some other things. Again, it’s really more
of a brand focus.
We also are seeing competition from some charity
sites. You see USO is actually up there. They’re actually
– they’re really monetizing e-cards for the purposes of
getting donations. USO is doing it, Worldwide Wildlife
Fund, and a number of other folks are actually using
e-cards as a hook to get people to the site. They can
send an e-card that’s really around a charitable cause
that means something to a consumer, and then try to
get them to make a donation. Again, you can kind of
imagine if a donation is 70 or 80 bucks, that’s a little
bit different than somebody like us selling e-cards for
$13.99 per year.
This is a typical – and I’m going to dovetail kind of into
our Valentine’s Day case study – but this is a typical
e-cards results page. MyFunCards is actually up here as
number one. This is us for American Greetings, and then
we have Blue Mountain down here. We’re actually more
aggressive, typically, for American Greetings, primarily
because American Greetings typically underperforms
versus Blue Mountain in terms of organic. Blue Mountain
tends to have better organic results, so we are more
aggressive in the paid-search side for AG. Then over
here you get a lot of the traditional competitors I
mentioned, Rattlebox, VidiGreet, and so forth.
This is actually a Valentine’s Day – or this is actually an
e-card results page during Valentine’s Day – and this is
what, most of you probably know, is search arbitrage.
One affiliate of MyFunCards was actually – well, not
one. I should say five were actually bidding illegally,
in fact, on Google, going after – really sending traffic
to the exact same landing page. Really, six of the nine
paid links – you’ve got these two up here, and I believe
there’s seven here. Six of those lined links were really
going to the same landing page.
What’s scary about that for us is we do so much of our
business in a concentrated period of time that even
though we’ll contact Google, and they’re fairly good
about turning that around, there might be three or four
key peak hours in which we just lose out because so
many of these other sites are directing elsewhere. That’s,
again, something that we deal with, and I’m sure many
of you probably deal with as well with affiliates of some
of your competitors.
things that really factors into it, as most of you, or
as many of you probably know: Your Google results are
really driven by not only your CPCs, but also … your
landing page quality. So, looking at whether or not you
have relevant keywords that you’re bidding on actually
on the landing page itself.
I mentioned the non-traditional competitors. This is
actually – I think this is just for generic e-cards. If you
just type in e-cards during Valentine’s Day, this is what
you would see at this point in time. You can see here
is Mrs. Printables is actually another similar retail CPG.
Here we’ve got – this is actually somebody selling hair
extensions, a totally different business. Again, they’re
just looking at e-cards as an opportunity for traffic
during this time period. It really changes the game,
or changes the rules of the game during these peak
periods.
We’ve actually become a lot more advanced in terms
of making sure that certain key phrases appear in
those landing pages, and also making sure that we
have the breadth of landing pages available. We’re not
just sending everybody to a one-size-fits-all kind of
approach. I highly encourage – we’ve seen very tangible
results in doing that approach. I highly encourage many
of you to kind of look at the different keyword buckets
that you have, and look at the landing pages you’re
sending them to, and make sure that those pages are,
in fact, relevant to what you’re bidding on.
I’m painting this horrible picture, all these terrible
things are going on. Why are we doing this? Why are we
even bothering? Why are we doing this? As far as what
we did for Valentine’s Day, we did a number of different
tactics. One, we deployed what we called a very sort
of selective aggression approach. What I mean by that
is you really identified which buckets were producing
the most profit for us. We stopped looking at it on a
what had the best clickthrough, or what was the best
conversion, and really looked at where were we making
the most of our dollars, and where were we getting most
of our subscribers from in terms of general keywords.
Utilize real-time reporting. I mentioned that we do –
Valentine’s Day, for example, is really a two-day holiday
for us. The day before, and really up until about – it’s not
even two full days. It’s actually up until – about 2 or 3
on actual Valentine’s Day is really the peak time period.
We were more or less tied to the hip with our search
agency during that time period. We had them working
until 3 or 4 in the morning. People on our staff were
checking from home, monitoring the bids, monitoring
performance. During those key time periods, if you’re in
a business like ours, you need to be relentless in that
approach.
Really, we decided that while we would still play in a
lot of the other areas, we really were not going to bid
aggressively. We kind of picked our spots. Certain buckets,
certain keywords, we really let our competitors have at
it, and really decided that there were certain phrases
that we were really just going to be very aggressive
about. We were bidding hour by hour. If somebody was
going above us, we were bidding aggressively to get
back in that spot. There were certain terms that we just
decided we weren’t going to lose on.
Also, utilize keyword level bidding – again getting
very specific, not just bidding generically on certain
keyword buckets, but getting to that keyword level but
also looking at the long-tail keyword list. I mentioned
before, we were sort of waving the white flag on certain
search terms and saying, “We’re not going to be number
one or number two.” But really looking at, focusing on
getting as much coverage as we could at the expense
of really going after terms that just weren’t profitable
for us.
We reduced our broad match keyword. Most of you know
from Google, you have the ability to sort of restrict the
keywords that you’re purchasing. Again, with all these
other things going on we really had to do away with a
lot of the broad match stuff that was happening, and
really kind of zero in on the keywords that made the
most sense to us.
So what happened, right? As far as what we saw – and
this is actually aggregate results for February ’08 versus
February ’07. You can see that impressions actually shot
up 119%. We were a little bit surprised by this. A lot
of this – or at least some of this – is obviously driven
by the – it might be the – we suspect there were more
eyeballs, obviously, coming to the category.
We focused on landing page quality.… We have our
champion paid search landing pages, and then we are
constantly testing challengers against that. One of the
But, a big reason impressions went up by that amount is,
again, we focused more on coverage and not necessarily
on the ranking on certain terms. On other terms, we
| 123
may have been more aggressive and really felt like we
had to have that one or two top spot. We were actually
OK with being four, five, or six, or even lower in some
cases. So, we were able to get impressions on a lot more
terms, and really kind of focused on the ones that were
more important to our business.
Clicks, obviously a function of impressions, went up by
23%. Clickthrough rate was very significantly down. The
reason I shaded that in yellow was we kind of saw that
as a good and a bad thing. We weren’t getting as many
clicks, but a lot of those clicks were coming previously on
terms that really weren’t all that profitable for us. They
may have been terms that looked very, very attractive
from a clickthrough rate, or a conversion perspective,
but in reality really weren’t driving the business for
us. So we scaled back those efforts and really kind of
focused on what made sense.
CPC: I shared with you earlier what we had been seeing
on a year-over-year basis in our business. CPCs went up
by 11%. Obviously, any increase in cost is not good, but
considering the space that we’re in and considering sort
of the hand that we’ve been dealt with our competitive
situation, we’re actually very happy with only getting
out the door with an 11% increase.
Subscriptions, obviously that’s the money metric. It was
actually up 26%, so we were very encouraged and very
happy with that number. We did so by only increasing
our CPA by 4%. Again, we were able to really kind of
optimize and focus those campaigns appropriately.
As far as some other recommendations – that was
Valentine’s Day. In terms of other recommendations that
I would mention, some of these things we’ve been doing
for a while. Some of them are a little bit more recent.
On a more tactical level, I would say definitely look
at the second tier engines. Ask.com, MIVA, LookSmart,
especially some of the smaller guys. They’re hungrier
for the business; there’s less competition. There are
going to be more profitable terms for you. Obviously it’s
not going to generate the kind of traffic that a Google
would, for instance, but it’s a good way to really kind
of extend what you’re already doing via Google, and
Yahoo, and MSN, and wherever else you may be.
Also, utilize international traffic. Obviously you have
to have the ability to monetize that, either through
subscriptions or advertising, but we’ve been able to
look at international markets, which in our business
tend to be less competitive. The CPCs tend to come
down a little bit less in competition for rankings.
124 |
Also switching to standard match, and I talked about
broad match and keyword level bidding. Switching to
a standard match on Yahoo actually did a lot for us in
terms of really being able to target those campaigns.
On a more strategic level, I would say, as I’ve sort of
shared with you, we really have gone from focusing
probably too much on the clickthrough rate and
conversion and really kind of focusing more on the
profit. We actually have all of our financials, or cogs,
loaded into the back end with our search partner, so all
of our metrics we’re able to look at on a profit basis.
We can look at what the variance in price per click
terms, or the actual keyword terms, combine that with
a conversion rate, what they’re actually buying. Is it
an e-card subscription? Is it Create and Print? Is it a
bundle? And really factoring in what the overall profit
is to our business. I would definitely encourage you to
– if you’re not looking at it on those terms – to really
kind of focus it that way.
Similar to – I think I’ve seen this bullet now at least
three or four times, which is very good, and that’s test,
test, test creative. We are very similar in that. I think
a good direct marketer is one that knows that there are
no right or wrong answers, and what works today may
not work tomorrow. We definitely fall in that camp.
Also, I’ll talk about purchase path crediting, and I’ll get
more specific about that in a minute, but really kind
of looking at not only the terms that are closing sales
for you, but really what’s happening further up in the
funnels? What actually originally brings them to your
site? I think TheLadders talked about a TV commercial
– I heard her mention paid search where they were
actually seeing conversions on the paid search side that
were originated from that TV commercial. You should
definitely look at your media and paid search in that
way.
I’ve been kind of preaching focusing on profit here. This
is actually a creative test that we ran a little while ago,
and you probably can’t read the creatives very well, and
it’s probably just as well. It’s not really all that relevant
to the actual point I’m trying to make. It’s roughly:
Version A was actually a dynamically served headline.
Version B was the headline “Free E-cards,” and Version
C was using our brand term, American Greetings.
We were trying to test which of those headlines,
combined with copy, would actually work the best. This
is just to sort of illustrate for you that really success
depends on what metric you’re looking at. You can see
that Version A, kind of across the board, was number
two. It was actually number three in total orders that
it generated. Between B and C, it would be easy to
look at some of the front end metrics like if you look
at conversion rate, Version C actually outperformed the
other two on this test.
The deeper we get into this, and the more we look at
our metrics, the more we’re starting to believe that
really we should focus on giving 100% of that credit
to birthday e-cards – the actual search term that first
drove them to our site itself, especially when a branded
search term is involved.
If you look at what’s really important, which is profit,
it actually was Version B, and Version C was, in fact,
number three. If you were making your decisions based
strictly on a conversion rate in this standpoint, you
would actually be mistaken and credit the wrong ad.
Again, just kind of looking at that big picture and really
trying to focus on what it is that’s most important to
your business.
As far as the interaction, I think I mentioned, or it was
mentioned in the description, I was going to talk a little
bit about how SEO interacts with our paid search efforts.
I mentioned earlier on paid search we tend to be more
aggressive on brands that are – I mentioned American
Greetings tends to underperform Blue Mountain for us
in terms of SEO, organic traffic.
I mentioned purchase path before. Forgive me, I’m
actually a big sports fan, so just allow me one bad sports
analogy. It’s a little bit like in baseball where you have
the closer and the starting pitcher, where the starting
pitcher goes at least six, seven innings, maybe more.
Then you have a closer that comes in in the ninth, and
he may put three guys on base, might give up a couple
of runs. Regardless, if he comes out of it with a win, or
at least having the lead, he still gets a save. Who really
worked the hardest in that situation?
Purchase path crediting is kind of a similar story. In
this particular case, we’re assuming that somebody
came in on birthday e-cards, which for us is one of
our more popular search phrases that we – one of our
more popular categories – and then actually closed by
searching American Greetings. The first time, on the
first visit, they searched for birthday e-cards. They
most likely browsed our collection, maybe just didn’t
decide to buy, or maybe they sent a free card, or maybe
they just got busy and did something else. When they
came back to our site, we’ll see them actually type in
American Greetings in the search engines. They may not
remember exactly what the site URL was, or maybe they
sort of know but they were just not sure, so they just
type it into Google, or Yahoo, or whatever. We often see
those branded search terms coming at the very end of
a sale.
What we’ve decided to do – a year and a half ago we
would have given American Greetings 100% of the credit.
We told that closer, “You’re the reason we got that sale.”
Starting last year, actually early this year, we decided
to give 50% credit to each of those. We’re recognizing
that they were closed on American Greetings, but that
birthday e-cards is what actually drove them to the site
in the first place.
What we decided to do on pay per click is we’re actually
less aggressive – or I should say more aggressive – on
terms in which we don’t have that SEO placement.
That’s regardless if it’s Blue Mountain or if it’s American
Greetings. We actually take into account what the paid
search efforts are, or what the SEO efforts are, before we
decide how aggressive we’re going to be on paid search.
Those two are very much tied together for us in terms
of our efforts.
We are pursuing multi-level SEO strategy, and I think
we’re going to hear about SEO more tomorrow. SEO is
– on site is really just a piece of it. A much bigger piece
is what happens off site in terms of back links, and
traffic, and links pointing to your site. We’re looking at
on-site blogs which actually have been, for the most
part, very successful for us. We’re also looking at some
off-site stuff, experimenting with doing off-site blogs
that might be more topic or community-based. We’re
in the early stages of sort of building out an actual
financial model to see if those efforts are going to bear
fruit.
I mentioned blogs. They’re successful at driving – I
think we’ve been able to prove that they can drive some
back links for us. But in terms of really monetizing that,
again, we’re in the very early stages of being able to
measure that.
We have a very distinct landing page strategy for paid
search versus SEO. Pay per click tends to convert for us
definitely at a higher rate than our SEO efforts, which
intuitively makes sense. People who are clicking on the
paid search results, tend to be a little bit more focused
on what they’re looking for, whereas on the SEO side
they might be a little bit more likely to browse, or look
around.
I’ll leave you with – this is actually a couple of landing
| 125
page examples. In the bottom left here, this is actually
an SEO page. This is actually a paid search page. These
are both for the same holiday, so this is Mother’s Day,
just to kind of give you the contrast.
What they share is they have – we talked a lot about
buttons, I think, today, but they both share a very
similar call to action. Obviously on the SEO page, you
have a lot more content, a lot more sort of content-rich
keywords that we’re obviously focusing on here. It’s a
little different focus for us. What we’re doing, since we
are seeing a pretty significant lift in conversion on this
page, on the paid search page, we’re actually looking
to take elements of this and incorporate that into SEO
without really sacrificing the SEO value of the page.
We’re leaving the copy and the text as is.
We’re really trying to look at taking this box, this free,
printable Mother’s Day cards, and really bringing that
into a similar place on the actual SEO page, trying to
incorporate what we’re learning on the paid search side,
and looking at it for SEO. As many of you know, paid
search is really a – I don’t want to say it’s a short term
game, but it’s something you can get into very quickly.
Whereas a good SEO strategy is – it takes months and
months to really see the fruit of that labor. So we can
take learnings from the paid search side that we get very
quickly and apply that to a longer term SEO strategy.
With that I’ll actually, before I take questions, I did want
to recognize, again, our search agency, Clear Sailing.
They actually helped me a lot with the building of this
presentation, and they’re obviously a good partner for
us in terms of some of the things we’ve been able to
accomplish.
MODERATOR: That’s great … All right, Jon, we’ve got
one question up there.
Participant: Hi. Sorry if I missed this, but those final
two pages that you showed us, were they optimized
for the same exact term? Was it Mother’s Day e-cards?
In other words, would your goal be to have the SEO
page ranking number one on the same term that you’re
driving the paid traffic to?
Jon Eggleton: Yeah. In fact they – well, the SEO page
was optimized for Mother’s Day e-cards first, and there’s
some other terms we’re really going after on the SEO
side. In many cases, we do expect that they might see
both of those pages. They would see the lower left, the
SEO page, and the organic results. They would see the
paid search landing page and the paid results. We do
expect those to appear together in some cases.
126 |
Participant: Have you seen any increase in your
clickthrough rate when you get top organic rankings
and you’re running advertising on the same term?
Jon Eggleton: For certain key terms – and Mother’s
Day e-cards would actually be a good example for us
where we do, in fact, go after both of those. We want
to be on the page twice, essentially. As I mentioned, we
tend to – if it’s not a core term for us, like Mother’s Day
e-cards is obviously a pretty core holiday. Valentine’s
Day, birthday cards, birthday e-cards, those types of
holidays, we’ll be very aggressive in trying to get both
placements.
It’s more of the secondary holidays, where if we happen
to have very good organic placement for Administrative
Professionals Day e-cards, or just general terms related
to admin day, we may be, we likely will be less aggressive
on paid search because we have that organic placement.
So it’s not: We’re better off placing our bets elsewhere
than going after some of those secondary holidays if we
have a strong organic placement.
Participant: When you’re dealing with some of those
more sort of long-tail search terms, how do you balance
between absolute profits versus average profits? At what
point is it worth sacrificing a little more profit if it’s just
to get the volume up a little bit more?
Jon Eggleton: Yeah, well again, I would say it really
depends on what the holiday is. For Valentine’s Day, we’ll
be very aggressive and going after the core terms related
to that. Again, like Valentine’s e-cards, or Valentine’s
Day print, key words that are related to that.
I guess for us, we look at, obviously, the traffic based
on a certain term when we make that decision as far
as how aggressive we’re going to be. I don’t know that
there’s really a – there’s not really a formula, I guess,
if that’s part of your question. There’s not a formula
that we decide where OK, if it doesn’t meet this criteria
we’re just not going to be aggressive on it. It just really
depends on what that particular holiday is and what our
overall goals might be.
If we know that, for example, if Easter, which is another
big holiday for us, if we’re looking as if we’re going
to be a little bit short in our plan overall, we might
be more aggressive on some of those terms based on
what’s happening overall in the business, in the macro
business.
Participant: I have two quick questions. The first is
within your unbranded paid search. You’re competing
against people running free e-cards, so did you do any
copy testing? That’s a tough thing to compete against.
Jon Eggleton: Yeah, actually, in fact, I don’t think
I did it justice in pointing it out, but I believe the
winning creative in the example I showed was ‘Free ecards,” which is very strange for us. We’ve tested that,
and I think – and I’ve seen this in other businesses,
or at least heard in other businesses, where the word
free is – you can put anything against it and it’s always
going to win.
You would think in our case, given the fact that we do
compete in a space where there’s a lot of free options,
that that would actually hurt us. Free consistently tends
to perform pretty well for us in a lot of our paid search
efforts. We try to incorporate that, and actually we
could say free e-cards, by the way – I should point this
out – because we have a 30-day free trial. We do offer
the opportunity to use our product for free for 30 days
initially. We incorporate that into the messaging that
we use for the actual subscription.
Participant: And then my second question is within
your branded pay searches, have you seen your costs
climbing? Have you done any position testing because
they’re your branded terms? Have you done that? Is your
volume holding up as you’re moving positions?
Jon Eggleton: Yeah, and this kind of relates to the
previous gentleman’s question. On branded terms, we
sort of feel like that’s kind of our turf. So where we
tend to be, we haven’t really done position testing on
those terms. Where we have done position testing is on
some other terms where we see what the value of being
number two versus number four or something on that
page.
But, on the branded side, we try to be fairly aggressive in
going after those particular terms. I should say relative
to the cost of general e-card and Create and Print terms,
our branded terms are still a lot less expensive. It’s just
relatively speaking, they’ve become a lot more so over
the last year and a half.
Participant: On your slide for Valentine’s Day 2008,
you indicated that your clicks had gone down, but
your clickthrough rate had gone up. For search engine
marketing or pay-per-click, obviously your click rate
is clicking on the ad, but how you are defining the
clickthrough rate? What’s the follow-on action?
Jon Eggleton: Actually the clickthrough rate was going
down. The overall clicks had gone up. The impressions
had gone way up. Clicks went up by – I think it was
23%. impressions were 118. Clickthrough rate actually
went down by about 44%.
Participant: How are you defining clickthrough rate?
When you click on the ad, that’s a click, then what’s the
clickthrough rate?
Jon Eggleton: Oh, it’s just strictly a matter of clicks
to Impressions. It’s really – that’s a front-end Google
metric.
Participant: Oh, I see. You’re just doing the division is
what you’re saying.
Jon Eggleton: Right. Clicks to impressions.
Participant: With all the landing pages that you have
up, do you take any steps to block some of them from
the search box? I’ve heard that if you have very similar
content, Google can punish you in certain ways if you
have too many A/B tests or landing page examples up.
Jon Eggleton: Yeah, that’s a great question. I don’t
know off-hand which pages we’ve done that to. We
have actually blocked some of our landing pages in the
past. The content that’s on our landing pages tends
to be different enough from any of the other pages
on AmericanGreetings.com. You saw the SEO example
where those pages couldn’t really be any more different,
other than the fact they’re both Mother’s Day focused.
We haven’t run into too much trouble with that just
because there isn’t a similar page.
Our paid search landing pages are sort of a condensed
version of our order path – the first page of the actual
order path itself. We have actually blocked, in certain
cases, landing pages from the search engines, but
overall we haven’t really seen a huge issue with that
personally.
Participant: Can you comment on the – where you
manage an outside agency with your internal efforts,
and how you optimize that with your paid search
strategy?
Jon Eggleton: Sure. As I mentioned – and actually I
don’t remember if I did bring it up or not. We’ve been
on a universe of up to 70,000 terms, obviously not all
at the same time. That requires having a lot of sort of
legs on the ground, so to speak. So, from a strategy
perspective, we really drive a lot of the strategy ourselves
internally. But we also rely on our partner to really give
us a lot of the insights that are kind of happening more
industry- wide.
For example, using Valentine’s Day. We had – I mentioned
| 127
our agency was looking until the wee hours of the
morning and checking bids. We had people on site that
were also doing the same things, but they may not be
going quite to that keyword level. Maybe just looking
at overall buckets. They might look for a certain bucket
of terms, what might be happening on a macro level,
but really a lot of the detail was being handled by the
agency itself.
I also think from an analytic perspective, we rely on
them for the technology. For us, we went through the
integration on the front end to really get involved and
have them be able to have all of our information, so
they have all of our cogs, they have basically everything
they would need to tell us what our profit margins are.
In that case, we can sort of forget about the technology
piece and really let them manage that. From a technology
and, really, a reporting standpoint, it really allows us to
focus on what’s important and which of those buckets
we want to optimize.
MODERATOE: All right. Thanks a lot, Jon.
128 |
The Economy of Paid Search: When
and How to Use It For Maximum Effect
Jon Eggleton, Vice President, Marketing
AG Interactive
available from::
Who Are We?

About AG Interactive



Hosts over 3.7 million eCard subscribers
One of the largest network properties on
the web
About Me

Responsible for acquisition, conversion, &
retention marketing for AG.com division
available from::
2
| 129
What Are Our Paid Search
Challenges?

Maintaining growth is difficult




available from::
eCards is a mature category growing
slower than the web overall
Relatively low LTV
Increased competition from multiple
sources, especially during holidays
Net result is higher CPC’s, lower
CTR’s, need for optimization
3
Greetings Category: Paid Search
Assessment
available from::
Q4 ’07
Category Leaders – Avg.
CPC
$.23
CTR
7.4%
Impressions
18.3M
Clicks
1.4M
Spend
$309.6K
4
130 |
AG: CTR’s Dropping, CPC’s Rising
AG Buckets
Branded
eCards
Create & Print
CTR
Avg. CPC
CTR
Avg. CPC
CTR
Avg. CPC
’08 vs. ’07*
-9%
267%
-31%
33%
-19%
15%
available from::
* AG results from Google: November 2007 through January 2008
5
YOY – AG Click Through Rate
Click Thru Rate
available from::
6
| 131
Traditional Competition Increasing…
available from::
7
…As Is Non-Traditional
available from::
8
132 |
Typical eCards Results Page
available from::
9
Valentine’s Day: Search Arbitrage
available from::
10
| 133
Valentine’s Day: Non-Traditional
Competitors
available from::
11
Valentine’s Day - What Did We Do?






Deployed ‘selective’ aggression
Reduced broad-match keywords
Focused on landing page quality
Utilized real-time reporting
Utilized keyword-level bidding
Expanded long-tail keyword lists
available from::
12
134 |
Valentine’s Day YOY Results
available from::
Impressions
2008 YOY %*
119%
Clicks
23%
CTR
-44%
CPC
11%
Subscriptions
26%
Net CPA
3.9%
* Based on February ’08 (full-month) compared to February ‘07
13
Other Recommendations

Tactical




Strategic


available from::
Launch with 2nd tier engines; Ask.com,
Miva, LookSmart, etc.
Utilize International traffic
Switch to ‘Standard’ match on Yahoo!

Focus on profit, not CTR!
Test, test, test creative
Purchase Path crediting
14
| 135
Focus on Profit, not CTR…
Performance By Creative Version
available from::
A
B
C
CTR
#2
#1
#3
Orders
#3
#1
#2
Conv. Rate
#2
#3
#1
Profit/Click
#2
#3
#1
Profit
#2
#1
#3
…and keep testing!
15
Which Keyword Do I Credit?
Birthday eCards >> American Greetings

Purchase Path Crediting



available from::
In 2007, this path would have given
“American Greetings” 100% of the credit
In 2008, this path gives “birthday eCards”
and “American Greetings” each 50% credit
We are exploring potentially excluding
branded terms at the end of a path
16
136 |
Paid Search, SEO & Social Media




available from::
We are more aggressive in PPC on terms
where we don’t have top organic placement
Pursuing multi-level SEO strategy both On
and Off-Site
Blogs have been successful at driving backlink value, but difficult to monetize
We have a very distinct landing page strategy
for PPC vs. SEO

PPC converts at a higher rate
17
Paid Search vs. SEO Landing Page
Paid Search
Call To
Action
available from::
SEO
18
| 137
Credits/Thank You
Thank you ClearSaleing!
available from::
Jon Eggleton
AG Interactive
216-889-5362
[email protected]
19
138 |
A/B Testing Case Study
Session Description
Encyclopaedia Britannica has been prolific in their
use of A/B testing over the past 18 months. This
information-packed session calls upon the findings of a
number of tests (both winners and losers) designed to
enable marketers to revisit all elements of a campaign
including strategy, design, execution, and of course,
results.
About the Presenter
Joe Miller
Senior VP Consumer Sales
& Marketing, Encyclopaedia
Britannica Inc.
Newly appointed as senior vice president of consumer
sales & marketing for Encyclopaedia Britannica, Joe
Miller brings more than 17 years of direct response,
online marketing and product management experience
into his new role. After marketing stints at SBC
Ameritech and GE, Miller joined Britannica in April
2006 and since then, has spearheaded the company’s
PRESENTATION
MODERATOR: Our next speaker is actually – I want to
take a quick poll of the audience. How many of you
use a free trial as part of your subscription marketing
efforts? OK, good. Lots and lots.
Obviously, then, free trials are, depending on your brand
recognition, how well known you are, they’re a huge
part of getting people to actually convert and become
paid subscribers. You might get some people who just
know they want to subscribe. They don’t need the free
trial. For a lot of you, the big case is they’re going to
want to take that trial before they actually convert.
Our next presenter is going to talk a little bit about
what they’ve done with their free trial conversion path.
Sort of that key question is: Are we making it as easy
as possible to get people into that free trial funnel
because once we get them in there, maybe we have a
much better chance of converting them. What are we
doing to optimize that registration path for the free
trial and get these people sort of into our marketing
world?
new customer acquisition, business development,
SEO, and e-commerce activities. Currently residing in
Chicago Illinois, Miller holds a BS in Communications
and a MS in Advertising from the University of Illinois
at Urbana-Champaign.
About Encyclopaedia Britannica
Encyclopaedia Britannica, Inc. (www.britannica.com)
is a leader in reference and education publishing
whose products can be found in many media from
the Internet to cell phones to books. A pioneer
in electronic publishing since the early 1980s,
the company also still publishes the 32-volume
Encyclopaedia Britannica, along with such services
such as Britannica Online School Edition and
new printed products such as Britannica Student
Encyclopedia, which are available online at http://
store.britannica.com. Britannica’s editorial operation
is overseen by some of the world’s most distinguished
scholars, making Britannica’s online product second to
none. Britannica’s online content is available in its
entirety to both institutions and consumers via paid
subscription. The company makes its headquarters in
Chicago.
Let’s bring up Joe Miller from Encyclopedia Britannica,
and he can tell us what they’ve been doing.
Joe Miller: Good afternoon. I will try to keep this
very swift. My job is basically to give you folks some
understandings of how we do A/B testing, hopefully
provide some actionable type examples that you can
take to any business model.
One of the things at Britannica, and it’s funny, and it’s
not necessarily here because we’re all selling content.
Let’s say I’m back at home at soccer games and stuff
like that, and folks are like, “Britannica? What are you
guys - you still selling books?” They’re actually hugely
expensive, so we have tried to do other things. They are
sometimes surprised that we’re actually online.
The thing I wanted to try to show you real quick is
what our business model is today as we’ve tried to get
away from just selling big, hugely expensive books.
Basically we have, as most of you all have, a bunch
of content databases; millions and millions of pages
of practically every topic that’s out there. We basically
do still sell books; we go online and then we put that
| 139
into software. Those are really basically our three main
products today.
We do sell the bulk of our content through to
institutions. So this is the B-to-B play. Libraries,
universities, elementary schools. This is a very lucrative
business for us, and it continues to be. Then the other
part is consumers, or our B-to-C. Basically, I would
say probably 70-80% of our revenues is through that
institutional channel. So we’re the poor step-child. Is
that an accurate phrase?
What we also do is we have two – there is a free-site
experience and there’s a premium experience. If you
were to purchase a subscription to Britannica Online,
there’s basically five databases that you get access
to. They vary from different writing levels. We’ve got
kindergarten databases all the way up to collegiate
levels. Today, on the consumer side, we have two
versions. We have just a regular general reference one,
and then we’ve also created a student edition, which
takes some of those lower level databases, and then
we also have the integrated learning activities and
homework help in some of those areas.
Actually, both of those editions are $70 a year, and I’ll
show you, we only do annuals only today. Right now
we’re at a seven-day free trial. Basically, our visitors are
mainly students, info gatherers, learners, however the
buyers end up being primarily parents with their kids in
that middle-age type area.
What I thought was kind of interesting, too, to kind
of give you a little bit of evolution because, and this
will be my Web 1.0 slide, and I think we’re actually
still in Web 1.0 for us. This is our view in 1998. This
is right when, as we all remember, the golden age, so
to speak, of the Internet and everything was all great.
This is fairly – you had the big call to action up right up
there. Why subscribe? Seven-day free trial. It’s funny.
We haven’t changed the seven day in what? Ten years?
140 |
there.
This is actually where we are today, even though we are
now changing this again to hopefully a fresher look.
This is also an interesting thing. Talking about scope
treatment and everybody just jamming stuff into a
home page: This is how our home page has grown. Now
you’ve got a skyscraper on that right, so hopefully we
can change that, too.
The first thing I want to do is – about two and a half
years ago is when Britannica really started to get into
A/B testing. At that time, we wanted to kind of start
off, and we looked at our metrics, because we get
millions and millions of people coming into the site. Yet
we had very few people hit the call to actions and even
a fraction of those folks signing up for free trials.
One of the first places we start was really taking
the registration path and really doing an overhaul.
Unfortunately, that’s really not A/B testing, as you’ll
see. What we really wanted to kind of say is what’s a
new path, what it should be. At that time, we brought in
MarketingExperiments to basically create a registration
engine. For us, we couldn’t afford, technology-wise, to
have our developer spend three years trying to develop
a registration path. We went outside to kind of build
this and try to do our testing in an outside lab, so to
speak, but one that mirrored our own registration.
Our original path was basically a five-page offer page.
We gave you options …We had information fields where
you clicked, and all that basic stuff. A thank you page
that went out or a confirmation, and then a thank
you page. So a pretty basic registration, but it took
forever to get through this stuff. What we wanted to do,
though, is try to freshen this up.
This is how it – and you can’t necessarily see the year,
but this is kind of flashed forward to about 2000 or
1999. One of the things back then is we just started
changing things all the time. It was just slap up creative
here and there. I joined the company two years ago, so
I can’t take credit for any of this creative.
Now you can see, this basically was the landing page
where “Why try?” and all those benefits, more benefits
on the right, a bunch of pictures with happy people,
and good stuff. What we want to do, though, is try to
make this more simple. When we came in, we wanted to
try to go right in to say, “What is someone getting with
that headline?” This was something where in the old
one, it was really just “Why try?” And it really went into
asking questions, which isn’t bad, but just too much.
We also, then, started to go into more free copy.
Then we went to – and this is actually somewhere –
Britannica actually split off and was going to do an IPO
and all this great stuff, and it was going to be separate
from Encyclopedia Britannica, so they started doing
some branding here and more stuff. I’ll get through
The pages before were like a purchase. It was stuff where
we went in and we were pitching a free trial. Then it
was, “Buy this. Here’s your subscription.” All this. You’ll
see as we go through this, there was tons of things like
that. We started to bullet and bold. We didn’t do that
before. Larger images. Now obviously, tons of variables
were changing, so this is not – I’ll get to the A/B part
pretty soon.
then a 53% lift in total revenue, which also told me
that we had more refunds with the new path, but still it
was, net/net, a very positive thing.
Leading with free language. I’m going to go through
these pretty quick, because it’s pretty basic. We used
to mention credit cards on other pages, but here we
wanted to just mention it here. Establish the value was
something we didn’t do before. We compared those to
savings on a – and actually, I don’t know if you knew
this, but it costs $1,400 to still buy an encyclopedia.
I love people that buy that, but it’s not happening a
lot.
Now we have this baseline, and now we’ve got this new
registration path, so now it’s time to start tweaking
some things. What we ended up trying to do is – I heard
this earlier, talking about monthly, annual bundles, and
we actually had three different bundles that were going
on.
We eliminated some unnecessary fields, and then we
began with more a proactive and beneficial button. We
actually went with orange. We haven’t tested red or
gray, but that may be something. Then just risk-free
points. There’s a lot where we tried to simple up. Then
a testimonial.
We also did the same thing on the subscription options
page. Here it was just credit cards on the left. It’s all
about purchase. It was pick an option. Here’s your price.
Remember, I’m talking free trials. I need people to sign
up for a free trial. Now we get in here and say, “Hey, here’s
a case after your free trial if you want to go forward,
here’s some options you can set up.” We changed it
from – before it said subscription, registration, the free
trial. Credit cards, we took those out. We spelled out
some savings in there, and then we had an incentive
call to action. So that’s some of the stuff.
It’s funny. I was putting these slides together, and I was
all ambitious, getting all these bullets, and you’ll see
these start to not get so ambitious. Anyway, the paid
look is still going through. Now we get into – it’s still
disarming copy. It’s free trial. We just want you to set
up your access. No big deal. Once again, free. You get
the idea. It really was starting to just get them through
where it’s not a paid process. Confirmation page. Here
was order confirmation. They still think it’s a purchase
in this situation. The credit card images are still there.
Hey, free trial confirmation. Go ahead and get going.
What we ended up doing is – what we found in this,
once we revamped this whole thing, and then we got
us a baseline, we had 125% more free trials per unique
visitor. So all the people that hit that page, how many
free trials did we get? We more than doubled that. Sixtyfive % of those became paid subs per unique visitor.
Now you’ll notice that went down, so that meant we
widened that funnel quite a bit. My conversion rate off
that went down, but I still had 65% more paid subs and
The way this process worked, as you recall, we had this
in a new path, landing page, offer page, gave you some
options, and then you could set up your account at that
point. What we ended up doing is just trusting annualonly registration. We ended up taking out this middle
page. It really gets into basically a paradox of choice.
What we found out is that people go through and they
see this. If we didn’t give them a choice, and we went
right through here, and in this particular test we ended
up getting 117% lift in free trials. Once again, more
than doubling what we were getting. 61% lift in paid
subs. Same type of behavior you saw before. I think in
this one it turned out we had annuals where it was more
of a commitment. You’re taking this free trial, you’re in
this seven-day period. If you’re going to get dinged for
$70, you may think about it. So we had more people
canceling there, and net/net, 130% lift in revenue.
Now we just do annual only. We did – and this was about
a year ago – we retested this to try to see – because
one of the things we had found in a test is that we
underestimated the monthlies’ retention. I think before
we were getting about five months location life, and
some of them we were getting eight to nine months.
When we retested, it actually reconfirmed that annual
was still better.
This is another one which – and you hear in new customer
acquisition especially, and we’ve had different partners
come to us and say, “Your path is too long. You need to
shorten it up. If you could go to a one-page order form,
we’ll give you half the rate.” So we wanted to try to test
that first before just developing it.
This was a situation where we actually tested taking
that initial page – on the right, actually, is the control.
So we took the first page, and we actually ask for a
credit card, by the way, up front. You can’t see the
fields. These are hard free trials. Then we combined that
with the account information page. We put those all on
one page with the idea that, hey, this is the one-pager.
They fill this out, set up your access and you go. We say
thank you and you’re done. Review it and you’re done.
| 141
This one surprised me. We got 3% less free trials on
this one. We had slightly less paid subs. This one – the
only difference here on these, we had the two columns,
so you’ve got – this was just one column, this is two
columns, so 4% and 6%, so that was even worse. This
one was not one that did well for us at all. It did
surprise me. I wonder though, because we tested this
through organic traffic, so this is a situation where if
someone came through a paid search link, or just came
to Britannica, we probably will resurrect that for new
customer acquisitions. If we’re doing it like affiliate
marketing, some advertising out there, or even paid
search would be an area to try to try it, to see if that
would work.
I’m possibly going too fast. I’m just blasting out a bunch
of our different tests, and hopefully it makes sense. This
was one – our article page is really our landing page,
if you think about it. Most of the time for us someone
comes into our site, and really our business model is
they come in and you see this abstract or this gist, and
then we say, “Hey, if you want to read the rest of this
you’ve got to click through, and then you’re going to
have to take the free trial.”
The challenge for us is really now to start to play around
– this used to be green. We went from green and actually
give away a thing to orange. I tell you, we tested, and
this was before my time. They tested everything to get
rid of this green arrow, because it just was ungodly
ugly, but it just kept working, until, thank goodness,
the orange came out. This was the only thing you’re
changing here. We took – we added this, ‘activate your
free trial,” and then put it in this ‘activate my free trial’
call to action. It was a little more promotional.
In this one here, we finally found something that
went, so we went with orange here. It had 30% less
clicks, and I think the reality here is we got people
to get clickthrough, you know, get the full article, and
they – but here it’s like activate a free trial, which is
probably more of a commitment. We got more qualified
clickthroughs, because we had 13% more free trials per
unique visitor, and then 61% more free trials per click.
So that’s after they click, and then we measured the
click to the free trial rate. Bottom line, 24% more paid
subs per unique visitor. So, hence, the orange button.
Now this is something, as we go forward and, actually,
as we move forward and we do some more stuff with
multi-variable testing and such, this will be areas where
we can start playing around with it too. This is another
test which was interesting in that we tried to – this is
142 |
the test. If you remember that other page before, and
I’m trying to see, we added an email address capture
on the front here. The idea here was before people click
through, could we get their email address and then be
able to do some recovery after the fact?
The idea was, if you want to read this, activate your
free trial, put in your email address, and then when
they came over, it was already … pre-populated with
their email address, one less thing for them to actually
fill out.
After they abandon, say someone went through there
and they abandon the process, we would send an email
an hour afterwards promoting all the great things. Come
back, please, type of thing. Then 24 hours afterwards to
see what happened as well. Try to recover these folks if
we had their email address.
This one was – we had 53% less – and that was
expected. We thought you see this email, that’s going
to be a barrier and we’re going to lose some folks here.
We did get 11% more free trials per unique visitor, 45
more paid subs per unique visitor. Now this was one
that scared people. They were like, “Oh, you’re going
to collect emails up here.” It was one where we ended
up not doing it, but then we did that one, and then
we tested that orange button and such. That’s why we
ended up with that. That was an example, though, that
it wasn’t quite compelling enough for folks to say, “Hey,
let’s collect all these email addresses.” But then – and I
think we also had some customer service issues.
This is one now, and I didn’t actually talk about it too
much in the beginning, but our consumer business
model has three prongs to it. When someone comes
to our page, I’m trying to push you to buy a premium
subscription. At the same time, and you didn’t see it
on that page, we also have advertising on those pages.
Before you actually click over, we’re selling you ads.
We’ve got Google ads on there, and we also have some
display ads that are out there. We’re trying to actually
generate revenue for advertising.
Our third way is through e-commerce. So we have people
who click through and go to our shop and buy stuff.
What we try to do, then, is measure total revenue per
visitor. It’s not just the premium site, but also looking at
advertising. One of the deals that our ad group brought
to us was MS Live, where what they wanted to do was
run a find more about dog instantly with Live Search.
This was a CPC offer where for every click we were going
to get paid by MSN to send them off. As a premium
service guy, well, I’m going to be funneling off people
that I want to try to click this. This was one of these,
what’s the best total revenue model, that we need to
go to? We did an A/B test here where we ran just the
control and didn’t have this in here with this.
The way this worked out, 20% less clicks per unique
visitor. We had people that were clicking on that link,
whereas they would have not before. We had 14% less
free trials, and 12% less paid subs. Subscription dipped,
however, because of how lucrative that link was, and
we got tons of people clicking through, a 70% gain in
overall revenue. This was one that – you know what,
MSN? As long as you want to keep paying us, we’ll run
that link. That’s been a nice thing for us.
The other thing we are doing here, and I know most of
you folks – I know on the etail side – you see the Hacker
Safe, or the VeriSign type logos, and I’ve always been
curious – and we have never done on our side, is what’s
the value of that? You read the material and they’ll talk
about 40% lift if you have it. Since it costs money it’s
an ROI issue. Do you we really want it and so forth. We
actually, then, wanted to try the test and see what this
would work out to be. The only difference here from the
A/B is you have the control, and we threw in this – the
Hacker Safe part here.
We ran this, and this one took a long time because we
only ran it on a certain part of the site. It’s actually
still – I think there’s a 60% confidence level. Right
now we’re at about a 13.2% lift at free trials just by
having that one little element. This is one where, as
we continue to go through, is it – and I don’t know if
we’ll go and try to play around with placements and
stuff – and sometimes, we used to have two of these
things. We had the thought – I don’t know if that’s how
you pronounce it, but another logo. Does that make a
difference? So this was one that was very positive.
In the future for us – so now we’re moving off this A/B.
If I had a choice, back in that example where we had the
old path, new path type of area. I could have done that
probably much more quickly if I had this. What we’re
looking to do now on our landing page is taking each
of these different areas, and we’re using Offermatica as
our resource here, and really start to throw out different
variables and see if I can optimize this page.
We’ve actually now created a separate registration path,
just to do multi-variable testing to see how this works
out. I don’t have results yet. We’re just launching this
out, but this is where our future goes on this stuff.
Actually we moved through that pretty quick. Anyway,
that’s it. So that was just some of the stuff we’re doing
out there. Any questions?
Participant: I was just going to ask on your competitive
information. Clearly you must compete against a lot of
the free information sites like Wikipedia? How do you
differentiate yourself?
Joe Miller: I’m sorry. Wiki - what? No, actually it’s
funny. We do compete against all those. Google was
probably our biggest competitor in terms of free
information. For us, it’s been more of an SEO story in
that how we’ve done things on the SEO side, which is a
whole other presentation, but trying to get those gisttype pages showing up in search engines.
Actually I joke about Wikipedia, but since they came
on board for us, it’s really been one of the best things
that’s happened for Britannica. Before that, if you think
about online encyclopedias, I mean nobody really talked
about them. It wasn’t – it’s never going to be sexy or
anything like that. But it’s now more relevant where
people were dealing with Wikipedia.
Since they’ve come online – and I’d love to say it’s
been all marketing and all SEO – our traffic over the
last 18 months has gone up three-fold. It really has
been coming from the people looking for information
online from an encyclopedia or from Google and SEO.
Competitively, yeah, I think it’s Google.
People are going to Wikipedia, but I think we’re also
seeing people going from Wikipedia to us to try to verify
it, so to speak. We’ve got the vetted editor content and
all such. You still have to have that balance. The reality
for us for Wikipedia and then Google, you could find
everything out there. You don’t know if it’s right or not,
but it’s out there. There’s stuff that’s going to be great
that we don’t have. It is, for us, all those things. It
helped us, I think, competitively.
From a paid side, I think Encarta used to be fairly strong
in terms of that, but I think Microsoft has somewhat
started to downgrade that. Then there’s other reference
sites like encyclopedia.com and such that were all
playing in that same sandbox trying to get our listings
up. That’s a good question.
Participant: You spoke about the recovery email that
you send out, I believe. Is that correct? One trend in the
industry with these is marketers are using more. They’re
both sending these emails out, sending a short series,
and also continuing to email these people promotional
offers. I had a two-part question for you because I’m
| 143
guessing that you guys are pretty protective of your
brand.
Part one would be how do you ensure that your emails
reflect that they’re a customer service email as opposed
to a promotional email? Secondly, do you do anything
longer term with these people or do you just send out
one or two recovery emails and leave it at that?
Joe Miller: Well, I think – and that’s a good question
because we are very protective and obviously want to
be very compliant with all those rules. That may have
been one of the hesitations with that program. I mean,
we do have an opt-in script that you didn’t see but it
was in there somewhere in terms of how we go through
it. That one was a little bit weird.
For the recovery, it was like, “Hey, you stopped doing
this action. Can we have you come in?” Then we tied
it a second time. We needed to definitely separate
those folks off from a compliance point of view, to say,
“That was it. We’re not going to keep bugging you.” We
really wanted to keep those messages on message. Say,
“Hey, this is the only thing we’re contacting you about.
There’s no other selling point.”
We do today, we are trying to figure out, to take advantage
of abandoners as they go through that funnel. As folks
are dropping off, how do we – not necessarily on the
recovery side, but really going out promotionally – say,
“Hey, why did you stop? Can we give you a different
offer?” Once again, keeping it focused from that point
of view. The same kind of messaging that we do when
someone cancels a subscription, too. I mean, those
emails.
I think, at least to answer the first part, we do try to
keep those separate and really just keep focused on that
certain target audience in that message.
Participant: Have you tried trials with and without
requiring a credit card at the trail point? Can you speak
to the difference that you’ve experienced there?
Joe Miller: I personally haven’t. It’s one of those you
heard it happened back in the day. They did supposedly
test a free trial without the credit card, and they could
never collect on it. They got a lot of free trials and then
nobody really paid. Obviously no one – it was just some
rate where doing it this way, we received more of it.
One of the things we are looking at when you come
into our site is you’re either an unregistered visitor or
you become a free trial. There is no in-between. One of
the things we’re looking at, especially as we launch a
144 |
new site, is can we create this middle tier? As I start to
benchmark different areas - like I was looking - and it’s
not the same industry. But Morningstar, for instance, as
I recall, has an unregistered visitor, a free registered
visitor, and then they’ve got the premium.
So, we are looking at basically a free trial period where
someone can come in and register for free without the
credit card, and have privileges for that certain period
of time and see if there is a way for us to increase our
eligible base, and then try to do some remarketing with
some of the tools and some of the learnings we have.
What we also are trying to do with the new site is get
away from, not get away, but enhance not just selling
content but some of the tools that you use with it. And
can we start to create a model where – and I use the
example of Web hosting, or Flickr, where you have a
basic, where you’re just doing some things very basically
on the site – if you want some of the bells and whistles,
then you’ve got to pay.
So, maybe that basic one is one where you don’t need
a credit card. You can have some limited privileges,
but see a lot of the content. Right now, though, our
experience is that the hard credit card one is the best
bottom line, but I am going to test that again to see
where it’s at.
Participant: You talked about the MSN Live link
earlier. It looked pretty clear to me that that particular
link was actually cannibalizing subscriptions. From your
perspective, is it more important to have subscribers or
that overall revenue that you have coming in?
Joe Miller: You know … I think where we’re going
to is that it’s going to be more important from an
overall revenue, because the future model for us will
end up pushing and pushing more than free content
line. Though we want to have that relationship in terms
of the premium services, I think the reality for us is
going to be the ad model is going to surpass what the
subscription model is going to be, especially if we’re
able to open up the content and create more page use.
The thing that’s keeping us from doing it, and I’ll tell
you, if we could, if we didn’t have that institutional
base that had all those libraries and schools paying
millions of dollars in licenses, we’d be free today for
that. We would be trying to do – you know, if you think
about Wikipedia, for instance, I think the figure is saw
is like three billion page views per month, which is like
187 times what ours is, if you can do the math.
For us, if we were to open up and be able to compete
on that level, and I’m not saying we’d get all the way to
three billion page views, but that’s a lot of money when
you think of CPMs on advertising. I think for us, though,
long term, I could see us being free, but then how do
you – you don’t want to cannibalize those institutional
revenues, and so we’re kind of doing it halfway.
Actually, to give you a little bit of story, from ’98 to 2000,
when I was showing those slides before, we were free
when they were trying to spin off. Those institutional
licenses just plummeted. Britannica, it used to be, I
think, a $3 billion to $4 billion company. So now we’re
a fraction of that. It’s obviously that fear from our
leadership is when do you go to that free side.
Going back to answer your question, for me it’s total
revenue. I think subscribers - and we are growing our
subscriber base surprisingly. We were going down, and
now we’ve pushed it back up a little bit. Part of it is,
I think we were repositioning and doing some of those
optimizations that are making it work. I kind of want
both. We’re going to try to make it work both ways.
Participant: What was the minimum amount of time
that you allowed for each test? Also, what was the
maximum amount of variables that you test at a given
time?
Joe Miller: Well, our basic variable is we’re measuring
free trials. We looked for each leg to have 400 free trials
per leg to get 90% confidence level. That generally
took us about two weeks to achieve that, depending on
where the placement was and if the measure truly was
free trial. That was our basic for that.
The second question, you talk about maximum number
of variables. On the A/B side, we tried to keep it just that
one variable. On multi-variable testing, when we go for
that, we’re going to be looking at – you know, you can
alter, I think, it’s up to those seven different variables
and create recipes. Within each of those variables, you
can have – and I don’t know the exact figure. Somebody
here is probably going to slap me down, but I think
it’s like four to five different creatives that you’re just
routing through to get that best recipe.
On those, we are looking at a lot more. Obviously, the
more variables you have in there, the longer time it
takes to get that optimal. We’ve got to play around with
that since we just started to launch it.
| 145
Joe Miller, Senior VP of
Consumer Sales & Marketing
Encyclopaedia Britannica
available from::
Today’s Business Model
Books
Institutions - B2B
Content Databases
Online
available from::
Consumers - B2C
Software
146 |
2
The Product
Britannica Online Premium
• 5 databases with millions of pages of reference
content
• Vary by writing level, from kindergarten thru collegiate
levels
• Two consumer versions
• Core Edition – most robust
available from::
• Student Edition – fewer articles, edited for
school-level readers + learning activities
• Current Offer - $70/yr with 7-day free trial
• Site Visitors – students, info gatherers, lifelong
learners
• Buyers – parents with kids 7 to 15
3
The Evolution of Britannica.com
available from::
1998
1999
2000
2002
2007
4
| 147
Homepage Evolution
available from::
1998
1999
2000
2002
Today
5
Original Registration Path Flow
Registration was a 5-page process that came off more like a
purchase than a free trial registration.
Offer Page
Subscription Options
Customer Info Collect
Confirm Info and Submit
Confirmation Page
available from::
6
148 |
Offer Page Comparison
New headline
Lead with
- what
FREE
the
customer
language
will get
Original
New
Mention
Heavy
FREEcredit
copycard,
but with no-risk, no
obligation
Establish
value
Bullets
and bolding
Eliminate
unnecessary
Larger
image
fields,
i.e. “show”
the customer
address.
what they will receive
Risk free
reinforcement
available from::
Button is proactive
and beneficial
Testimonial adds
credibility
7
Subscription Options Comparison
Original
New
Changed from
“subscription registration”
to “free trial activation”
available from::
Credit cards and paid copy
removed
Savings spelled out
Incentive CTA
8
| 149
Customer Info Collect Comparison
Original
New
The “paid” look/feel
continues through the
registration process
Disarming copy in the
header - what a customer
actually would like to do
Subscription Type is
“FREE” as part of free
trial
Verify your address replaces
“billing information” to continue
the free trial approach
Start using Britannica Online
copy use on button to incentivise
the customer to continue
available from::
9
Confirmation Page Comparison
Original
New
Copy confirms that the
customer just made a
purchase instead of
activating a free trial.
Free Trial
Confirmation replaces
“order confirmation”
from the original
version to reemphasize that it is
not a “paid” process.
available from::
The credit card images - On
any up-front paid offer, these
are excellent. On a free trial,
they are the opposite.
10
150 |
WINNER - New Path
• 125% more free
trials per
Unique Visitor
• 65% more Paid
Subs per
Unique Visitor
available from::
• 53% lift in Total
Revenue
11
Monthly/Annual/Bundle vs. Annual Only
Offer Page
Subscription Options Page
Account Setup Page
Monthly/Annual/M-W Bundle Registration Path Flow
Annual Only Registration Path Flow
available from::
WINNER
117% lift in FT
61% lift in Paid Subs
130% lift in Revenue
12
| 151
1 Page Order Form Test
1 Page, 1 Column
1 Page, 2 Column
3% less FTs
1% less paid subs
4% less FTs
6% less paid subs
2 Page (Control)
available from::
13
Article Page CTA Test
Control Article Page
available from::
Test Article Page with New CTA
WINNER
• 30% less clicks but…
• 13% more FTs per UV
• 61% more FTs per click
• 24% more Paid Subs per UV
14
152 |
Basket Recovery Test
Test Article Page
Offer Page with
Pre-Populated Email
Email Sent 1 Hour
After Abandoning
Email Sent 24 Hours
After Abandoning
available from::
• 53% less CTA clicks
But…
• 11% more FTs per
Unique Visitor
• 4% more Paid Subs
per Unique Visitor
15
MSN Live Text Link Test
Control Article Page
available from::
Test Article Page with MSN Live Text Ad
• 20% less clicks per UV
• 14% less FT per UV
• 12% less Paid Subs per UV
BUT
• 70% gain in overall revenue
16
| 153
Hackersafe Logo Test
Control Offer Page
Test Offer Page with Hackersafe Logo
available from::
Logo delivering 13.2% lift
in free trials to-date
17
Future Multivariate Test Areas
available from::
18
154 |
Credits/Thank You
Joe Miller, Encyclopaedia Britannica
312-347-7065
[email protected]
available from::
19
| 155

Part 2 - MarketingSherpa

Transcription

Similar documents

Exploring the ARUCC PCCAT Transcript and Transfer Guide

How to Subscribe Daily Thanthi E-paper? Step 1: Log on to http

Tarot copy - Aim4Media

Transcript Instructions

Bob Cox, Publisher, Winnipeg Free Press

special $$$ savings! - Sophisticate`s Black Hair Styles and Care Guide

Requesting an Official Transcript

Firehouse hosts Great Big Home Show

to your all access subscription