Akamai | Managing Bots in Travel Companies

MANAGING BOTS IN
TRAVEL COMPANIES
TRAVEL’S BOT PROBLEM
Bots are a fact of life for travel companies, accounting for 30% or more of a travel site’s traffic.1
From the outside in, bots are a natural fit for a very dynamic industry - prices and schedules
change constantly yet “deadlines” still need to be met. With so much constant change, it is
challenging for legitimate and malicious actors to stay up-to-date - enter bots. What better way
to track and aggregate all of the changes to flights, schedules, prices, and hotel availability than
with an automated program that scours the Internet for the most recent and relevant data.
However, bots can impact site performance, search
visibility, on-site experience and even the competitive
landscape. If a travel company doesn’t know what types
of bots are on its site, it can’t act to reduce the impact
of those bots on customer experience.
So what do bots have to do with travel?
Simply put, bots are automated traffic on a travel
company’s site. They have many different purposes,
both positive and negative. Bots impact: how
customers find a site; the quality of the onsite
experience; the accuracy of marketing data; and the
resulting analytics, competitive positioning and even,
directly, the bottom line.
How travelers search and select travel and hotel
accommodations: Once a traveler has decided to
plan a trip, the first step is often to search out travel
and hotel options. The path can start with traditional
search engines (e.g., Google, etc.), OTA sites (e.g.,
Expedia, Priceline, etc.), or even on the provider’s site
(e.g., United, Hilton, etc.).
Bots play an important role here. Search engines create
bots to crawl websites and return information on a
site’s content, helping shape how those websites are
prioritized in search results. It is necessary to ensure
that search engine crawlers receive high performance
to avoid jeopardizing their rankings.
Travel companies also need to consider the changing
role of search engines. Take Google Flights as an
example. In addition to connecting travelers to OTAs or
provider sites, Google can now provide actual itinerary
search functionality, results and booking information.
This introduces another consideration for travel
MANAGING BOTS IN TRAVEL COMPANIES
companies that may have been stopping bots crawling
their itinerary builder. What if Google (or Yahoo or Bing
or others) wishes to provide this functionality? Do you
prevent them or encourage them?
Travel companies also create bots or contract with a
third party service to crawl their own site in order to
evaluate how effective their SEO efforts are. A related
aspect of bot interaction involves partners that sell
a travel company's product or service through other
channels (OTAs or traditional travel agents). They will
scrape the site to ensure they have the most up-to-date
pricing and content. The purpose is legitimate and
the benefit real (extending the reach and audience).
However, travel companies need to ensure that these
bots get the information they need without negatively
impact on-site experience.
On-site Experience: Once the travel company has
brought the customer to its site, it must ensure a
high-quality experience so that the shopper can
easily and quickly find the products or services they
want. However, with 30% or more of traffic being
bots [definitely need a cite up above if going to
repeat this], this means that regardless of the type or
intention of the bot site performance will degrade,
causing legitimate, human traffic to have a negative
experience.
Marketing Data and Analytics: On-site experience
is also driven by data. As more travelers expect
personalized, concierge type experiences, travel
companies need to leverage the data they are
capturing on their sites. This enables them to deliver
a more customized experience, leading to more,
higher-value sales. However, a by-product of the
proliferation of bot traffic is that marketing data,
which drives key tactical and strategic decisions, is
corrupted. Bots skew the data and misrepresent the
true nature of customers, invalidating conclusions
drawn from the data set.
Customer Loyalty/Brand Awareness: Loyal customers
are awesome - they buy more, more often and support
the brand socially. Building trust with customers to
move them into this upper echelon is challenging but
very rewarding. However, third parties can use bots to
get between retailers and their customers, jeopardizing
that customer relationship.
For example, bots that scrape sites for information
can often confuse and divert customers as they search
for a travel solution (flights, hotels, rental cars, etc.).
If identical offerings come up on multiple sites, the
impact to travel company's brand can be significant,
leading customers to potentially shift loyalty. Travel
companies also lose out on the ability to collect
customer data, a growing concern as data quickly
becomes the currency of the future.
Balance sheet: There is direct correlation between
bots and costs in the flight booking industry. If bots are
on a site generating queries, whether for good or bad
reasons, these requests are sent to global distribution
system (GDS), which sends back the appropriate
information. Every time the GDS communicates back to
a request, the site generating the request is charged.
If 30% of traffic is bots and there are thousands if not
hundreds of thousands requests a day, that can quickly
balloon the cost to a travel site for activities that are not
driving any revenue.
Another way that revenue is impacted is when a travel
site’s content is scraped by an unauthorized party. If
this data is used to capture travel revenue on the 3rd
party site, the victimized site loses the opportunity to
cross-sell or up-sell. By some estimates, this can be
$20-$40 per traveler for airlines - a significant source
of additional revenue.2
MANAGING BOTS IN TRAVEL COMPANIES
Competition: On site scraping is a big concern when it
comes to competition. As information is shared openly
in order to provide a better customer experience, it also
means that competitors can scrape sites to ensure their
pricing and product offerings are comparable or better.
This is made even easier through bots, automating and
scaling the process beyond what human competitors
could do.
What does good look like?
The ideal solution would allow a travel company to
manage bot traffic to ensure the best possible outcome
- maximizing the positive results and minimizing the
negative - depending upon the type of bots it sees.
Simply blocking all bots or even just bad bots is only
a temporary solution that is ineffective in the long
run - blocked bots will return smarter and faster. By
managing how those bots are allowed to interact with
the site, travel companies can minimize the negative
impact of those bots without tipping off the operator
that they have been discovered. Two common solutions
are simply slowing them down to reduce the value and
timeliness of the information they are gathering or
serving them alternative information - pushing them to
a page with intentionally inaccurate content. Both are
great solutions considering how travel is governed by
deadlines.
Bots absolutely provide a benefit to travel companies.
They can help improve search results, improve SEO,
and in the case of search engines turned booking
agents (see Google Flights), enable travel providers
to get their brand in front of customers with fewer
clicks. Rather than simply blocking (bad) bots, travel
companies need to understand what bots are on their
site and what they are doing - this will allow them to
initiate a strategy that elevates good bot behavior and
manages bad bot behavior.
Akamai’s Bot Manager: In direct response to the
current state of bot solutions, Akamai has created a
unique alternative for travel companies struggling to
deal with the bot problem. The solution is designed
to allow an organization to identify, categorize, and
manage the bots - good and bad - to achieve the
goals for their website. Bot Manager provides a range
of management actions beyond just blocking to help
organizations maximize the positive impacts and
minimize the negative impacts of their bot traffic.
They are able to then analyze and report the activity
to improve visibility on the bot issue. In addition to
visualization and reporting on bot traffic, Bot Manager
can also help organizations with their own internal
marketing data. Website and page view statistics can
be significantly skewed by bot traffic, making it difficult
to understand the behavior of real users interacting
with the site. Bot Manager identifies bot-generated
requests in order to filter out bot traffic from human
traffic, which can improve marketing’s data and
analysis, leading to better business decisions.
To learn more about Bot Manager visit us here.
©2016 Akamai Technologies, Inc. All Rights Reserved. Reproduction in whole or in part in any form or medium without express written permission is prohibited.
Akamai and the Akamai wave logo are registered trademarks. Other trademarks contained herein are the property of their respective owners. Akamai believes
that the information in this publication is accurate as of its publication date; such information is subject to change without notice. Published 04/16.