Tait and Cloud Computing - Canterbury Software Cluster

Cloud Computing Experience
presented to Canterbury Software Cluster
Dan Van Wieren
ICT Infrastructure Manager
April 7, 2015
1
www.taitradio.com
Confidential – Not to be distributed outside of Tait Communications
Introduction
• 
Tait has around 750 staff globally, with about 580 based in ChCh. We have offices in 8 countries
and clients in around 90. Staff, clients and partners in all those regions access some resources
from public cloud infrastructure as well as on-premise and colocated infrastructure. We do not
expect to ever go 100% into the cloud for many reasons.
• 
For Tait cloud based service delivery means focusing on an HTML5 compliant browser as the
primary application presentation tool to deliver services to anyone, anywhere, anytime, any
device, any platform, any browser, however, this is very much a transitional challenge.
• 
We minimise the use of client side proprietary tools as much as possible and we don’t want to be
tied to a particular vendor and therefore where possible we prioritise vendors who support open
standards. But we do recognise there will always be exceptions due to business needs or vendor
product limitations especially if there is a lack of support for open standards.
2
Confidential – Not to be distributed outside of Tait Communications
What Has Tait Moved to the Cloud
2009
•  IaaS - Review of VMware as an internal service delivery platform and NZ based shared
infrastructure service providers were investigated.
•  SaaS - Migrated CRM to SalesForce from various on-premise systems.
•  SaaS - Implemented Learning Management System.
•  SaaS - Implemented Timesheets.
•  SaaS - Migrated to different email filtering from on-premise.
2010
•  SaaS - Migrated email & calendaring to Google from on-premise. Docs was not a focus.
2011
•  IaaS - Migrated public website to Rackspace from on-premise (www.taitradio.com).
2012
•  SaaS - Implemented organisational charting.
•  IaaS - Review of VMware as an internal service delivery platform and NZ based shared
infrastructure service providers were investigated for alternatives.
3
Confidential – Not to be distributed outside of Tait Communications
What Has Tait Moved to the Cloud
2013
•  IaaS - Implemented document management system hosted with AWS to replace on premise fileservers (CFM).
•  SaaS - Implemented system for performance management (reviews and appraisals).
•  SaaS/IaaS - Consideration *aaS new ERP system.
2014
•  SaaS - Migrated to different incident management system (IMS) from on-premise.
•  IaaS - Migrated Intranet / Collaboration environment to AWS from on-premise private cloud
infrastructure.
•  IaaS - Migrated public website (www.taitradio.com) from Rackspace to AWS.
•  IaaS - Migrated R&D workloads to local public shared infrastructure provider.
•  IaaS – (WIP) Migrate Azure based workloads to AWS.
2015
•  IaaS - Document management on private cloud infrastructure co-located at local data centre
provider used for R&D based cloud file management for IP and bandwidth reasons.
•  Currently having a look at AWS WorkMail.
4
Confidential – Not to be distributed outside of Tait Communications
What Has Tait Moved to the Cloud
• 
• 
• 
• 
• 
The key system that we are now delivering from AWS is ‘Cloud (Document) File Management’
which is based on Alfresco running from a RedHat AMI and using PostgreSQL.
For the initial rollout we were supported by a vendor.
The app, DB and content server were on separate hosts and integrated with our corporate
OpenLDAP directory server. The master LDAP is located on-premise in ChCh and hosted on a
Solaris virtual zone and replicated to an AWS node running on another RedHat AMI.
Initially deployed without HA. But supported with node replication and backups.
Over time we have augmented the underlying infrastructure using AWS building blocks to deliver
a highly resilient scalable system. This has included:
• HA across multiple availability zones (AZ’s) in Sydney and complemented with ELB. AZ’s
are 2 physically and geographically separated data centres in Sydney.
• DR to another AWS region in Singapore.
• RDS for PostgreSQL.
• In the future Route 53 will be reviewed. This is already used on our public website.
5
Confidential – Not to be distributed outside of Tait Communications
Which Cloud Vendor and Products are Used
• 
• 
• 
• 
We have used Rackspace for IaaS.
We are using Azure, for some client tools but migrating those workloads to AWS.
We are now focused on AWS and use the following features:
-  Route53, ELB, S3, RDS, Auto scaling, EC2 (AMI’s), ElastiCache, CloudWatch, VPC, multiple
AZ's, and DR.
-  CloudFront is on the radar.
-  AWS Business Level Support.
-  We have partnered with Fronde and they now handle our monthly invoicing and this means we
are not tied to a credit card bill. This also gives us access to AWS support via Fronde as well
as direct with AWS.
-  We have 3 system engineers that have completed the AWS architecture course and one of
who is working full time on service delivery from AWS.
We use Google for email and calendar, video. Docs is in limited use but not promoted.
• 
We use SalesForce for CRM and RemedyForce (IMS).
6
Confidential – Not to be distributed outside of Tait Communications
Evolving from a Stand-alone System to Full HA & DR
• 
The following diagrams provide a high-level illustration of how we have evolved the CFM
environment form a stand-alone, replicated and backed-up system to a fully HA and DR
configuration using the following components:
- 
- 
- 
- 
7
Elastic Load Balancing (ELB)
Relational Database Service (RDS) - Managed PostgreSQL DB for metadata
Simple Storage Service (S3) - Content storage
2 Availability Zones (AZ) - Separated Data Centres
Confidential – Not to be distributed outside of Tait Communications
Initial Stand-Alone Setup
8
Confidential – Not to be distributed outside of Tait Communications
Introducing ELB
9
Confidential – Not to be distributed outside of Tait Communications
Introducing RDS
10
Confidential – Not to be distributed outside of Tait Communications
Add New Application Server
11
Confidential – Not to be distributed outside of Tait Communications
Introduce New Content Repository Server Using S3
12
Confidential – Not to be distributed outside of Tait Communications
Remove Old Content and App Server
13
Confidential – Not to be distributed outside of Tait Communications
Final HA Implementation
(Singapore DR Site not shown)
14
Confidential – Not to be distributed outside of Tait Communications
Full System View Including SSO
15
Confidential – Not to be distributed outside of Tait Communications
How Long Have we Been Using the System
in Production
• 
The key system that we are now delivering from AWS is ‘Cloud (Document) File Management’
and this was started in February 2013. After pilots and testing it was operationalised in November
2013.
• 
There has been a major focus on IaaS over the last 3 years.
• 
We still operate on-premise private cloud infrastructure for relevant workloads.
16
Confidential – Not to be distributed outside of Tait Communications
How Many People Are Using the Systems
We have many systems deployed which have different numbers of users. But we have global
systems in production with user numbers at around 750 staff, with about 200 being overseas.
Some systems are also used by clients and partners.
17
Confidential – Not to be distributed outside of Tait Communications
How Long Did it Take to Migrate the System
• 
The key system we have been focused on is global cloud file/document management (CFM).
• 
It would typically take a few days to create a test or production environment. This is because all the building blocks are already
in place with AWS. There are no debates around funding new hardware or carving up existing infrastructure for capacity or
obtaining capex. Spinning up new hosts only takes a few minutes.
• 
The usage of the system is being staged over a much larger period of time which means we are staggering the adoption of the
platform across departments and regions and decommissioning local file-servers as these entities migrate across.
• 
The environment has been enhanced over time to meet availability requirements related to RTO and RPO to provide HA and
DR.
• 
We still have a huge task to complete the data migration and work out which data belongs in our CFM and which should stay
on-premise on traditional NFS and CIFS servers.
Because the public cloud model is pay as you go you don’t have to request capex to deploy infrastructure and wait for this to happen. You can
just consume at will what you need. But you must understand the cost implications of this approach to minimise unexpected bill shock and this
places more self responsibility and discipline on managers, engineers and project leaders. We understand the financial parameters within
which we can leverage IaaS for service delivery and work within approved budgets. AWS pricing is well documented so you can make informed
decisions about the costs of the infrastructure you are creating.
18
Confidential – Not to be distributed outside of Tait Communications
Which of the Predicted Benefits Worked Out
and Which Did Not
19
• 
Agility - anyone, anywhere, anytime, any device, any platform, any browser.
• 
One fundamental goal has been to be able to perform service delivery with a number of systems
as effectively as possible to staff, partners and clients around the world. The Cloud is a great
platform to use for this in terms of time-to-market, accessibility and resiliency. Our experience
demonstrates that this has been a positive outcome.
• 
There are some views that the Cloud can simplify service delivery and this is true, however, but
it is easy to add complexity when integrating hybrid Cloud platforms, particularly around
connectivity. So while you maintain multiple service delivery platforms which includes public
cloud, private on-premise cloud and colocation you still risk a degree of process integration
complexity when connecting multiple environments together across international borders.
• 
Another positive has been the ability to utilise AWS infrastructure in different parts of the world to
bring some service delivery closer to end-users by leveraging AWS’s global network for access
(Route53) and content delivery (CloudFront).
Confidential – Not to be distributed outside of Tait Communications
What Surprised You About Moving to the Cloud
20
• 
The range of solutions available to be used as building blocks to facilitate a resilient platform for
service delivery.
• 
To date we have not seen any contention issues impacting on performance.
• 
Auto scaling - the ability to have an on-demand pricing model that allows you to deploy systems
using a thin provisioning approach to minimise costs. So you can turn on or off systems to
manage costs and/or meet demand using policies to auto scale the required infrastructure. This
is extremely useful for test & dev workloads that are not required 24x7 and quiet periods on
production systems. AWS provides the infrastructure to facilitate auto scaling, but this can still
be impacted by traditional budget, product licensing and product functionality constraints.
Confidential – Not to be distributed outside of Tait Communications
What Lessons Did We Learn for Others Starting on this Path
21
• 
Be very clear about the business problem you are trying to solve to ensure that the Cloud is appropriately aligned with your strategic
direction and security policy.
• 
Understand issues around your intellectual property and data sovereignty requirements and match those with your security policy which
may dictate on-premise service delivery or at least using a local data centre to keep data in country.
• 
Consider the cost model. If your ICT budget is under pressure then some aspects of Public Cloud may not be appropriate.
• 
Consider your view on IaaS, SaaS or PaaS. While we use IaaS and SaaS be clear on which is relevant to you.
• 
IaaS will not necessarily simplify service delivery for your infrastructure team as there are still some common challenges, i.e. building
OS’s and integrating networks. This extends to traditional Change, Release & Configuration Management as well as Availability &
Capacity Management. You are still managing infrastructure.
• 
You may still need to provision your own tools to monitor the environment as the provided ones are for high level use only. You can
augment this with advanced tools from 3rd parties or roll your own open source tools.
• 
With one Cloud provider we did get caught out with some bill shock in relation to systems we had shut down but not deleted. In this
situation with AWS there is no charge but the policy of this other provider was to charge anyway and we got stuck with some unexpected
costs.
• 
Be conscious of the data that flows out of your AWS environment as this is charged. Generally it is not a significant cost, but if something
goes wrong you don’t want unexpected and excessive costs.
Confidential – Not to be distributed outside of Tait Communications
What Lessons Did We Learn for Others Starting on this Path
• 
• 
Security when selecting a cloud service provider
• 
Tait is working toward ISO27001 certification and we are applying these principles whenever we are investigating potential
systems and vendors.
• 
When selecting a cloud service provider it is essential to research around the level of security provided. In a cloud environment
security depends on the level of trust and transparency.
• 
Trust about policies and procedures. To provide confidence on these, cloud service providers certify their data centres against
recognized international security standards such as ISO 27001, SSAE 16, etc and third party audits.
• 
Transparency is more about technical security controls in network architecture and design.
Security in AWS deployment
• 
In any cloud service model, responsibility of data security lies with the client not the service provider. Hence it is
essential to develop and deploy necessary processes for secure deployment.
• 
We use SSL certificates in all situations as wells apply a policy of securely hardening all systems in the cloud without
exception.
• 
• 
A good thing with AWS is that we can do vulnerability assessment or penetration testing on our systems. (to conduct a
vulnerability test on AWS systems, we need to take prior permission from AWS).
Security in Operation
• 
22
AWS trusted advisories provide information on security of the AWS environment. These also provide baselines of current
deployment in terms of performance, security and cost optimization.
Confidential – Not to be distributed outside of Tait Communications
Finally - Support from AWS / Fronde
• 
• 
• 
• 
23
We have chosen AWS as our preferred cloud service provider.
AWS themselves have been very supportive of what we have been doing
with their infrastructure.
We attend their lunch and learn sessions in Christchurch.
With support from Fronde, AWS have been available for on-site workshops
to helps us understand how their building blocks can facilitate service
delivery.
Confidential – Not to be distributed outside of Tait Communications