White Paper_RF_Data_Migration

White Paper
Migrating Data into Remedyforce
Getting Started with Remedyforce Series
Dale Jones
24 March 2015
Data Migration
Getting Started with Remedyforce Series
Welcome to the “Getting Started with BMC Remedyforce”
Series
Today’s IT departments must drive business growth and innovation, while coping with less resources and increasing complexity. To
do this, they require an IT Service Management solution that provides best practices while minimizing costs. BMC Remedyforce is
built on Salesforce—the world's most widely used cloud platform—to deliver complete IT service management functionality with the
secure social, mobile, and collaborative capabilities users expect.
With the “Getting Started with Remedyforce” white paper series, our aim is to help you leverage BMC Remedyforce to improve the
effectiveness and efficiency of your ITSM operations. Each paper addresses a specific area of interest and provides you with
conceptual, functional and technical best practices to make configuration decisions and take action to gain value from your BMC
Remedyforce investment.
Remedyforce Data Migration
When making a major change to an ITSM system, such as moving from one tool to another or adding new organizations to an
existing tool, decisions have to be made about what to do with the information from another system. Should data be brought over
into the new tool? Is the complexity of doing so too high to justify the investment in personnel and time? What data should be
brought over, and what is best served by placing the data in a backup for as-needed access?
The purpose of this white paper is to address data migration considerations, provide instructions on Apex Data Loader setup and
provide some useful data cleanup formulas to make the migration of data easier.
Data Migration Considerations
Most often, organizations are faced with a decision about whether to migrate data when they are either implementing a new
system, or onboarding an organization into an existing system.
Data Migration: The movement of data from an existing system (legacy or prior) to a new system.
The first thing to consider when faced with a migration decision is – WHY? What is the benefit of migrating the data? Some reasons
for data migration could include:

Historical Reporting – There is a need to access historical information on a frequent basis.



Legal (or Regulatory Requirements) – Certain rules/regulation require retention of data in a readily accessible
system for a period of time.
Contractual Requirements – There may be customer or organizational contracts in place to have the data
accessible for a period of time.
Starting Point for a New System – Migration of open requests or last 3 months of data or system foundational
data (eg. menus and organizational information).
There are many reasons to migrate data to the new system, and just as many alternatives to migrating all the data.
Archival Solution Option
One way to save cost and time is to move the historical data to an archival solution. An archival solution does not mean
data is not accessible and reportable. Most systems have their data stored in a database. A database copy can be moved
to another database, which now can be accessed and reported on by hundreds of database reporting tools. Existing
reports would just need to be tweaked to point to a new database location. With the Archival Solution approach, you can
PAGE 2 OF 9
Copyright BMC Software, Inc. 2015
Data Migration
Getting Started with Remedyforce Series
still meet the requirements for historical reporting, legal and contractual requirements, while reducing the complexity for
implementation and ongoing maintenance of the new solution. Archival does not mean data is not accessible. Archival
simply means the standard reporting solution will need to pull data from one more source. Most reporting solutions can
pull data from multiple sources.
Partial Data Migration Option
Another alternative to migrating all data is to only migrate part of the data. This will reduce the data migration effort and
cost. This is a great option to get foundational data like groups and categorizations and also to migrate all open requests.
Some companies also choose to migrate some closed records by choosing a key reporting date, like the start of a year or
the start of a quarter or start of a month. (eg. Migrate all data created since January 1 2015)
Implications of Data Migration
Having a clean system at the start of go-live keeps the new system uncluttered and enables it to have the out-of-the-box
(OOTB) best practices easily adopted without customization to accommodate the old data. Migrating all data from an
existing system to the new system can double the amount of time to implement the new system and prepare it for
production use, which will increase the budget for the project. Choosing to migrate all data often adds complexity because
you need to add fields to the new system to store data in a different format (from the legacy system). This can change the
screen layouts and user experience, meaning that help files and other OOTB tools will be out of sync with the customized
solution. Whether you choose to do a data migration, a partial data migration, or archive your existing data, you should be
informed about the implications and risks of each approach.
Data Migration Best Practices
There are many considerations when migrating data that can make a project more successful. The biggest impact on a
data migration project is the volume of data. The more data to be moved the longer data will take to export, clean and
migrate. The older pieces of data usually have more mapping issues to account for, as processes have usually changed
over time. The older the data is, the less likely the data is clean. If key people with historical data knowledge are not
around to properly map this data to a new system, there is a greater risk of more data issues to resolve. Another key
question is whether the data being migrated from the “Single Source of Truth.” If the data is more accurate in another
location, you should consider if it makes sense to move the data from a less authoritative location. This usually comes into
play when migrating CMDB/Asset data, HR data or Active Directory data. It makes sense to go right to the source of the
accurate information.
Geographic and security discussions need to be held with key stakeholders to address questions like:
 Can the data ever leave the current country? Some solutions have backup servers or mirrored servers in other countries
so it is important to know the rules/regulations regarding data storage and movement.

Does data need to be encrypted if moved over an external network? Some industries (Banking, Governments,
Pharmaceutical…) have security standards that have to be followed when transferring data.
Another key step is to identify the data archival plan. This plan needs to account for the historical data in the current
system and the future archival plan for historical data on the new system. All systems run more efficiently when the
volume of data is reduced. The more data that can be archived and not moved to the new system, the smoother the new
system will run in production.
There are many things that can make the data migration project more successful, but the biggest success factor is
planning early. Data migration takes time and cannot be rushed. Get the key people involved that have historical data
knowledge and people that can make decisions for the future state of the data. Some ways to alleviate data mapping
conflicts are to create a matching legacy field (like legacy category), or concatenate data into a larger text field (like
‘Description’). If an archival solution for the current system and the future system can be put in place, the project will save
data migration time and budget. Make sure to have a unique identifier that is unique on the old and new system. This is
PAGE 3 OF 9
Copyright BMC Software, Inc. 2015
Data Migration
Getting Started with Remedyforce Series
normally a new custom field that will hold the old systems unique record number. This will save a lot of time during data
validation and speed up the cleanup of any data discrepancies. This unique identifier will also allow Administrators to do
multiple delta data migration before the cutover to production.
Data Migration Methodology
Stage
Notes
1.
High Level Analysis, Design and Overview
2.
Extraction Plan
-
3.
Data Cleanup
-
4.
Data Mapping
-
-
-
5.
Testing Plan
-
6.
Recovery Plan
7.
Initial Data Load
-
8.
Delta Migrations
-
9.
PAGE 4 OF 9
Go live
-
Owner of Data
Volume of Data
Type of Data
Sources of data
Are Attachments being migrated
Geographic Concerns
Security Concerns
Archival Solution
How data is pulled from the existing system.
What is pulled and what is not pulled from the existing
system.
How data is cleaned to match the new system requirements.
If there are a lot of data issues with the current system.
These issues need to be addressed before or during the
migration. Nothing worse than migrating bad data.
The old saying “Garbage In Garbage Out”
New required fields but the old system does not always have
a value in this field
Data validation rules in new system that were not enforced in
the legacy system
Data size issues (Size limits on new system) (Plan to deal with large
data records)
Map the existing Application to the New Application
o
i.e. Existing ITSM System will be moved to
Remedyforce
Map the existing forms to the new forms
o
i.e. Help Desk data will go to Incident object in
Remedyforce
o
i.e. People data will go to User Object in
Remedyforce
Map the existing fields to a new field or multiple fields on the
new System
o
Incident # - External Ticket
o
Status – Status
o
Field A, B, C concatenated into Description
o
…
How to validate the data was moved successfully
Normally done with record counts and reports with counts
How is data recovered or fixed if a discrepancy is found
Leverage a Unique Identifier field for cleanup
Leverage Data Loader Upsert or Update
Load all needed data into the new system
Make sure a unique identifier exists in both system (IE Ticket
Number or Employee ID)
This allows data to be validated in the new system months before
go live
This allows for data that has changed since initial Data Load
to be loaded multiple times before the go live
This allows for discrepancies or process changes to be
address
This allows for practice of the Go Live Delta migration
multiple times
This also limits the amount of data that is migrated during Go
Live, which limits the outage and Go Live validation
Old System switched to Read Only
Final migration of the changed data since the last Delta
Migration
Copyright BMC Software, Inc. 2015
Data Migration
Getting Started with Remedyforce Series
Apex Data Loader
There are many tools that can be leveraged for moving data in and out of Remedyforce. Data Loader is just one tool, but it
has one advantage: Data Loader has a download link built right into the Salesforce platform.
Instructions for using Data Loader
1.
2.
PAGE 5 OF 9
Open Salesforce/Remedyforce
Navigate to Setup, Administration, Data Management, Data Loader
Copyright BMC Software, Inc. 2015
Data Migration
Getting Started with Remedyforce Series
3.
Press the “Download the Data Loader” link
4.
Run Install exe
5.
Get Security Token (Personal Setup, Reset My Security Token)
6.
You will get an Email similar to the following:
Dear Sample User,
When accessing salesforce.com from outside of your company’s trusted networks, you must add a
security token to your password to log in to a desktop client, such as Connect for Outlook, Connect
Offline, Connect for Office, Connect for Lotus Notes, or the Data Loader.
New security tokens are automatically sent to you when your salesforce.com password is changed
or when you request to reset your security token.
Your new security token is below. Note that security tokens are case sensitive.
User Name: [email protected]
Security Token: GwCiI0DdWHOMRai68lZVaPABC
Please add your security token to your password. Note that you do not enter a security token in
place of your password when logging into salesforce.com via a browser.
----For Outlook Edition and Connect for Outlook:
1. Go to Outlook > Tools > Salesforce Options
2. Enter the password and security token in the password window. For example, if your password is
"myPassword" and your security token is "XXXXXXX", you will enter "myPasswordXXXXXXX"
into the password window.
3. Click verify. This tests that you have correctly entered your password and token.
4. If verification succeeds, Click 'OK'
PAGE 6 OF 9
Copyright BMC Software, Inc. 2015
Data Migration
Getting Started with Remedyforce Series
7.
8.
Launch Data Loader
Login
User Name: [email protected]
Password/Security Token: myPasswordGwCiI0DdWHOMRai68lZVaPABC
9.
Data Loader Settings
10. Data Loaders Functions

Export – Export data

Update – Update existing records



Insert – Insert new record
Upsert – Update if a matching record is found, if a matching record is not found then an insert
new record occurs.
Delete – Delete records
Data Loader Useful Tips
Apex Data Loader is a very powerful tool that can be very productive in the right hands and very destructive in the wrong
hands. Always perform backups (Export) of data before any changes to data occurs. In Settings, the Batch size
sometimes needs to be reduced if the data imports causes triggers to run on additional forms. This needs to be done
when importing Remedyforce Changes or Remedyforce Tasks. Also in settings the Login server setting needs to be set
to https://test.salesforce.com for Sandbox servers and https://login.salesforce.com for Production servers. The Time Zone
in settings, needs to match the data’s time zone for the data that is about to be imported.
PAGE 7 OF 9
Copyright BMC Software, Inc. 2015
Data Migration
Getting Started with Remedyforce Series
Data Cleanup Formulas
When you’re preparing data for import into BMC Remedyforce and the Salesforce platform, there is some cleanup that
has to be done to ensure an easy load. This includes ensuring the data formats are consistent, that lookup fields are
defined and that data is correctly formatted for load into the destination tables.
Date/Time Formatting
Importing Dates into Salesforce is tricky, as Salesforce is expecting the dates to be in a particular format. The best way to
see how Salesforce expects the date to be formatted is to export some existing data and mimic the date format when
importing new data into Salesforce.
Some date conversion example formulas:
UNIX Time example = 1420651550
Start of UNIX Time = 1/1/70 12:00 AM
Unix time Formula = “=1/1/70 12:00 AM +(1420651550/86400)”
Example of UNIX Time = 1420651550 would convert to “1/7/2015 5:25:50 PM EST”
Salesforce Date Format = “=TEXT(C9,"yyyy-mm-ddThh:mm:ss.000Z")”
Example of Date/Time = “1/7/2015 5:25:50 PM EST” would convert to 2015-01-07T17:25:50.000Z
Lookup Fields
Salesforce uses a lot of lookup fields. The GUID value for the lookup field is needed during the import. The way to get the
GUID value is to export the existing values with their GUID. In the import file a VLookup column will need to be added with
a formula to perform the VLookup.
Example to get the GUID of a User in the excel file usersALLUAT.csv.
"=VLOOKUP(E2,usersAllUAT.csv!$A:$B,2,FALSE)"
Concatenating Data
Some salesforce fields need the Last Name and First Name together or the need to concatenate multiple fields into one
Description or Notes field.
Example of Concatenation:
=CONCATENATE(A9," ",B9)
In Summary
There are many considerations when migrating data from an existing ITMS system into Remedyforce. Early, detailed
planning for data migration is key to your success.
BMC Remedyforce Professional Services specializes in complex data migrations. You can learn more by visiting BMC
Remedyforce Consulting Services at: http://www.bmc.com/it-services/remedyforce-services.html
Hopefully this document has helped to answer some of your most pressing questions about data migration planning and execution,
but you can always get more information by visiting us at the BMC Remedyforce User Community at: bmc.com/communities.
PAGE 8 OF 9
Copyright BMC Software, Inc. 2015
BMC delivers software solutions that help IT transform digital enterprises for the ultimate competitive business advantage. W e have
worked with thousands of leading companies to create and deliver powerful IT management services. From mainframe to cloud to
mobile, we pair high-speed digital innovation with robust IT industrialization—allowing our customers to provide amazing user
experiences with optimized IT performance, cost, compliance, and productivity. We believe that technology is the heart of every
business, and that IT drives business to the digital age.
BMC – Bring IT to Life.