EBS TESTING: HOW TO GET YOUR BUSINESS USERS ON-BOARD ……………….…………………………………………………

EBS TESTING:
HOW TO GET YOUR BUSINESS USERS ON-BOARD
AMIR M. FARHI, PANAYA INC.
……………….…………………………………………………
ABSTRACT:
Are you getting the cooperation you need from your business users? Do you have real-time visibility into your
key users' test activities? Do you have all the test evidence needed to pass an audit? This paper explains how
you can reduce 70% of the time it takes to create and execute an Oracle® E-Business Suite (EBS) test plan,
make it easy for your entire testing team (IT & Business) to manage and track all their testing activities and
improve the collaboration between IT and key users.
INTRODUCTION
Oracle® E-Business Suite systems must evolve at the speed of business change. It is therefore imperative for
Oracle E-Business Suite managers to adapt their EBS landscapes in a timely manner to address business needs
such as process changes, rollouts to new sites, and regulatory updates. In addition, they must cater to the
maintenance needs of their EBS systems and implement EBS support packs and enhancement package upgrades
on a regular basis.
Oracle E-Business Suite systems tend to be highly complex, built on hundreds of millions of lines of standard
code and customizations. This means that even a small change to the system could trigger a series of
inadvertent consequences. To prevent production failures, companies regularly perform a series of functional
test cycles – which can be executed manually and/or automatically.
Industry papers discussing approaches to test automation have been available for well over a decade. Little has
been discussed, however, about the applicability of test automation to Oracle E-Business Suite. This document
describes the specific challenges of EBS functional testing, when test automation is inappropriate, and how to
perform manual testing faster, better, and at a lower cost in these cases.
-1-
WHY IS AUTOMATED TESTING SUCH A CHALLENGE FOR ORACLE E-BUSINESS SUITE?
A survey performed by Panaya in 2012 of 100 enterprises revealed that even the most mature testing
organizations perform only 15% to 25% of their functional regression testing using test automation tools; the
remainder is manual. There are three main reasons, inherent to Oracle E-Business Suite, which explain why test
automation coverage is relatively low:
(i)
Oracle E-Business Suite functionality is heavily data driven. EBS business logic is dependent on and
functions differently with change in the input data; this translates to changes in screen flow,
screen content and screen format that must be factored into the test automation script.
(ii)
For business processes that change frequently, the test automation scripts must be continually kept
up to date. A challenging and costly proposition.
(iii)
Preparing the required test data for test automation is also challenging due to EBS’s complex data
structure and the various dependencies that exist in an EBS business process. Think about an
automatic test script for a procure-to-pay process that requires five different material types for full
testing coverage. Maintaining integrity of the data required to support the end-to-end process for
the five material types is a daunting task.
EBS MANUAL TESTING IS UNDERSERVED
Since it’s virtually impossible for a company to develop automation test scripts to cover every possible EBS
business scenario, manual testing is heavily relied upon to ensure that all the needed functionality is covered.
Yet manual testing is severely underserved in terms of tools that increase productivity, enforce processes, and
provide control.
Performing manual tests requires an intimate understanding of the business processes to be tested, and so, it is
typically the functional experts in IT or key business users (sometimes referred to as “super users”) that are
tasked with performing manual testing. They do their best to test, report defects, report progress, and
document test runs, on top of their other daily duties. This is all performed manually, typically with tools such
as Microsoft® Word and Excel.
In organizations where a dedicated test manager is allocated to the EBS team, management of the testing cycle
is performed by the test manager. In the majority of EBS shops, however, there isn’t a dedicated test manager;
the test manager’s role is performed by project managers, in the scope of their respective projects, as an
added duty.
Test management includes defining the testing strategy, identifying the tests to be performed, assigning tests
to the manual testers, chasing the testers for their individual status reports and consolidating them to a “bigpicture” testing progress and coverage status, managing defects, ensuring that testers perform the testing on
time, and more. Approximately two thirds of EBS shops use general productivity software like spreadsheets in
support of these test management activities. It is little surprise, therefore, that they lack visibility and control
over the testing cycle, and put too much time and effort into managing the testing activities.
-2-
DECIDING WHAT TO TEST: COST VS. RISK
The first pain point comes early in the testing cycle: determining what, exactly, to test. Following a technical or
functional change in a massive application like Oracle E-Business Suite, it’s a challenge to determine where
exactly in the system testing should be focused.
Customers have two options. One is to try to test every possible EBS transaction that may be affected to minimize
risk of production failures. That, of course, increases testing time and cost. The other alternative is to essentially
‘guess’ where the greatest impact will be and test those areas. The second option comes with lower costs but
much more risk. In practice, most organizations seek a happy medium between the two extremes, but the costly
nature of manual testing makes many lean toward less testing.
Finding a method to test more functionality within a strict timeline is one of the biggest challenges associated with
implementing changes in the ERP application, so finding a cost-effective resolution is extremely valuable.
-3-
CREATING TEST SCENARIOS FOR SCRIPTED TESTING
Test scenarios describe the steps needed to test a given function or process and the anticipated results of every
step. In theory, this should work well: each manual tester has a written script telling him or her what to do, how to
do it, and what results to expect. But in practice, this is far from the case.
Typically written in MS-Word or Excel, test scripts contain varying levels of detail, with no standardization as to
what should and should not be included. It’s a time-consuming process, as each script can take from minutes to
several hours to create. And there’s no guarantee that others will be able to reliably follow the script; what’s clear
to whoever wrote the script may not be to those charged with following it.
UNSCRIPTED “EXPLORATORY” TESTING
In some cases, test scenarios don’t exist at all. Instead, organizations practice a form of ad-hoc testing, known as
exploratory testing, where manual testers are tasked to perform a test as they see fit and report results.
There are two main use cases for exploratory testing:
1. The company doesn’t have pre-defined scripts and there is no time to create them. Therefore, manual
testers run a test cycle based on their knowledge of the way the application is used. While this reduces
test preparation time, it adds risk that the business process will not be tested adequately.
2. The QA team or functional testers create test scripts and conduct tests based on these scripts. At the
User Acceptance Test (UAT) phase the test manager doesn’t want the key user to repeat the exact same
scripts, because they were successfully tested already. Furthermore, the test manager needs to ensure
the application is tested exactly as the business users will use it in their daily work, so he asks the user to
simply do their day-to-day tasks on the QA system and check that everything works properly. This method
provides an additional assurance for the quality of the application.
Problems may arise when exploratory testing uncovers issues that developers must address. With no record of
what, exactly, the tester did, it’s difficult for the developers to accurately recreate problems that need fixing.
What’s more, there’s no knowledge transfer of the testing process from the business user to IT to enable IT to
create test scenarios for future use.
CAPACITY OF BUSINESS KEY USERS
Testing takes the business key users away from their pressing day jobs and limits their ability to fulfil their other
duties, such as supporting end-users, delivering training, defining processes, or liaising with IT regarding
functional needs. These key users are typically among the most experienced and highest paid in the user
organization and have very little time to spare. Since the key users are forced to fit test activities around their
normal workload, testing often has to wait.
DOCUMENTING TESTS PERFORMED
At the same time, these time-strapped key users are often required to collect evidence to prove whether or not
these tests were successful. This is especially true for organizations subject to government regulations, such as
Sarbanes Oxley (SOX), U.S. Food and Drug Administration (FDA) rules, and others. Users must collect detailed
proof of what was tested, by whom, and how. It can be a painstaking, time-consuming process for testers to
constantly take screen shots, then copy and paste them into a document to show which tests were performed.
This process can take at least 20 to 30 minutes per test, which translates into several days of non-productive work
per test cycle per key user.
-4-
MANAGING THE MANUAL TESTING PROCESS
Finally, when testing is manual, the testing manager has no real-time visibility into the process. He can distribute
detailed instructions to testers but has no way of knowing if, when, and how they are conducting the tests. As a
result, s/he cannot unequivocally know the level of progress being made and the precise issues which have
cropped up. When testers are located at multiple sites, the challenge becomes all the greater, forcing the
manager to resort to emails, spreadsheets and phone calls to keep tabs on testers.
OPTIMIZING THE MANUAL TESTING PROCESS
So it’s clear that manual testing will always be required; there’s simply no way in sight around having experienced
users verify that the system functions as it should. However, many aspects of the testing process can and should
be accelerated and managed. These include:

Determining what to test (test scoping)

Creating detailed test scenarios

Executing manual tests

Reporting defects that can be easily reproduced

Documenting test run evidence

Tracking test run progress
In short, manual testers require better tools to make the process more efficient, accurate – and less painful.
Testing managers need visibility and control over manual testing cycles. This visibility will enable organizations to
conduct more testing in the same time window and hence, find more problems before the code goes into
production. Increasing testing efficiency also has the effect of driving down the cost of testing and potentially
speeding up project implementation, thus increasing business agility.
EBS IT teams seeking a solution to accelerate the different facets of the manual testing process should evaluate
manual testing solutions on the basis of the following attributes:
1. AUTOMATED TEST SCOPING OF EBS-DRIVEN CHANGES
Determining which functionality of the Oracle E-Business Suite to test following a major change –
especially when implementing EBS-driven changes, such as patches, upgrades or custom code releases
– is crucial to minimizing the risk and cost of change. Analyzing the production usage of your EBS
transactions combined with an impact analysis of the changes on your system can yield a risk-based
approach to test prioritization that can reduce testing efforts by as much as 70%.
2. ACCELERATED CREATION OF TEST SCENARIOS
Writing test scenarios is a painstaking process but it’s also one that’s well suited to automation. Your
testing tool should capture the business process to be tested while a functional expert or key user
navigates the EBS application. The testing tool should translate the steps into structured human-readable
test scenarios, including documentation of expected results for every step of the scenario. You’re
essentially capturing the knowledge of that key user and making it available for all others to emulate in
their own testing. At the same time, you ensure the scripts are highly accurate – but take far less time to
create.
-5-
3. ACCELERATED EXECUTION OF MANUAL TESTING
Similarly, your testing solution should ease the burden on testers by offloading many manual tasks
associated with testing, such as walking the testers through the steps required when manually executing
the test scenarios.
4. ACCELERATED DOCUMENTATION OF TEST RUNS
Another aspect of test execution is the process of capturing and documenting test evidence for every test
run. Seek a testing solution that enables you to determine exactly which process the tester followed, what
the results were and how those results compare to what was expected. This is a crucial step in terms of
providing proof of testing for auditors, as well as tracing production failures to the gaps in your test
scenarios. It also relieves testers of the tedious process of manually documenting test results.
5. ACCELERATED REPORTING OF DEFECTS
The ability to effectively report functional failure is crucial in enabling IT address issues uncovered by
testing. Developers should get an accurate, screen-by-screen and keystroke-by-keystroke depiction of
how the tester reached a problem point, so they can recreate and fix the problem. This is exactly the kind
of data that testers struggle to capture and communicate accurately.
6. DESIGNED AND BUILT FOR BUSINESS USERS
Remember your key business users will ultimately have to use the testing tool as part of their (almost)
daily routine. They are already pressed for time; the last thing they want is to spend time learning how to
use a testing tool that they never wanted to use in the first place. So they need seamless access to the
testing tool from their EBS application. It should require minimal, if any, training to perform such testing
activities as: viewing the assigned test tasks, executing the test tasks, testing in collaboration with other
-6-
testers to complete a full business process, and reporting defects and progress. The tool should not
require complex installation of client-side software, and must be intuitive and simple to use, without
interfering with the non-testing tasks of the key user.
7. TEST MANAGEMENT
Finally, your manual testing solution should provide you with ongoing visibility and governance over the
end-to-end testing process. You should be able to plan test projects, assign tasks to all manual testers,
determine how close the testers are to completing their assigned tasks, and receive alerts when problems
arise or steps are completed. The solution should also enable you to track a distributed multi-site testing
operation.
8. SIMPLE SET-UP, MINIMAL ONGOING MAINTENANCE
You need a solution that doesn’t require cumbersome software installations, frequent patching or painful
upgrade processes on the manual testers’ client machines. It should take IT less than a day to set up the
full solution.
KEEP UP THE PACE
The pace of business change isn’t likely to slow down, so you can’t expect the Oracle E-Business Suite testing
burden to diminish either. Organizations that take the initiative to optimize their manual testing processes will
ease the burden on manual testers, enabling them to spend less time testing while allowing IT to complete more
projects. This ultimately empowers the company to become more agile and competitive. Learn how Panaya can
help you optimize your EBS testing processes. Visit us at: WWW.PANAYA.COM.
-7-
ABOUT PANAYA
Panaya Quality Management Cloud (QMC) for SAP, Oracle E-Business Suite, and Salesforce combines change
impact simulation, automated code remediation, collaborative test management and test execution, providing
companies with the agility to address their constant flow of business application changes.
IT departments use Panaya QMC to easily scope and accelerate the delivery of any business application change
(such as upgrades, patches, chart of accounts changes, etc.), while minimizing costs and eliminating risk.
More than 1,000 customers worldwide rely on the Panaya Quality Management Cloud, using “cloud wisdom” from
thousands of projects, transforming common practices into best practices.
DISCLAIMER AND TRADEMARK NOTICES
This report is provided by Panaya Ltd. It is completely independent of and not affiliated with Oracle or SAP AG.
Oracle is a registered trademark of Oracle. Oracle and other Oracle products and services mentioned herein as
well as their respective logos are trademarks or registered trademarks of Oracle. SAP is a registered trademark of
SAP AG. SAP and other SAP products and services mentioned herein as well as their respective logos are
trademarks or registered trademarks of SAP AG in Germany and in several other countries. All other product and
service names mentioned are the trademarks of their respective companies.
DISCLAIMER OF WARRANTY
Panaya Ltd. makes no representation or warranties, either express or implied by or with respect to anything in
this document, and shall not be liable for any implied warranties of merchantability or fitness for a particular
purpose or for any indirect special or consequential damages.
COPYRIGHT NOTICE
No part of this publication may be reproduced, stored in a retrieval system or transmitted, in any form or by any
means, photocopying, recording or otherwise, without prior written consent of Panaya Ltd. No patent liability is
assumed with respect to the use of the information contained herein. While every precaution has been taken in
the preparation of this publication, Panaya Ltd. assumes no responsibility for errors or omissions. This publication
is subject to change without notice.
© 2014 Panaya Ltd. All rights reserved.
-8-