Rational Quality Manager and Rational Team - Eclipse

Rational Quality Manager and Rational Team Concert
adoption in Tivoli Rome Lab
Ilaria Gorga ([email protected]) Software Engineer, IBM
Alessandro De Micco ([email protected]) Staff Software Engineer, IBM
Abstract - Improve product quality it’s a matter of both team process and tools adopted. In this article, we address
this consideration by providing a story example showing the migration from TTT – Notes DB – text Documents to the
development lifecycle approach, using Collaborative Application Lifecycle Management artefact integration. IBM®
Rational Team Concert™ Version 2.0.0.2 and IBM® Rational® Quality Manager Version 2.0.1 were used to achieve a
new day-by-day process. Moreover, adopting those tools made architects, project managers, developers and testers
able to feed and maintain in a centralized way the product artefacts linked together: product requirements, developed
story and test cases are easily traced and tracked within a single point of entry. The entire above issues ensure the quality and completeness of the product delivered to customer.
Introduction
Starting from 2008, the Rome Lab has adopted the Rational Quality Manager (RQM) to support the test process. IBM Rational Quality Manager is a collaborative, Web-based tool that offers comprehensive test planning, test
construction, and test asset management functions throughout the software development life cycle. Rational Quality
Manager is based on the Jazz platform.
The RQM introduction integrated with the RTC adoption, improved the implementation of the Agile development
process in the Tivoli Lab.
Main strengths identified using RQM are the following:
1.
Project lifecycle management with a test plan centric approach
Integrated test management with a WEB interface across all the test aspects (business objectives, test strategy, test cases, resources, environments, entry/exit criteria, risk assessment, plan and test cases review and approval, test tracking ...). All project related data (iteration plans, test, and defects) are linked together.
2.
Collaborative and adaptive test plan management
Structured and customizable test plan with multiple user defined sections, possibility to assign different
ownership for specified sections, team collaboration improvements
3.
Collaborative and adaptive test cases design
Test cases easy to create, maintain and evolve, test cases re-use, possibility to assign different ownership for
specified sections
4.
Easy link between RTC epics-stories and requirements and test cases on RQM
For example, it is possible to link test scenarios defined in RQM with related user stories entered in RTC.
Increased requirement traceability and direct linking with test cases identified for a specific requirement
5.
Execution paths optimization
Easy determination of the most efficient configuration coverage patterns and execution paths and related
execution record generation
6.
Extensible and open architecture
Leverage test automation feature provided by RQM integrating automated test suites developed internally
7.
Test Assets and Test environments management
Test asset discovery, inventory and software provisioning by RQM. Structured test environment definition.
Automated test environment installation and direct linking with test cases.
Requirements definition
A clear and detailed description of product requirements is one of the most effective elements to have a product release starting with high-level quality. Moreover, traceability of requirements through the software development
lifecycle is a key factor to easily identify whether all requirements are developed and have a proper level of test coverage.
Product Management was populating a Lotus Notes DB repository, giving access to a re-
stricted list of people to discuss and refine product requirements list. This process lead to some side effects: limited list
of empowered people; a time consuming process to define, refine and maintain requirements list artefact; information
kept isolated and unlinked to other product artifacts.
So, we moved requirements definition and management in
Rational Quality Manager. Because RQM comes with a built-in tab that allows the business analyst to populate a database
by
simply
complete
You can see an example of defining a requirement in Figure 1.
Figura 1 – Requirement definition using RQM
a
web
form.
The inserted requirements are then collected and collected in a view like the one shown in Figure 2.
Figure 2 – Requirements view in RQM
This adoption makes the entire team empowered to contribute to discussion on requirements design and development; the fast access to the web interface makes the tool availability uninterrupted; it makes the requirements record
ready to be associated with other artefacts.
Design your Test Plan
The product test plan is the central artefact in our process workflow. It describes the scope of the test and the
strategy adopted to ensure product quality. It contains assumptions; risk assessment; list platform coverage and software
dependencies; test phase entry and exit criteria; quality objectives and more. This was usually accomplished by writing
a text document from a single author, then stored on a Lotus Notes DB for review process from a close list of reviewer
and
then
saved
untouched
once
approved.
But, as the Agile methodology is adopted in our organization several of the Test Plan items change rapidly. The effect
of that is that we need to adopt a new tool to respond to the changes. Again, we found in RQM the right tool to solve
this
issue.
When you create a test plan in Rational Quality Manager, you can design the plan on a template containing section that
you can create, based on your needs.
You can also create several templates suitable for different teams working on different aspects of the product, like
the component verification tests, the scalability and performance tests, the globalization verification tests, and the integration test.
All of those teams have their own customized Test Plan but all of them centrally managed and linked to the other
product artefacts.
So, the following figures show details of our Quality Assurance phase. Meaningful section for our process are now
easily designed and kept update by web interfaces access.
Figure 3 – Summary of TWS QA test plan
Each section is different but all of them have an editor (Figure 4) while some sections, provide a rich-text editor having usual capabilities: font selection, inserting pictures, creating tables.
Figure 4 – Objectives defined in the TWS QA test plan
Other sections have a specific layout and behaviour, like the Test Schedules shown in Figure 5.
Figure 5 – Test schedules for TWS QA test plan
The picture below shows the project test schedules and a S-curve approach in completing the test effort. In our process, especially coming from PMR analysis, we decide to specify the environment condition in which to test.
Figure 6 – Execution Trend report view
In the Test Plan definition, it’s possible to associate a list of Test Environments as shown in Figure 7.
Figure 7 –Test Environments associate with the Test Plan
Again, RQM allows us to manage such kind of information in a centralized way. Information is stored in the database and from now on it will be easy to be retrieved and compared in next product release or to be a reusable asset.
Test Case Definition
Test cases are fundamental to maintain the desired level of quality in the developed product. Test cases are the
declaration
of
the
objective
for
the
test
team
to
state
that
the
product
works
as
expected.
In the past, each test team member was asked to design test cases in several artefacts like text document, or sheet document that would be aggregated in a document called Test Design. Again, after a review and approval phase such a
document would be added on a Notes DB as a record for the testing quality process. Problems were: different granularity in designing test case because different author style or skills; redundancy of test cases; lack of clarity on expected
results
since
developed
artefact
could
differ
from
the
starting
picture.
Rational Quality Manager provides a built in test-case template with some predefined sections that you can modify in
order to obtain your own template. You can reference and link your requirements with the test case. And you can link a
test script to the environment you execute it.
You can see an example of Test Cases list in Figure 8.
Figure 8 –Test cases list
Test Execution Records
The physical execution of a test script, associated to the environment dedicate to run it, it is called Execution
Records.
By tracking the list of execution of those records and by knowing the weight (the effort) of each one of those, the product management and the product team itself get the execution progress of the overall product.
In the past, a manual insert and modification of entries in a Notes DB, plus the association of style-sheet document
obtained those tracking purposes.
This produced an isolated level of project control, with the entire team out of the chance to verify in real time the
product status. The sense of feeling part of the global lifecycle was totally missed.
With RQM and RTC, we come to the opposite: daily scrum meeting attended by the team, in which each team member reports its daily duties or problem to be addressed make the entire team responsible to insert and update any kind of
information that goes to the product tracking.
In RQM, by running the Execution Record trough a console an automated process is computing the executed effort.
An example of execution record is shown in Figure 9.
Figure 9 –Test execution record
You can even design your test script by hand, so to uniform different author style in a template manner. (Figure10)
Figure 10 –Test Script overview
You can manually or automatically execute a test script (Figure 11)
Figure 11 –Test script execution
And finally you can adjust the status of the test script. An example of test execution result is shown in Figure 12.
Figure 12 –Test execution result
Defect Management
When a designed test script execution encounters a problem on the developed software you need to open a defect to manage this event.
In the past, we manually opened and managed defects on an additional tool called CMVC. This tool was a shared
point for tester, developer and product management to count and track all the defects encountered and resolved on the
project. Then, the tester manually inserted on the Notes DB called TTT defect id extracted from CMVC and associate it
to the test script. And also defect management was a problem since any change on CMVC has to be manually reflected
on TTT.
In RQM simply go the View Execution Results. Select the records and simply you can add the defect with a web
form.
Figure 13 –Defect definition
We decided to set up unified database access between Rational Quality Manager and Rational Team Concert so you
can create and track defects in Rational Team Concert. You can also create and track defects in the Rational Quality
Manager since the information is shared and no duplicated.
We usually open the defects from the RTC console for the CVT phase, while in RQM in the SVT phase. This because, CVT is mostly tracked on RTC since stories are linked to basic component test, while SVT is a complex and
large test cases list where RQM is the central tool, as seen in previous chapters.
Reporting Capabilities
We already talked about our tracking process. We move from Style-sheet document to the reporting capabilities of RQM and RTC. This mean a process evolution from static business analysis to a dynamic project indicators
evaluation, based on data warehouse query.
RQM gives a predefined list of built-in reports:
Figure 13 – List of reports
To run each of them you have just to select values from field in a mask and the output will be shown (Figure 14-15):
Figure 14 –Report generation
Figure 15 – Sample of report in RQM
The example above is based on the Execution Trend of our product Quality Assurance Phase.
But another built in report is very useful: we talked on the beginning of the article about the key value added from
clear evidence in linking Requirements to Test Cases. Here is a report coming from our product:
Figure 16 – Sample of report in RQM
Depending on analyst interest, you can look at information by requirements, tester, owner, test plan, or environment.
While in RTC, trough a BIRT programming customization, a project dashboard is showing product indicator to provide
daily status:
Figure 17 –TWS Dashboard
Test assets and Test environments management
In a Development Software Lab the test asset management gains a key role, indeed to ensure successful software deployment, it is crucial to have a stable environment dedicated to testing.
The Verification Team in Rome has a machine fleet composed by hundreds of physical and virtual machines. Each
team has tens of assigned machines. The mainly difficulties in managing a lab that size are described below:
Inventory: each machine (physical or virtual) has to be inventoried. For each machine is required to collect
technical information (hardware and software data) and business information (i.e. machine owner).
Keep track of machines updates: all the collected information needs to be up to date. Each change must be
notified. For example, an operative system change or owner change.
Sharing the machines data with stakeholders: all the machines data need to be shared with the stakeholders.
There are different stakeholders, with varying levels of interest: for example the lab manager needs to know
the high level machine information, a tester want to know all the configuration information about the machine
assigned to a project, etc. The kind of information to share is different based on the stakeholders.
Reservation: the resources should be reserved for any longer than is strictly necessary. We identified two reservation levels: project level reservation and team level reservation. The first is related to the reservation for a
project, when a project starts, a group of machines is reserved for it. The second one is related to the machines
reservation in the same team, a single tester reserves a machine for own test.
Definition, configuration and installation of test environment: during the test plan preparation, the test
leader defines the test environments. A test environment is a logical description of the environment required
for a specific test or group of test. When a tester needs to execute a test, he has to configure and install a physical environment. It’s very important to have an environment ready for the test in the shortest possible time.
Sharing the test environment data with stakeholders: when an environment is ready for the test, it’s important to share all the information about this environment with the other team component.
For all the issues identified above, RQM provides us a solution. In the following section we compare the solution
used before RQM adoption and the RQM solution.
Inventory:
In the past, several tools were used to collect the information about the IT assets: a tool collected the technical
data for physical machine, a tool collected the business data, a tool collected the data for virtual machines, a tool
managed the virtual machines request, etc. Mainly, the data were manually updated.
RQM, in the Lab Management section (Figure 18), provides an interface to extend and customize the inventory
capabilities. TPM provides an extension of RQM Inventory capabilities (TPM Adapter). TPM Adapter uses the
TPM discovery to find the machines (for example all the machines contained into an IP range).The collected data
are automatically imported into RQM.
In this way the information is always up to date.
The physical and virtual machines are described in RQM using the Lab Resource artifact.
Figure 18 –Lab Management tool
Keep track of machines updates:
In the past, each tool manages the data update in different way. Usually, the data were manually updated.
Using the Lab Management extension, for example the TPM Adapter, the information could be automatically
updated. The update request could be triggered manually or automatically.
Sharing the machines data with stakeholders:
As said previously, the stakeholders could be different. Different stakeholders are interested to different information.
So, another problem, using several tool is that each user needed several account and on each tool was defined
several roles.
Using RQM as central information access point, we assign different roles to different users. This is important
because the role associated with each account determines the options that are available and the functions that can be
accessed in the lab manager editors.
Reservation:
In the past the “project level reservation” was managed by a specific tool. The team level reservation was managed with sheet document or mails. The problems with this reservation management were: static management, each
team keeps the machines the entire duration of the project, also if the hardware machine is not required for the test,
each person was owner of some machines and he kept them also if he no longer used them.
Again, RQM allows us to manage such kind of information in a centralized way.
To reserve the machine is need to specify the time interval. The reservation management is transparent, all user
view the machine reservation status. It’s possible to integrate RQM with tools to know the effective use of a machine during a time interval.
Figure 19 shows a sample of reservation for a Lab Resource.
Figure 19 –Reservation of a lab resource
Definition, configuration and installation of test environment:
The steps to have a “ready to use” test environment are: definition of test environment (components and prerequisites), environment configuration, machines retrieval, machines reservation, middleware software installation and
product build installation.
In the past, each team leader was asked to design test environment associated to each test cases, in several artefacts like text document, or sheet document.
Again, after a review and approval phase such a document would be added on a Notes DB. Problems were: a
time consuming process to define, refine and maintain requirements list artefact; information kept isolated and
unlinked to other product artefacts, for example, there wasn’t a link between a logical environment (the logical description of an environment) and a instance of the test environment (a list of real machines and configurations that
compose
an
environment).
The retrieval of the machines in order to instantiate a test environment was done using sheet document, this document was manually updated. Problems were: keep the information up to date about the various test configurations
and keeping track of who is running what tests on which machines.
The environment was installed almost always manually. The test automation environments are installed using
automation, but they are predefined and preconfigured.
All test environment life cycle can be managed using basic RQM features and RQM extension.
The test environment can be defined using the Test Environment artifact. The tester defines all the environment
components. For each component, he can define the prerequisite for the component (for example a specific operative system, a required middleware…)
Figure 20 shows a sample of Test Environment definition.
Figure 20 –Sample of Test environment definition
When a Tester needs to instantiate a test environment, he defines a Test Cell associated with the specific Test
Environment.
In the Test Cell, RQM associates a lab resource for each component defined in the associated test environment,
if it’s available.
Figure 21 shows a sample of Test Cell.
Figure 21 –Sample of a Test Cell
The machines reservation could also be done starting from a Test Cell definition (Figure 22).
Figure 22 – Reservation of Test cell components
You can reference and link your Test environment and Test Cell with a Test Case or a specific Test Execution
Record (Figure 23).
Figure 23 –Relation between Test case and Test environment/Test Cell
You can link a specific Test Cell with a test script, so you can run the script on the specific environment.
The environment configuration (for example middleware installation) could be done using the integration between RQM and the Tivoli Provisioning Manager, called TPM Adapter.
The product under test could be installed using the integration between RQM and the Test Installation Suite
Automation, called Build Provisioning Adapter.
The Build Provisioning adapter (BP Adapter) uses RQM API to perform several operations programmatically
with the data in RQM repository (create, read, update, and delete the data in RQM repository).
Figure 24 shows a sample of operations provides from BP Adapter.
Figure 24 –List of BP Adapter operations
Sharing the test environment data with stakeholders:
The information sharing about the installed test environment was done using sheet document, manually updated.
Problems were: keep the information up to date about the various test configurations and keeping track of who is
running what tests on which machines.
As said previously, RQM API enables several operations with the data in RQM repository. The BP Adapter uses
the API to update automatically the test environment information after an install operation.
For example, if we use the BP adapter to install a specific TWS Component on a lab resource, the adapter updates automatically the Lab Resource information (Figure 25).
Figure 25 –Sample of software information automatically updated for a Lab Resource
Conclusion
In this paper, we have shown as the Rome Lab takes advantage of RQM Adoption. The RQM and RTC capabilities made possible to design a fully integrated architecture for Test and Development.
In the specific, RQM provided to the Verification team a unique tool to cover all test phases, based on integrative and
collaborative approach.
This change is not immediate, but the Rome Lab is changing little by little. Several points discuss in this paper are
underdevelopment or in pilot projects.
Next steps are related to the integration between RQM and other IBM products (official products but also internal
products as Test Automation Suite).
Resources:
The official site of RQM: http://www-01.ibm.com/software/awdtools/rqm/
The official site of RTC: http://www-01.ibm.com/software/rational/products/rtc/
RQM jazz communities: https://jazz.net/projects/rational-quality-manager/
RTC jazz communities: https://jazz.net/projects/rational-team-concert/
Developerworks site: http://www.ibm.com/developerworks/: IBM developerWorks is the industry-leading and
award-winning technical resource and professional network for the developer community.