Lessons Learned in How to Generate a Complete, Correct and... Requirements the First Time and Every Time

2009-01-0529
Lessons Learned in How to Generate a Complete, Correct and Usable Set of
Requirements the First Time and Every Time
Thomas E. Austin and Lori H. Runk
Thermal Systems Division, Delphi
James P. Waters
Powertrain Division, Delphi
Copyright © 2009 SAE International
ABSTRACT
From a quarter to one half of all projects that fail to meet
their imperatives of cost, performance / quality or
schedule are in some way associated with missing,
poorly written or misunderstood requirements (1). This
results in re-design, re-test and continual frustration to
both the originator and the user of these requirements.
Thus, a process for generating complete, consistent,
unambiguous and verifiable requirements is essential to
today’s automotive development process which focuses
on “fast to market” and “doing it right the first time.”
Lessons learned from evaluating the customer, internal
and supplier requirements specifications show that the
following requirement deficiencies regularly occur –
•
•
•
•
•
•
•
•
Hidden
Incomplete or unclear
Incorrect
Ambiguous
Missing
Unknown
Secret (competitive sensitive)
Unknown Correlations.
This paper will review the purpose of requirements,
characteristics of good requirements, and will provide
examples of what did go wrong in two automotive
projects, what deficiencies / problems occurred, and how
difficult it was to find and define some requirements.
INTRODUCTION – THE BASICS OF
REQUIREMENTS
A requirement is defined as an unambiguous and
verifiable statement of essential product or system
characteristics. IEEE Standard 729 further defines a
requirement in the following manner “A condition or
capability needed by a user to solve a problem or
achieve an objective. A condition or capability that must
be met or possessed by a system to satisfy a contract,
standard, specification, or other imposed document.”
Specifications are those documents which record sets of
requirements. A requirement typically has four essential
parts:
1. A description of performance –
• A function performed or a constraint on the
design of the product
2. A negotiated value –
• Defines the magnitude relative to a standard
measure
• Answers “the how much” and is usually numeric
• May include tolerances
The Engineering Meetings Board has approved this paper for publication. It has successfully completed SAE’s peer review process under the supervision of the
session organizer. This process requires a minimum of three (3) reviews by industry experts.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic,
mechanical, photocopying, recording, or otherwise, without the prior written permission of SAE.
ISSN 0148-7191
Positions and opinions advanced in this paper are those of the author(s) and not necessarily those of SAE. The author is solely responsible for the content of
the paper.
SAE Customer Service: Tel:
877-606-7323 (inside USA and Canada)
Tel:
724-776-4970 (outside USA)
Fax:
724-776-0790
Email: [email protected]
*9-2009-01-0529*
SAE Web Address:
http://www.sae.org
Printed in USA
3. The conditions under which it applies –
• Ambient conditions (e.g. temperature, humidity,
vibration)
• Operating conditions (e.g. moving, stationary,
parked)
• Pre-conditions (e.g. burnished brake pads)
4. The requirement verb indicating the degree of
obligation
• SHALL or MUST (European standard) - are
used for contractually binding provisions that
must be verified
• SHOULD – denotes preference when specifying
alternatives or desired conformance with
objectives, standards or procedures
• WILL – used to state requirements associated
with the development of the product and not the
product itself
Note that a defining characteristic of a requirement is
that it must be verifiable. Typical means of verification
are:
•
•
•
•
TEST – collection and analysis of data from a
controlled test
DEMONSTRATION – verification that an event
occurred as designed
INSPECTION – a sensory examination is performed
and compliance achieved
ANALYSIS / SIMULATION – use of applicable data
from previous programs, or an examination of data
using engineering judgment or mathematical
calculation
Verification is usually documented through means of a
verification cross reference index (VCRI) or compliance
matrix that is included within the specification. A
personal observation is that no requirements
specification should be issued without an accompanying
VCRI. This ensures that the specification author has
completely thought through the requirement by
understanding its verification.
When writing a requirement, there are characteristics of
a good requirement which should be considered (2).
•
•
•
•
•
•
•
•
•
Verifiable – by test, demonstration, inspection or
analysis
Necessary – essential to complete its meaning
Unambiguous – there is one and only one meaning
Implementation Free – the requirement is written in
such a manner that there is no preferred solution
Complete – all parts of the requirement are present
Concise – written succinctly
Consistent - compatible with all other requirements
Traceable – follows from other higher requirements
Feasible –able to be implemented, practical.
BAD REQUIREMENTS - PROBLEMS AND
IMPLICATIONS
An earlier SAE article (3) describes and illustrates the
problems that result from requirements with deficiencies
as listed and described in the Abstract above. In that
article, four complex product projects were examined
using the principles and processes of Six Sigma.
Twenty-five percent of the “What went wrong” were
associated in some way with requirements or their
verification as described by the Systems Engineering
(SE) process. These problems significantly impacted
their respective project’s cost, schedule and
performance. While many of these problems were
associated with the initial Customer requirements
statements, there were other problems which occurred
throughout the project’s duration. Interestingly, each of
these four projects independently listed “lack of
understanding customer requirements” as a primary
cause.
The content and purpose of the remainder of this paper
is to present typical examples of requirement
deficiencies found in two automotive projects as
analyzed by their respective systems engineers. In the
first case, Lori Runk presents her experience as she
worked to understand customer requirements and the
System Engineering (SE) tool she used to clarify and
improve those requirements. In the latter example
presented by James Waters, he found that hidden,
unspoken, and unrecorded requirements had to be
found by working closely and diligently with the customer
and suppliers.
When both are considered together, the reader can see
that the development of initial requirements was not
sufficient. The total scope of a project must be carefully
considered, such that all requirements are understood,
fully identified and properly documented. In summary,
the authors will offer lessons learned that the reader can
use to generate a complete, correct and usable set of
requirements. When applied to any process, this will
result in requirements that are correct and usable the
first time. This will save cost and time for corporations
and projects. It will result in meeting or exceeding
customer expectations.
THE VALUE OF THE SYSTEMS ENGINEERING
(SE) METHODOLOGY CAN BE SEEN IN A REAL
WORLD EXAMPLE
The value of the SE methodology can be seen in a real
world example. Delphi Thermal Systems designs,
develops, validates and manufactures high quality
HVAC (Heating, Ventilation and Air Conditioning)
modules for many automotive manufacturers worldwide.
In some cases they are design responsible for the whole
integrated HVAC subsystem in the vehicle. In other
cases they are design responsible only for the HVAC
module – one component of the vehicle’s HVAC
subsystem. The HVAC module is a package which
generally consists of:
•
•
•
•
•
•
•
plastic cases
heater core
evaporator core
blower motor and fan
valves which direct airflow
electric actuators for the valves
etc.
Customers provide Delphi with the requirements for the
HVAC subsystem. Delphi experience and expertise is
used to provide a product which meets and exceeds
customer expectations. Delphi experience and expertise
is also used to make sure they design, develop, validate
and manufacture a product which meets Delphi Corp.
business objectives. In other words, they are part of a
business which is expected to make a profit!
In order to address the problem, Delphi Thermal
System’s management assigned a temporary resource
to clarify and come to a mutual understanding of the
product design and validation requirements. In order to
streamline the process, it was decided that they would
approach this problem from a SE methodology
perspective, using some of the Delphi SE common
process tools.
If the SE “V Diagram” is reviewed, it can be seen that
the starting point for the entire process begins with
CUSTOMER REQUIREMENTS.
One of Delphi’s recent HVAC module product
development projects was not proceeding smoothly. The
customer was getting frustrated because they did not
feel Delphi was fully addressing their stated
requirements. Delphi was getting frustrated because the
customer seemed to be frequently changing design
direction. Delphi was asked several times to rerun
testing which had already been completed. In other
words, Delphi had an unhappy customer. The profit
numbers for the project were diminishing because of the
redesign and rework activities occurring in engineering.
These requirements are defined, analyzed, developed,
and allocated from system to subsystem to components.
In addition, all of the work on the right side of the “V
Diagram” deals with the validation and verification
activities associated with meeting those requirements.
The diagram shows that a successful project needs to
start with good, clear complete requirements. In this
way, those requirements can be correctly defined and
allocated to lower level parts. Then valuable, meaningful
validation activities can produce a product which meets
or exceeds those requirements. If this same information
is viewed from a negative perspective, it can be seen
that if a project does not start with good, clear complete
requirements, there will not be valuable, meaningful
validation activities. The end product will not make the
customer happy. If a project does not begin with good,
clear complete requirements, time and money will be
wasted trying to develop and validate a product that in
the end may or may not meet customer expectations.
By working cooperatively with our customer to address
the above deficiencies /issues, Delphi was able to bring
the HVAC module development project back on track.
This provided customer satisfaction and profitability for
Delphi.
THE BASICS OF THE RAR TOOL
Based upon the above information, Delphi Thermal
System’s engineers hypothesized that the source of the
problems with the HVAC module development project
was a misunderstanding of customer requirements. In
order to determine if this was truly the source, it was
decided to evaluate those requirements using a SE
common process tool called a Requirements Analysis
Report or RAR.
The RAR is a tool which provides the words, format and
process for obtaining clear, reasonable requirements
from the customer. It helps to identify deficiencies and
problems as well as their associated risks, so that
actions and mitigation planning can be initiated. Once
the customer requirements have been defined, this tool
can be used internally to evaluate product requirements
before they are passed down to subcomponent
suppliers.
The results of the RAR were eye-opening! Using the
RAR tool Delphi identified a total of 288 customer
requirement line items. The SSTS (subsystem technical
specification) RAR results summary chart below lists the
five
common
weaknesses
discovered.
These
weaknesses were associated with 194 of the 288 line
items. The chart also shows how many line items were
associated with each of those weaknesses.
The RAR is a simple spreadsheet which is used to
manage the work required to analyze customer
requirements. The RAR is populated using the following
steps:
1. LIST ALL CUSTOMER REQUIREMEMTS:
• Every requirement paragraph from the CRD
(Customer Requirements Document) is loaded
onto the spreadsheet. The “Requirement
Number” & “Requirement Title” are kept
separate so they can easily be sorted. In this
format, they can also be easily copied to a
validation test planning document.
• The “ORIGINAL WORDING” of the requirement
is copied into the spreadsheet cell. This is not a
required field, but it can be a time saver
because it :
• prevents one from switching back and forth
between the CRD and the RAR when
discussing a certain technical requirement /
paragraph.
• prevents misinterpretation of a requirement
based upon the “Requirement Title” alone.
•
List
any
ADDITIONAL
CUSTOMER
REQUIREMENTS you have been given. These
requirements are not found in the main CRD.
They may have been communicated verbally in
a customer meeting, or by e-mail, or through
some other “Statement of Requirements”
document received from the customer. Make
sure ALL KNOWN product design related
customer technical requirements are captured.
•
•
2. EVALUATE EACH CUSTOMER REQUIREMENT:
•
•
After reading the ORIGINAL WORDING, judge
the quality of the requirement. Make sure it is a
good, clear complete requirement.
Before setting up the spreadsheet, review the
characteristics of a good requirement as listed
above in “Introduction -The Basics of
Requirements”. Then pick the characteristics
which are most critical to the project.
3. PROPOSE AN INTERPRETATION FOR EACH
REQUIREMENT
•
•
•
•
•
misunderstood? Can there be only one
interpretation of the requirement?
Complete – Is the information stated with all
of the details necessary to understand the
requirement and / or to run the test?
Requirements with “to be determined” are
not complete.
Verifiable - Is the requirement stated clearly
so it can be verified by defined subjective or
objective measurements?
Does the
verification method and equipment exist
within the company? Must and an outside
resource be used? A requirement is NOT
verifiable if the methods and/or equipment
required are inaccessible due to cost,
geographic location, or other limiting factors.
If a requirement meets all of the evaluation
criteria, it should remain as stated in the original
wording. An interpretation is not necessary.
If a requirement does not meet the evaluation
criteria, it needs to be interpreted (or restated)
until it does meet the evaluation criteria.
Consulting reference documents and subject
matter experts within the organization and at the
customer’s organization may be required in
order to develop a proposed interpretation.
The RAR spreadsheet includes a “COMMENTS”
section so any notes or references can be
recorded which might be helpful when
discussing the reasoning behind a proposed
interpretation.
For the Delphi HVAC module, the “ORIGINAL
WORDING” was reviewed to see if the
requirement was:
•
•
HVAC Module Applicable (Component
Applicable) - is the requirement written in
such a way that it applies to the part to be
developed and validated and can it be
verified at that part level? Delphi is
responsible for the design of the HVAC
module, but many of the requirements are
specified at the HVAC subsystem level. In
order to validate the HVAC module,
requirements must be stated at the module
level.
Unambiguous – Is the requirement
completely
clear
and
cannot
be
4. FINALIZE AN INTERPRETATION FOR EACH
REQUIREMENT
It is important to remember that all of the
requirements must come from a customer. A
proposed interpretation isn’t valid unless the
customer agrees with it.
•
•
•
Review the proposed interpretation with the
customer and make any changes necessary to
obtain a customer’s agreement.
Make sure the final wording still meets all
original evaluation criteria.
Use the “COMMENTS” section to record the
date of the discussion and any significant
customer comments or clarifications associated
with the final interpretation.
5. FILE THE FINAL CRD
When the RAR is complete, it becomes the new and
final customer requirements document for the project.
It contains the:
•
•
•
Requirement Number
Requirement Title and
final wording or Interpretation
for every customer requirement.
It is the list of requirements which must be used
internally to manage the development and
validation of the product. This document must
be
saved
and
then
distributed
and
communicated to all of the project team
members so that work can proceed for:
•
•
•
Designing and validating the product
Estimating the cost and investment
associated with the project
Allocating and developing requirements for
lower level subcomponents.
IDENTIFYING AND UNDERSTANDING BASIC
REQUIREMENT FLAWS
As stated earlier, the results of the Delphi Thermal
Systems HVAC module RAR were eye-opening. Out of
288 lines items, a total of 194 did not meet the
evaluation criteria for being component applicable,
unambiguous, complete and verifiable. In other words,
over 67 percent of the stated requirements were not
readily usable for developing and validating a high
quality product. No wonder the project was struggling.
After completing the work, it was decided to take a step
back to try and figure out why the requirements were so
inadequate. The review provided key insights into the
basic flaws existing in any requirement document;
deficiencies which could be easily identified in the future.
Analysis of the data led to the identification of five basic
recurring requirement flaws. These flaws are listed
below.
In the following section, each basic flaw will be defined
and a simplified real life example will be given.
HIDDEN REQUIREMENTS
Hidden requirements are requirements which have been
established and do exist, but in order for the user to
define and understand them, he/she must do research.
Hidden requirements require the user to obtain, and
read supplemental reference documents. Hidden
requirements may require the user to find and contact a
subject matter expert in order to clearly and completely
interpret what is required. Hidden requirements are
problematic because they can be ignored if suppliers
don’t have the resources necessary to perform the
research. Hidden requirements waste time and money
because research is performed to find and define
something which is already known but not properly
documented.
In the HVAC module project, a total of 28 requirements
were classified as “Hidden”. One of those requirements
was associated with governmental regulations. The
original wording of the requirement seemed
straightforward, but further investigation led to research
that lasted many months. For purposes of this example,
the original wording of the requirement is represented by
the following:
“The legal requirements of all market
countries specified in Reference Document
#1, section 3.0 shall be met.”
The first thing Delphi had to do was find and read
Reference Document #1. This document stated that the
product had to be “in compliance with all applicable
governmental requirements” and a list was attached
which identified 30 countries with multiple documents
sited for each country. The requirement was still hidden
because it was not believed that the product was being
used in all 30 countries. Additionally, the titles of some of
the referenced documents did not seem to apply to the
product being evaluated.
The second step of the research involved contacting the
customer’s release engineer to obtain a confirmed list of
all of the applicable countries. The engineer sent an email which resulted in the identification of 11 applicable
countries. Unfortunately, there were 65 documents
associated with those countries. Delphi ended up having
to obtain the documents and read them before the list of
HVAC module applicable requirements could be
identified. In the end, after six months of time and with
the assistance of several supplemental resources,
Delphi and the customer identified 17 governmental
requirements which needed to be met.
+35°C. Valve #1 ≤ 20 l/s, valve #2: ≤ 30 l/s, case to case
interfaces ≤ 40 l/s. To someone unfamiliar with HVAC
module performance and technology, this requirement
seems to be very simple and clear. Unfortunately, those
who are familiar with testing HVAC modules, will have
many questions:
•
•
•
Time and money can be saved for both the customer
and the supplier if requirements are clearly stated
without the use of references to supplemental
documents. When references are required, due to the
length and complexity of the document, those references
should be attached to the original requirement
document. When multiple requirements are listed, only
those which apply to the specific product and its
application should be identified.
INCOMPLETE or UNCLEAR REQUIREMENTS
Incomplete or unclear requirements are requirements
which have only partially been established. Some, but
not all of the information necessary to understand the
requirement and/or to run the test has been provided. In
order for the supplier to define and understand the
requirement, research and developmental testing may
have to be performed. Incomplete or unclear
requirements force the supplier to find and contact a
subject matter expert, and the details of the requirement
may have to be developed and negotiated with that
expert.
How should the test be set up? Where and how
should the leakage measurement be taken?
What air pressure should be applied? (Leak rates
depend upon the force of the air applied to the
interface.)
What test temperature increments should be used?
Is the customer just interested in the leak rates at
the temperature extremes? Is it necessary to test at
a mid-temperature value? Does the customer want
45 tests run in one degree increments?
The second step of the research involved several
conference calls between Delphi’s HVAC module
development engineer and the customer’s development
engineer. Both had experience with leak requirements
and testing, but neither had a previously established test
or set of requirements which could be copied. These
engineers worked together to define what was really
important about HVAC module leak performance and
then they defined a test to evaluate it. The resulting
requirement included eight defining sentences without
any external references. In those eight sentences, two
different leak tests were described and one of the first
three leak requirements was dropped. The work took
approximately two weeks to complete. Once the original
requirement wording is replaced by the new
interpretation in future requirements documentation, the
product will be developed properly. Time will not be
wasted by either the customer or the supplier engineer.
In the HVAC module project, a total of 83 requirements
were classified as incomplete or unclear. In many
instances the missing information was quickly and easily
defined. In other instances, time and money had to be
spent in order to clarify the requirement. One of those
requirements was associated with air leaks. The original
wording of the requirement seemed straightforward, but
further investigation led to research and test
development which lasted two weeks. For purposes of
this example, the original wording of the requirement is
represented by the following:
UNREALISTIC or UNCOMPETITIVE REQUIREMENTS
“The product shall meet all air leak requirements as
detailed in Reference Document 70, section 14.3Leak Testing.”
In most cases unrealistic or uncompetitive requirements
are not discovered until designs have been established,
tests have been run, and the requirement has not been
met. Once the requirement has not been met, attention
and resources are focused on the product and work
begins again to figure out how to meet the requirement.
The requirement itself does not get re-evaluated until it
becomes clear that it will be very costly and/or time
The first thing that had to be done was to find and read
Reference Document #70. This document stated that
“the product shall not exceed the stated air path leak
values when tested at temperatures ranging from -10 to
Unrealistic
or
uncompetitive
requirements
are
requirements which have been established, but don’t
represent realistic usage conditions or environments.
These requirements may:
•
•
•
•
Increase test time
Increase test complexity / cost
Create unnecessary design constraints
Drive unnecessary design improvements
consuming to meet it. In situations where the
requirement is unrealistic or uncompetitive, it eventually
gets changed.
In the HVAC module project, there were seven items
identified as being unrealistic or uncompetitive. If these
requirements had been identified and addressed in the
early stages of the product development, months of
customer and supplier engineering and project
management hours would have been saved.
One of these requirements defined valve actuation time.
The original requirement defined the maximum amount
of seconds it should take for a valve to move from
position A to position B, when exposed to a certain cold
temperature and with a minimal voltage input to the
actuator. Design validation testing was completed and
re-run for verification. It was determined that the product
did not meet the requirement. In fact, it missed the
requirement by more than six seconds.
The results of the testing were reported to the customer,
and focused resources were put in place to evaluate
design improvements and alternate actuator suppliers.
More tests were run. Finally, options were listed and
cost and timing estimates were reviewed. It was
determined that the only way to meet the requirement
would be to significantly increase the product cost and
delay program timing. It was also discovered that the
customer’s control architecture had a direct affect on the
ability of the Delphi product to meet the requirement. At
that point, the team began to seriously question the
validity of the requirement.
Actuation time for HVAC module valves is not
associated with new product features being offered to
consumers:
1. Why was this requirement so difficult to meet?
2. Do valves actually have to actuate under the
combined stress of both extremely low temperatures
and low voltage inputs?
3. In the most extreme situation of cold temperatures,
does the valve have to move that fast or does it just
have to move?
4. What is a realistic, real world, worst case operating
condition and how fast does the valve need to move
when exposed to that condition?
All of these questions were reviewed with Delphi HVAC
module benchmarking experts and the customer. It was
determined that the original requirement was not
realistic.
After more than six months of work, a new requirement
was established which set a low temperature value 10°C
higher than the original, and combined it with a low
voltage value four volts higher than the original. In
addition, at those conditions, it was determined that the
valve needed to actuate in 10 seconds, not five.
In this situation, an unrealistic requirement was identified
because product testing showed that the requirement
couldn’t be met due to application specific constraints.
Vehicle evaluations and benchmarking showed that the
original requirement did not need to be met. In other
situations, unrealistic or uncompetitive requirements are
questioned and identified when it is discovered that
product testing has become excessively complex and/or
validation costs are estimated to be excessively high.
Unfortunately, some unrealistic or uncompetitive
requirements are never discovered and/or never
changed. In those cases, unnecessary design
constraints are placed on the product. Unnecessary and
costly design changes are implemented when there is
no additional value provided to the consumer. In the
most unfortunate situations, efforts to meet unrealistic or
uncompetitive requirements may force an engineer to
choose a design option which compromises the
product’s ability to meet a separate and realistic
requirement which a consumer would want.
INCORRECT REQUIREMENTS
Incorrect requirements are requirements which have
been established, but they are wrong. These
requirements are not usually obvious and they remain
hidden until errors in the design and/or validation
activities are discovered. Incorrect requirements drive
changes to requirements and/or designs. Depending
upon when they are discovered, incorrect requirements
may:
•
•
•
Drive retesting
Delay timing
Force compromises.
In the HVAC module project, there were 54 items
identified as being incorrect. Most of those were minor,
but a few of them caused significant problems for the
development and validation of the product. As with the
unrealistic/uncompetitive requirements, if the incorrect
requirements had been identified and addressed in the
early stages of the product development, months of
customer and supplier engineering and project
management hours would have been saved. One of
these requirements defined the voltage point required for
developing and evaluating the maximum airflow
performance and maximum airflow noise. For the
purposes of this example, the term X voltage point will
be used.
At the beginning of an HVAC project, the module level
requirements are reviewed and then allocated to lower
level components. When an HVAC module is developed
to meet its maximum airflow performance at a specific
voltage point, the HVAC blower motor is also developed
to meet the maximum airflow performance at that same
voltage point. All other blower motor characteristics,
such as amperage draw, are determined based upon the
voltage input and the required maximum performance
output. For this project, the blower motor maximum
airflow performance set point was the X voltage point.
When Delphi engineers were well into the design
validation of the HVAC module, a complaint was
received from the customer. The customer was
experiencing extremely high amperage draw from the
blower motor when they operated it in the HVAC module
at their maximum voltage point. Delphi didn’t understand
this complaint until it was discovered that the customer
only wanted to develop the design of the module at the
X voltage point. The customer actually wanted to
operate the module at an X+U voltage point. The
customer had not specified a maximum operational
voltage in the requirements document, and had not
taken the mathematical relationship between HVAC
blower motor voltage and amperage draw into account
when module level requirements were first established.
By the time the error was discovered, it was too late to
completely redefine the blower motor and HVAC module
design. Options were discussed and a compromise was
reached. The module would operate at a maximum
blower voltage higher than the development point but
lower than the originally requested operational point: the
X + (U-N) voltage point. The new requirement was
considered to be acceptable, but it was not what the
customer originally wanted. If this incorrect requirement
had been identified earlier in the project, it would have
been easy to meet the desired X+U voltage maximum
operating condition.
NOT CORRELATED REQUIREMENTS
Requirements which are not correlated, are
requirements which have been established, but have
only been established for a part which is at a higher level
of complexity than the part which needs to be validated.
For example: looking at the SE “V” diagram: a
requirement could be established only at the “System”
level even though the part which needs to be validated is
at
the
“Product/Subsystem
level.
For
these
requirements, a technical relationship or link between a
requirement for the part to be validated and the stated
requirement has not been developed. In addition, the
acceptance or approval of the part is partially or
completely dependent upon other components and other
design changes over which the supplier does not have
control. Project management involving non-correlated
requirements is difficult because the test results and
timing are dependent upon design changes and test
samples beyond the control of the project manager.
Even more significantly, profit management is difficult
because design change responsibility is not clear cut.
Was the requirement not met because of a problem with
the supplier’s part? Was it caused by a problem with the
additional components required by the testing?
Non-correlated requirements are usually difficult to fix.
They require development. Tests have to be designed
and data has to be taken. Proof has to be provided
which clearly demonstrates that meeting a new
requirement at a component level is equivalent to
meeting the originally stated requirement at a subsystem
level. If correlations were obvious or easy to develop,
the requirement would have been initially written at the
component level. Because of this, when correlations are
discovered, they can become intellectual property. They
can become technical or competitive advantages.
Unfortunately, customers always ask for proof of
correlation. Project timing delays will occur when one
takes the time to protect the intellectual property before
it is shared with the customer.
In the HVAC module project, there were 29 items
identified as not having correlation. One requirement
which causes program and profit management
difficulties at Delphi is airflow performance. Delphi is
given requirements for the temperature and quantity of
airflow which must come out of each of the vehicle ducts
– the vents, the defroster and the floor heaters. Delphi
controls and is responsible for the air which comes out
of the HVAC module, but that air has to flow through
vehicle ductwork in order to get to the passenger
compartment and windshield. This means that the
module has to be developed and tested with vehicle
ductwork attached. Ducts have to be ordered and testing
scheduled based upon duct availability. If the design of
the duct changes, retesting must be run and more
design changes often follow. Customers understand this
difficulty and try to be cooperative. However, issues
always arise because two different companies are trying
to manage part procurement as well as design change
timing. In the end, the Delphi HVAC module may not get
approved if airflow performance requirements are not
met. This is true even if the reason for the problem lies
within the ductwork. When HVAC module changes are
required in order to meet airflow requirements, it is not
always clear what the driving force is behind the change.
Was the module deficient initially, or did something
change in the ductwork? This type of discussion always
takes place when the supplier and the customer try to
decide who is going to be financially responsible for the
design change.
SE METHODOLOGY IS EQUALLY VALUABLE IN
THE POWERTRAIN ENVIRONMENT
The SE methodology is equally applicable to the
powertrain arena. Even in organizations that have not
officially implemented the SE common process, its value
can still be realized by using many existing, incumbent
documents and methodologies.
They may have
different names and slightly different formats, but are
functionally equivalent to the official SE process
elements.
Airflow performance is a science, but it isn’t always
straightforward. In order to define a requirement for the
component, correlations to the subsystem have to be
developed.
When working with requirements which are not
correlated, engineers and managers have to decide how
to allocate their resources. Should the resources be
spent trying to develop a correlated requirement, or
should the resources be spent trying to better manage
and track parts and design changes? Managers have to
decide how much technical and financial risk is
associated with trying to obtain approval and payment
for a component, when its approval is tied to additional
subsystem parts which were not part of the original
agreement.
THE HVAC EXAMPLE - SUMMARIZED
The Delphi HVAC module example clearly shows how
important it is to start a project with good, clear,
complete technical requirements.
The discussion
demonstrates the value of SE Methodology and powerful
tools like the Requirements Analysis Report. It also
provides the definition of five terms which can be used to
better understand and communicate the weaknesses
associated with inadequate requirements. Those
weaknesses can be addressed and the requirements
can be improved.
SE (Systems Engineering) is defined as a formal
process for the development of a complex system,
driven by a set of established requirements, derived
from the intended mission of the system throughout its
life cycle. Experience teaches that this includes the
unintended, but nonetheless valid, mission requirements
imposed by the individual life cycles of the elements of
the system, and their habitat. It is through the relentless
pursuit and successful documentation of these other
requirements that SE brings its value to the system
development process.
The second example is the application of the SE
common process methodology in the development and
integration of Delphi camshaft phasers into a complex
engine control system such as the Dual Independent
Cam Phasing (DICP) subsystem of a typical Double
Over Head Camshaft (DOHC) V6 engine. Delphi is
supplying both the cam phasers and the cam phaser oil
control valves (OCV’s) that must work properly with the
incumbent base engine mechanicals, hydraulics,
electronics, software and control and diagnostics
calibration.
In a typical engineering program we may have:
• one cam phasing subsystem
• two components; cam phasers and OCV’s (four
of each: right bank, left bank, intake & exhaust),
• three Delphi engineering sites including
Henrietta NY, Luxembourg & Juarez Mexico,
and
• two Delphi manufacturing sites Grand Rapids,
MI and Chihuahua, MX,
that must work in synergy with:
• two or more customer engineering & test sites
and
• two or more customer engine manufacturing
sites and
• their associated off-site kitter’s and
• one or more vehicle assembly sites,
often spanning up to 13 different time zones and two or
more native languages.
All of these elements must work together to meet the
customer requirements and expectations. The existence
of a clear, concise, unambiguous and complete set of
requirements effectively eliminates corporate, legal,
contractual, language and cultural barriers that might
otherwise render such a complex arrangement
unworkable.
The V diagram from the SE methodology illustrates the
requirement development flow from system level
customer requirements to component specifications to
verifiable test specifications.
Requirements
Requirements
Allocation
Allocation
System Integration,
System Integration,
Test and Verification
Test and Verification
Vehicle Test Plans/Procedures
Systems Engineering
Vehicle
VehicleRequirements
Requirements
Requirements
Requirements
RR
Subsystem
SubsystemRequirements
Requirements
Allocate
Requirements
Validation Testing
Integrate the system into the
vehicle; System C/O procedure
Component
ComponentRequirements
Requirements
Design
Component
Component
ComponentDesign
Design
Functional Test Plans/Procedures
Verification / Validation
Analysis Report
Interface Matrix System Testing
Functional Block Diagram
ws
eviews
er Revie
n/Peeer R
esign/P
DDesig
Requirements
Bench, Instrumentation, Analysis Requirements
PRR
•Customer Interface Control Documents
System
SystemRequirements
Requirements
Allocate
Vehicle
VehicleVerification
Verification/ /
Validation
Validation
Customer Requirements Document
•Customer and Internal Interface Control Documents
Integrate the subsystems into
the system application
CDR
Product Definition Document
Verification / Validation
PDR
Subsystem Testing
Conduct
ConductSubsystem
SubsystemVerification
Verification
/ /Validation
Validation
Subsystems Requirements
Specifications
Verification / Validation
System
SystemVerification
Verification/ /Validation
Validation
DDeessign
ign/P
/Pee
eerr Rev
Rev iew
iewss
Allocate
Product Delivery Letter
Design Validation Plan and Report
Vehicle Testing
Management Plan
IDR
Component Testing
Component Requirements
Specifications
Integrate the components into the
subsystemComponent/ SS Verification
and Integration
Component/
Component/Subsystem
Subsystem
Verification
Verification/ /Validation
Validation
Integration, test and verification
planning and procedure generation
Verification
Design
DesignVerification
Verification
Design Analysis & Test
Integration and Test planning/procedures
Design Optimization
Design Optimization
and Product Builds
and Product Builds
BUT HOW DOES THE COMPONENT GET INTO THE
SYSTEM?
Dealer Service Bay
com What d
p
o
its j onent e es you
our
r
x
ass ney in perienc
em b
e
t
led o the f on
i
c
n
o
a
pro
duc nsume l
t?
r
Your
component
must endure
many
undocumented
environments,
creating
hidden,
unknown,
undocumented
, and even
accidental
requirements
Vehicle Plant End of Line
Dynamic Test
Engine Plant recycling of
parts from engines that
failed Cold Test
Engine Plant Cold Test
3rd party kitting
Shipping & transit
Dunnage
The SE approach to a typical variable cam phasing
(VCP) program can be summed up as the relentless
pursuit of requirements:
ƒ explicit
ƒ implicit
ƒ implied
ƒ unspoken
ƒ unknown
ƒ hidden and
ƒ unrecorded.
The product and system requirements and specifications
included in the quote package and commercial contract
documentation are a vital part of the requirements
documents, but often these documents only represent
the beginning of the SE journey.
Much like a field researcher, Delphi examines
the life cycle and habitat of the individual
product, before, during and after its time spent
in the customer system.
Looking at life cycle leads to requirements from the
underhood applications environment, as well as from
customers other than the main customer (customer
design release engineer):
• Dunnage
• third party kitters
• unplanned extended storage in uncontrolled
thermal and humidity environments
• ergonomics engineer at engine assembly
• cold test (motored engine test)
• any in-plant reclaiming and reusing of good
parts from engines that fail cold motored engine
tests for unrelated reason
• hot test (fired engine test)
• third party marriage house that mates engines to
transmissions prior to final assembly into
vehicle, often in close geographical proximity to
the vehicle assembly plant
• vehicle plant end-of-line test
• marshalling yard
• car hauler
• rail cars
• dealer service bay and
• recycling
all in addition to vehicle owner/operator and
governmental legislated emissions, fuel economy and
safety requirements, as well as European safety and
recycling laws.
Looking at Habitat gives us the applications
environment.
The typical DOHC V6 Engine’s Variable Cam
Phasing Application Environment
ƒ
ƒ
ƒ
ƒ
Variable Cam Phasing, or VCP is a hydraulic
system, so fluid power (oil pressure and flow
rate) is the primary motive force.
The oil pump is speed driven, making rpm
important.
Oil viscosity is temperature driven, so ambient
as well as operating temperatures are important.
The OCV relies on voltage, even though the
software compensates for voltage excursions,
system voltage and voltage at the OCV are vital.
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
Oil aeration characteristics of the engine as
installed in the vehicle (rpm-based aeration and
foaming and longitudinal and lateral G-force
driven sloshing of the bulk oil).
Operating schedules
The speed-load operating schedule, often
referred to as the fly zone
Engine
dynamometer
performance
and
calibration schedules
Engine dynamometer durability and thermal
shock test schedules
EPA, CARB, Japanese and ECE and EU
Emissions Standards
Engine plant motored-engine cold test
Engine plant fired-engine hot test
Oiling (lube) system – fluid power
Lube circuit
Oil pump
Measured oil pressure (typical)
Oil pressure extremes (-40oC cold start)
Different oil types and viscosities
Position Stability equals control effort minus cam
torques
Cam torques
Cam torque disturbances introduced by camdriven accessories, such as gasoline direct
injection fuel pump
Measured position stability
Within Delphi Powertrain Valve Train Engineering, the
vast majority of SE deliverables reside in the product,
process, test and validation activities.
Hidden
requirements are often embedded somewhere in these
test activities, often in an otherwise simple test schedule.
For example, an engine-mounted component may only
be exposed to 12 g’s of vibration in-vehicle, but may
experience up to 20 g’s on a dynamometer test due to
the differences in engine mounting rigidity. Regardless
of the easier in-vehicle requirement, the fact remains
that if the component cannot survive a required dyno
durability test, it will never make it into production.
Therefore it is faced with a de-facto vibration
requirement of 20g’s for successful application into a 12
g’s vehicle environment.
For the typical Delphi Valve Train customer engine
program, the main documents are the CPTR (Customer
to Product Technical Requirements) and the SFMEA
(Systems Failure Mode & Effects Analysis). The CPTR
is the pre-existing document that is functionally
equivalent to the RAR (Requirements Analysis Report)
from the SE Common Process.
Progress towards completion of each document is
tracked and major concerns reported at each design
review. Names for these reviews and the names of the
requirements documents may vary.
Customer to Product Technical Requirements
The Customer to Product Technical Requirements
document (CPTR) is the pre-existing document that is
functionally equivalent to the Requirements Analysis
Report, or RAR. Other organizations may have a
Component Requirements Document, (CRD), or a
Customer Component Checklist (CCC).
Most
organizations already have a pre-existing document that,
with minor adjustments, can fill in for the RAR until the
organization adapts the SE Common Process. The
official statement of requirements provided with the
customer contract is the best place to start when
populating this document, but it is imperative to look
everywhere possible in the relentless search for all of
the requirements. Below is a sample page of a generic
CPTR, as well as a graphic showing what other
documents flow their requirements into the CPTR.
Documents & functions captured in the CPTR
(known in the SE common process as the RAR,
Requirements Analysis Report)
Customer
Requirements
Design
Validation
Test
Results
Design
Confirmation
Test Results
Vehicle-Level
Test Results
Vehicle Test of
Hi Leak VCP System
Customer
Requirements
reduced to
phaser-level
requirements
CPTR
(RAR)
Cam Phaser
Bill of Design
Vehicle Test Plan
Cam Phaser
Bill of
Process
Applications
Environment
Customer
Verifies CPTR Changes
Systems Failure Mode & Effects Analysis (SFMEA)
Just as the design and process engineers have their
DFMEA and PFMEA, the Systems Engineer performs a
SFMEA. The document is the noun, but the systems
failure mode and effects analysis process is a verb, an
action verb. In the process of performing the SFMEA,
many other documents feed into it. For those who use
Microsoft XL, it is relatively easy to add additional
worksheets to the SFMEA document. This makes the
SFMEA a single point repository for the many other
documents that might otherwise lead the reader on an
Easter egg hunt, often resulting in hidden requirements.
As future programs will reference prior work in their
“lessons learned from prior programs” search, this is a
great place to document these hard-won lessons
learned and tribal knowledge so often lost, and then relearned at great cost to all involved. Below is a graphic
showing what other documents are fed into a rigorous
SFMEA.
Documents & functions captured in the SFMEA
SIDI Fuel Pump
Cam Torques
Cam Phaser “fly zone”
Base
Engine
Cam
Torques
Control-hold stability
OEM’s coverage of Highlevel RPN’s identified by
SFMEA
Hybrid Cranking Speed
Functional Map
Base Engine Oil
Pumping
Characteristics
Functional Outline
Free Body Diagram
SFMEA
VCP Circuit
Function Block
Diagram
Engine Plant
(motored) Cold
Test
On-car evaluation
procedure
OEM Hot (fired) Test
procedures
Base Engine
Lube Circuit
Diagram
Exploded Diagram
of Cam Phaser
VCP Circuit
Study other programs at that same customer and
interview domain experts for hidden requirementsPrevious customer programs are a great place to look
for hidden or undocumented requirements inherent in
every aspect of their enterprise in which your product will
come in contact. Most technology-driven requirements
are consistent, even between L4, V6, and V8 engines.
Even if a program is truly a first, it is still worth “data
mining” previous customer programs involving similar or
analogous technologies for valuable lessons learned
and undocumented customer requirements.
For example, in the early 1980’s, Delphi was able to
bridge between the extensive carburetor body of
knowledge and the then-new throttle body (or central)
fuel injection system by equating the new fuel injection
software functions with their analogous carburetor
function (idle, choke, power, acceleration enrichment,
etc.). This enabled the Electrical Engineers and software
algorithm architects to successfully capture and apply
the profound knowledge of the carburetor tuners. This is
a complex art that often resembles unmodeled science.
Returning to the variable cam phasing example, often a
new technology elsewhere in the system will introduce
unforeseen requirements so fundamental to the new
technology as to apply to all users and applications of
the technology.
For example, virtually every
implementation of the many different hybrid electric
vehicle
architectures
introduce
the
additional
requirement of very high cranking rpm, up to 650 rpm
instead of the usual 120 rpm. This 5x increase in
cranking rpm has a significant impact upon several
requirements and test schedules. By mapping the
consequences through the SFMEA, and then the
DFMEA, Delphi engineers were able to comprehend this
in the requirements and the validation to those
requirements.
Similarly, virtually every implementation of the many
different hybrid electric vehicle architectures also
introduces the additional requirement of dramatically
increasing the number of engine starts per life. At the
beginning of the development program it may not be
practical to obtain an engineering vehicle to measure the
environment and establish a good value for the number
of starts per vehicle life. It may, however, be possible to
rent a hybrid vehicle from another manufacturer which is
already commercially available to make measurements
on and get a close value for the number of starts per life.
If even this is not feasible, it is still possible to develop a
reasonable estimate of the number of starts per life by
analysis. Assigning one stop/start cycle for every cycle
of the EPA (Environmental Protection Agency) FTP-23
emissions test schedule, results in 23 starts over the
11.04 mile schedule, as opposed to 2 starts over that
same 11.04 mile schedule for a conventional electrical
architecture. A second, less aggressive estimate could
be obtained by also factoring in the 10.26 miles of the
EPA highway cycle, then extrapolate out to the vehicle
target life miles.
Conventional Hybrid
Starter motor Start-stop
Worst Case
27,272
starts/life
(150k miles/FTP miles)
(150k
5,500
miles/FTP & HWFE
miles)
starts/life
Best Case
313,628
starts/life
169,392
starts/life
Clear requirements help competitors cooperate
On some customer applications, Delphi components
must interface with the sensors and actuators of a
different Tier 1 supplier. Delphi works though the OEM
customer, but also in close cooperation with that other
supplier, who is often a competitor. It is common
practice for Tier 1 competitors to cooperate closely when
working together on a customer’s program.
This
potentially complex relationship is actually aided and
improved through the use of clearly documented
requirements with concise allocations of component and
interface requirements.
Often the customer, based upon how supplierdependent they are, relies upon the various Tier 1
suppliers to be the domain experts for their respective
areas. This requires close cooperation among parties
who would otherwise be competitors and is vital to the
overall success of the customers system, and ultimately
their individual success as a Tier 1 supplier. With clearly
defined requirements and roles and responsibilities, all
competition ends once the respective pieces of business
are awarded. The individual engineers are then free to
work together with the customer to achieve the best
system possible.
Summary / Lessons Learned from Powertrain
Example
1. Systems Engineering is a prevention-based
discipline: if the job is done right, nothing
happens. Everything goes as it should. Nothing
dramatic happens. While this is the desired
outcome, it does not create that dramatic event
which can be pointed to in trying to convince an
organization to incorporate the SE common
process.
2. Having
complete,
correct
and
usable
requirements improves communication across
organizations and within organizations. SE is
equally applicable in both the traditional
supplier/customer relationship and within a more
vertically integrated organization where both the
supplier and the customer reside in-house. This
is especially true when the corporation is a
compilation
of
previously
independent
organizations that were merged, yet still clinging
to their individual site cultures.
3. Having a complete set of clear, concise
requirements creates a safe environment for
different Tier 1 suppliers who might otherwise be
competitors to collaborate and cooperate with
each other and the OEM customer.
It is
mutually beneficial to all, especially the end
customer.
5. When integrating a component into a mature
system (where the previous component being
replaced was developed at the same time the
customer was developing their original system
and/or engine) there will be hidden and
unknown requirements that were never
documented. When problems and failures arise
during the original development process, a
product attribute or process was adjusted to
address the failures. Oftentimes the underlying
requirement is never identified or documented.
6. Where possible, embed lessons learned into
institutional documents, such as the RAR (or
CPTR), SFMEA and DFMEA for the next team
to find.
7. Parts, prints and specifications are a great place
to start, but they are not enough.
CONCLUSION
This paper reviewed the purpose of requirements, the
characteristics of good requirements and provided
examples what went wrong in two automotive projects.
The lessons learned prove that using tools such as the
RAR and CPTR in a SE based process for generating
complete, consistent, unambiguous and verifiable
requirements, is essential to today’s automotive
development process which focuses on fast to market
and doing it right the first time.
Sometimes this process might seem resource heavy and
a relentless pursuit, but it is absolutely essential for
project success.
ACKNOWLEDGMENTS
We would like to thank the management of both Delphi
Thermal Systems and Powertrain divisions for the
continued guidance and strong support for systems
engineering, the authors and the writing of this paper.
REFERENCES
4. Having a complete set of clear, concise
requirements prevents and reduces NTF (no
trouble found) warranty, which can account for
up to 70 percent of the warranty for a new
product launch. NTF warranty, when returned
parts are to spec but still problematic, are signs
that there are one or more requirements that
were not captured, and not being met.
Reducing NTF warranty provides cost
avoidance, as well as protecting the reputation
of a new vehicle being introduced to the
marketplace.
1. International Council on Systems Engineering
Systems Engineering Handbook, A Guide for
System Life Cycle Processes and Activities,
INCOSE - TP- 2003-002-03 version 3, June
2006
2. An internal Delphi document – “Requirements
Fundamentals” by L. Kelly, R. Madden and C.
Wentz
3.
“Why have a Systems Engineering (SE)
Capability
for
Automotive
Product
Development? – Questions and Answers, 2007
SAE World Congress, 2007-01-0782 by Thomas
E. Austin, Delphi – Thermal Systems Division
DEFINITIONS, ACRONYMS, ABBREVIATIONS
INCOSE –
Engineering
International
Council
on
Systems
RAR - Requirements Analysis Report
CONTACT
SE – Systems Engineering
For additional information,
SECP – Systems Engineering Common Process
Thomas E. Austin
[email protected]
DICP – Dual Independent Cam Phasing
Mr. Austin retired from Delphi and is available for
consulting and Systems Engineering training.
DOHC – Double Over Head Camshaft
Lori Runk
[email protected]
Delphi Thermal Systems
200 Upper Mountain Road
Lockport, NY 14094
OCV – Oil Control Valve, aka Cam Phaser Control Valve
VCP – Variable Cam Phasing
EPA – Environmental Protection Agency
CARB – California Air Resources Board
James Waters
[email protected]
Delphi Powertrain
3000 University Drive
Auburn Hills, MI 48326
ECE – Economic Commission for Europe
EU – European Union
SFMEA – Systems Failure Mode & Effects Analysis
DFMEA – Design Failure Mode & Effects Analysis
PFMEA – Process Failure Mode & Effects Analysis
OEM – Original Equipment Manufacturer (the Auto
Companies)