PPT - Applying Lean Six Sigma to Software Engineering

Applying Lean Six Sigma to
Software Engineering
Joe Schofield,
Sandia National Laboratories
Albuquerque, N. M.
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,
for the United States Department of Energy under contract DE-AC04-94AL85000.
Schofield:Sandia National Labs:IFPUG:9/2004
1
Applying Lean Six Sigma to
Software Engineering
Joe Schofield, yhs
Note: The CMMI and SW-CMM are registered copyrights of the Software Engineering Institute
Graphics assistance by Leila Starr
The opinions expressed in the presentation are solely / sorely, the presenter‘s.
Schofield:Sandia National Labs:IFPUG:9/2004
2
(Geographical) Context Diagram
You are here!
New Mexico
Schofield:Sandia National Labs:IFPUG:9/2004
Mexico
3
1
4
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
Frameworks
Informationweek, Behind the Numbers, March 29, 2004; pg 94
CIO, By the Numbers, December 1, 2003, pg 28
3
Baseline – The Project Management Center, We Did Nothing Wrong, March 4, 2004
4
Informationweek, Software Quality, March 15, 2004; pg 56
5
www.nctimes.com/news/050600/d.html
6
Albuquerque Journal; Computer A Real Labor For State; 6/04
2
More To Do
Path To
After more than two years of delay, the state Department of Labor’s $13M
million computer system to process unemployment insurance claims and
checks still isn’t fully off the ground6
Significance
Meta Data
In 2000, the nctimes placed the cost of one virus at $10B5
Reporting
CMMI ®
BMW, DaimlerChrysler, Mitsubishi, and Volvo experience product
malfunctions (engine stalls, gauges not illuminated, wiping intervals,
wrong transmission gears) due to software4
Examples
Vocab
Software technicians in Panama are charged with second degree murder
after 27 patients received overdoses of gamma rays; 21 have died in 40
months3
DMAIC Use
Savings
38 percent of polled organizations have no SQA program2
vs. JADs
Elements
Software defects cost the U.S. $59.6B a year1
Applying 6S
BIZ Case
Software Problems Have Been Eradicated (huh?)
Green Belts – varying periods of training and certification activities
Black Belts – varying periods of training and certification activities, usually
DSS – Design for Six (or Single) Sigma – do we create capacity only to turn
Flow – product not produced until recipient requests it (less inventory, better
Lead time – total time to complete a set of tasks (includes delays, queues, etc.)
5
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
Frameworks
Lean – as much as necessary, and no more
More To Do
Path To
quality if defect found in process that has inventory, less space, engages
workers)
Significance
Meta Data
around and squander it?
Reporting
CMMI ®
Measurement – reduced variation
Cycle time – time to complete a cycle of operation
Examples
Vocab
some mentoring; Change agents, improvement agents
DMAIC Use
Savings
Customer – focus is on “voice of the customer”
vs. JADs
Elements
Typically oriented toward manufacturing floor but has also been used in
business “transactional” setting
Applying 6S
BIZ Case
Elements of Lean Six Sigma
Elements
Textron saved $5M in six months.2
“We are making excellent progress, $15M to date” at Mount Carmel
Health
Magazine, Targeting Perfection, 12/1/2003, pg. 62
3Basic
Statistics, Kiemele, et al; Air Academy Press; 2000
6
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
2CIO
Frameworks
Sigma Academy, Scottsdale, AZ.
More To Do
Path To
1Six
Significance
Meta Data
General Electric reported a $1B savings in two years reducing defect /
re-work costs by ½ .
Reporting
CMMI ®
Motorola reduced manufacturing costs over 7 years by $1.4B3
Examples
Vocab
Allied Signal reduced costs over 5 years by $1.4B3
DMAIC Use
Savings
Ratheon Aircraft saved $.5M from one IT project alone.2
vs. JADs
Saves an average of $250K per project 1
Applying 6S
BIZ Case
A Sampling of Reported Savings
Elements
7
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
Definitions interpreted from Lean Thinking; Womack & Jones; ISBM 0-684-81035-2
More To Do
(quantities) or how fast you have to go?
Significance
Frameworks
takt time – daily available production time / daily customer demand
Reporting
Path To
poka-yoke – mistake proofing; preventing defects from moving forward
Examples
muda – waste
DMAIC Use
Meta Data
of product
CMMI ®
kanban – a indicator (card or light) that signals the movement or productions
Vocab
A kaizen event includes event planning, sponsor kick-off, objectives and goals,
some LSS training (can be accomplished prior to event), mapping current state,
waste identification, root cause analysis, B/Sing on solutions, solution
implementation, mapping the new (future) state, and ongoing reporting with the
sponsor.
Savings
kaizen – kai – to take apart; zen – to make new (better)
vs. JADs
kaikaku – (roughly) radical improvement
Applying 6S
BIZ Case
Lean Six Sigma Jargon Vocab (cont’d)
Elements
Vocab
Closing
8
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
More To Do
Frameworks
CMMISM Version 1.1; page 14
Significance
Path To
1 The
Reporting
Meta Data
Northrop Grumman reports eight operating units at CMMISM
Level 5 using Six Sigma; corporately certifying 3000 Green
Belts and 200 Black Belts
Examples
CMMI ®
CMMISM Level 5 calls for the analysis of common cause
variation and to improve the process while sustaining the
process with statistical predictability1
DMAIC Use
Savings
CMMISM Level 4 calls for the analysis of special cause
variation in process1
vs. JADs
Mentioned in at least six sessions at the 2004 SEPG
Conference
Applying 6S
BIZ Case
Lean Six Sigma and CMMI®
Reporting
Significance
More To Do
Closing
Leaning
9
Examples
Frameworks
Statistical Six Sigma Definition; http://healthcare.isixsigma.com/library/content/c010101a.asp
Schofield:Sandia National Labs:IFPUG:9/2004
Path To
Motorola’s Six Sigma program was a statistical target: 3.4 defects per
million opportunity (a chance for non-conformance)
Meta Data
Using Six Sigma as a process improvement approach – 7 ½ sites out of
10; statistical approach 2 ½ out of 10; 1 more undecided
DMAIC Use
CMMI ®
Six Sigma for process improvement, not six sigma as a statistical
target.
vs. JADs
Vocab
Not a cult group, not a slogan.
1
Savings
A topic mentioned on more than 1,770,000 web sites (Lean SS: 126,000)
(5/04)
Elements
“at many organizations simply means a measure of quality that strives
for near perfection”1
Applying 6S
BIZ Case
When Lean Six Sigma Isn’t (Six Sigma)
Elements
Vocab
Reporting
Significance
Meta Data
Closing
Frameworks
More To Do
Path To
Leaning
10
Examples
CMMI ®
Schofield:Sandia National Labs:IFPUG:9/2004
DMAIC Use
assumes long term,
is 1.5 sigma,
narrows the distribution,
6 sigma decreases from
2DPB (Defect Per Billion)
to 3.4 DPM
• Is really 4.5 sigma results!
•
•
•
•
vs. JADs
Savings
Sigma shift:
Applying 6S
BIZ Case
When Lean Six Sigma Isn’t (cont’d)
Elements
“What if” the sigma shift went to the right:
Sigma shift right
(3 PhT)
Close to but <
5 sigma
> 6 sigma
7.5 sigma
1
Car
1 (40,000 deaths
per year)
Close to but <
5 sigma
> 6 sigma
7.5 sigma (less than
1 death per year)
1
Airline and auto data cited from: www.panopoulos.com/ifaqs.htm
11
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
Frameworks
53,333 times better!
More To Do
.86 (1 for every
1.16M flights)
Significance
Airline
Path To
1
Reporting
Sigma shift
left (3.4 PPM)
Examples
Normal @ 6
sigma (2 PPB)
Meta Data
Fatal Events per
Million (actual)
CMMI ®
Mode
DMAIC Use
Vocab
Is this 7.5 sigma level of performance acceptable?
vs. JADs
Savings
• 6 sigma increases from
2DPB to 3.19 DP Hundred trillion T
• Is really 7.5 sigma results!
Applying 6S
BIZ Case
When Lean Six Sigma Isn’t (cont’d)
Elements
“What if” the sigma shift went to the right – a teraflop example:
Vocab
PetaFlops machine
12
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
PETAFLOP Imperative; Informationweek; June 21, 2004; pgs. 55 - 62
More To Do
Frameworks
1
Who can repair / afford /
manage that many defects?
Path To
943,488,000 defects per year @ 7.5 sigma =
50B (that’s BILLION) at “shifted 6 sigma”
Significance
predicted to be ready by 2005 or 2006
1,000 times faster than a 1TFlop machine =
Reporting
Meta Data
1
CMMI ®
50M+ a year at “shifted 6 sigma” (4.5 sigma)
(these numbers are rounded down)
Examples
18,144 per week =
943,488 DEFECTS per year =
DMAIC Use
3 defects per 100 seconds =
108 defects per hour =
vs. JADs
Savings
TeraFlops machine
1T floating point operations instructions per second =
Applying 6S
BIZ Case
When Lean Six Sigma Isn’t (cont’d)
Applying 6S
Elements
vs. JADs
Vocab
DMAIC Use
Savings
Examples
Closing
Frameworks
More To Do
Path To
Significance
Meta Data
Reporting
CMMI ®
Leaning
13
© 2001 Lockheed Martin Corporation
Schofield:Sandia National Labs:IFPUG:9/2004
BIZ Case
LM21 Path to Excellence
Elements
Significance
Closing
Frameworks
More To Do
Path To
Leaning
14
Reporting
Meta Data
Schofield:Sandia National Labs:IFPUG:9/2004
Examples
CMMI ®
IDEALSM – Initiating, Diagnosing, Establishing, Acting,
Learning (vs. improvise, dig-in, exclude, acquiesce, . . . )
DMAIC Use
Vocab
PDCA – Plan, do, check, act (vs. Postpone, defer, challenge,
avoid)
vs. JADs
Savings
DMAIC – Define, measure, analyze, improve, control (vs.
delay, minimal attention, ignore, quit) Define better include
defining defects and measures across the organization.
Applying 6S
BIZ Case
Lean Six Sigma and other Process
Improvement Frameworks
Everything has changed, but
nothing is different
Everything is changed, and
everything worthwhile has
improved
15
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
Resources as the first measure of
commitment
More To Do
Management soft interest
Significance
leaning events from a documented
process
Reporting
Emphasis on event types
Examples
a documented & repeatable
process for event types
DMAIC Use
How many BBs co-facilitate via
tribal knowledge
vs. JADs
# of sustained improvements
Frameworks
# of events
Path To
quality of data used for decisionmaking
Meta Data
How many AIs / events result
CMMI ®
impact of results
Vocab
How long a session is
Savings
Need more of . . .
Elements
Need less of . . .
Applying 6S
BIZ Case
Leaning Lean Six Sigma
Closing
Frameworks
More To Do
Path To
Leaning
16
Significance
Meta Data
Schofield:Sandia National Labs:IFPUG:9/2004
Reporting
CMMI ®
After
Examples
Vocab
Before
DMAIC Use
Savings
Means sort, straighten, shine, standardize, safety, sustain
vs. JADs
Elements
Means spic, span, spiff, slick, . . .
Applying 6S
BIZ Case
Notes on 6S
17
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
Frameworks
term
More To Do
Path To
After – long
Significance
Meta Data
term
Reporting
CMMI ®
After – long
Examples
Vocab
term
DMAIC Use
Savings
After – short
vs. JADs
Elements
When items are more visible, others will see them too!
Applying 6S
BIZ Case
6S – A Lesson Learned
Cycle time (both should reduce activity duration)
Number of hands touching (both should reduce number of
and non-value added contact)
18
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
Frameworks
Number of hand-offs (both should reduce queues, queue-time
More To Do
Path To
players)
Significance
Meta Data
Defects (both should improve activity results)
Reporting
CMMI ®
days or longer)
Examples
Vocab
Must last longer than a Brittney Spears marriage (both take
DMAIC Use
Savings
are revealing about current state, and potentially transforming for the
future state)
vs. JADs
Elements
Eye-opening, head-turning, jaw-dropping experience (both
Applying 6S
BIZ Case
LSS Events versus JAD sessions –
It’s the Same . . .
Work in process / inventory
PICK Chart
Leaning
19
Closing
Frameworks
Schofield:Sandia National Labs:IFPUG:9/2004
More To Do
Path To
PICK Charts
Significance
Meta Data
Pull (how work is triggered)
Reporting
CMMI ®
Failure Mode Effects Analysis
Examples
Vocab
Changeover time (minimized)
DMAIC Use
Savings
Takt time (how work is measured)
vs. JADs
Elements
Value Streaming
Applying 6S
BIZ Case
LSS Events versus JAD sessions –
It’s Different . . .
Analyze (opportunity)
Defect List & Analysis
Improve (performance)
Process Focus & Change
Control (performance)
Sustained Measurement & Improvement
Leaning
20
Closing
Frameworks
Hows
More To Do
Path To
Schofield:Sandia National Labs:IFPUG:9/2004
Meta Data
Whats
Metrics
DataBase
Significance
Peer Reviews & Defects
Reporting
Measure (performance)
CMMI ®
TWGs Projects
Examples
Vocab
Define (opportunities)
DMAIC Use
Savings
Measures:
vs. JADs
Elements
Actual cost benefit figures for software development
Applying 6S
BIZ Case
Applying DMAIC to Defect Data
Ops.
Impl.
Design
Planning
20
40
60
80
100
120
Distribution of Defect Cost to Repair
Failure
Cost not recorded
Less than or equal to $100
Less than or equal to $5
0
50
100
150
0
20
40
60
80
100
120
140
160
21
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
200
Closing
Frameworks
Less than or equal to $20
Aesthetic
More To Do
Path To
More than $100
Functional
Significance
Meta Data
Defect Types
Reporting
Analyze
CMMI ®
0
Examples
Vocab
Analysis
Measure
DMAIC Use
Savings
Deploy.
vs. JADs
Elements
Injected Defects for 12 Projects
Applying 6S
BIZ Case
Applying DMAIC to Defect Data (cont’d)
Exemptions – this doesn’t apply to me
Exclusions – Not all my work falls into this category
Closing
Frameworks
Leaning
22
More To Do
Path To
Schofield:Sandia National Labs:IFPUG:9/2004
Significance
Meta Data
Add your own “EXs” to this list!
Reporting
CMMI ®
Explanations – Let me tell you why this won’t work
Examples
Vocab
Excuses – I / we can’t do this
DMAIC Use
Savings
Exceptions – this is different
vs. JADs
Elements
(or why LSS won’t work for us!)
Applying 6S
BIZ Case
“Ex” (literally “out of” or “not” Webster) Rules
Applying 6S
vs. JADs
Elements
DMAIC Use
Savings
Closing
Frameworks
More To Do
Path To
Significance
Meta Data
Reporting
CMMI ®
Examples
Vocab
Leaning
23
Schofield:Sandia National Labs:IFPUG:9/2004
BIZ Case
An Example of the Steps in Action
2. Project Definition
Leaning
24
Closing
Frameworks
Schofield:Sandia National Labs:IFPUG:9/2004
Team was a mixture of practitioners
and “outsiders.”
More To Do
(1)
Path To
Commissions SILC VSM / Team 12/03
The event dates were scheduled in
December.
Significance
Center 9500 SEPG
Meta Data
The project charter is a template used
in the LSS course.
Reporting
CMMI ®
Plan (2)
Examples
Vocab
SEPG is the Software
Engineering Process Group.
DMAIC Use
Savings
VSM is a Value Steam Mapping.
vs. JADs
Elements
1. Identify Priorities &
Opportunities
Applying 6S
BIZ Case
Dissecting the Example
Significance
More To Do
Closing
Frameworks
Leaning
25
Reporting
Path To
Schofield:Sandia National Labs:IFPUG:9/2004
Examples
Meta Data
A lesson learned is that we would
capture quantitative data on a
spreadsheet for downstream analysis.
CMMI ®
In this “event” the interviews were
conducted with template-driven
interview sheets over about 30 hours
and prior to the event.
DMAIC Use
Vocab
Typically done with all participants.
Interview
(3)
Savings
3. Document & Measure Current
Reality
vs. JADs
Elements
24 Artifacts, 6 project interviews each,
14 pieces of data = 2024 pieces of data
6 supporting artifacts, 6 projects, across
5 activities, 14 pieces of data = 1520
pieces of data
Applying 6S
BIZ Case
Dissecting the Example (continued)
vs. JADs
DMAIC Use
Examples
Reporting
Significance
More To Do
Closing
Frameworks
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
A lesson learned is that we would
have created a summary template
to more quickly summarize this
26
information.
Path To
The artifact map depicts the
relationships (and traceability)
among our artifacts.
Applying 6S
These sheets summarize the 3500
facts about our process cycle and
lead times, defects, waste, queue
times, . . .
Meta Data
Summarize (4)
CMMI ®
CTCs
These sheets took six people about
four hours to create.
Vocab
Each set of sheets were analyzed
by sequence and also by project
size.
Savings
CTC (Critical to Customer) – places
the customer needs at the focal
point of the “event.”
Elements
4. Analyze and Identify Waste
BIZ Case
Dissecting the Example (continued)
Cycle time (hands-on) time
Leaning
27
Closing
Frameworks
Schofield:Sandia National Labs:IFPUG:9/2004
More To Do
Path To
This event examined projects in
three size classes: small,
medium, large.
Significance
Meta Data
Rework time (definitely
muda)
Reporting
CMMI ®
Lead time (total time to
complete a product)
Examples
Vocab
Queue time (a form of muda)
DMAIC Use
The heart of this analysis is the
derivation of:
vs. JADs
Savings
4. Analyze and Identify Waste
Applying 6S
Elements
Analyze
(4)
BIZ Case
Dissecting the Example (continued)
2 – Duplication between proposal and project plan /
reduce
Reporting
Significance
More To Do
Closing
Leaning
28
Frameworks
Schofield:Sandia National Labs:IFPUG:9/2004
Path To
The calculated rankings were
plotted (bottom figure) as a visual
representation of the rankings.
Examples
The average of the rankings were
calculated (middle figure)
Meta Data
PICK Chart
CMMI ®
6 – Incompleteness in models – Wide variation of
process modeling / step-by-step process description
Each improvement was ranked by
each participant to two
dimensions: easy vs. hard, and
big vs. small payoff (top and
middle figures)
Vocab
5 – Unneeded logical data model (and 9 – People
doing information modeling at the physical level
(i.e., incorrect data types in physical ERD);
additional methods for information model / possible
kaizen
DMAIC Use
4 – Peer reviews without defect identification –
missing defects / review practices of some
Savings
3 – Approvers exceed developers on artifacts /
reduce
5. Based on the data, process
observations
(improvements) were
developed.
vs. JADs
1 – Overkill boilerplate / reduce redundancy
Elements
Legend
Applying 6S
BIZ Case
Dissecting the Example (continued)
Reporting
Significance
More To Do
Closing
Frameworks
Leaning
29
Examples
Path To
Unlike a “just do it”, these
processes are documented,
trained on, and institutionalized
across projects.
Meta Data
Schofield:Sandia National Labs:IFPUG:9/2004
The SEPG agreed to include these
suggestions in the next release of
the software process.
DMAIC Use
CMMI ®
The team also started a Failure
Mode Effects Analysis (FMEA) on
the highest ranked items.
Risks & Lessons
Learned
Vocab
The team conducted a lessons
learned activity as part of it’s
wrap-up.
Savings
The expected outcome of this
event was a series of
recommendations to the SEPG for
process improvement.
vs. JADs
Elements
Report
back (8)
8. Communicate and
acknowledge success
Applying 6S
BIZ Case
Dissecting the Example (continued)
Applying 6S
vs. JADs
Elements
DMAIC Use
Savings
Examples
Vocab
Closing
Frameworks
More To Do
Path To
Significance
Meta Data
Reporting
CMMI ®
Leaning
30
Schofield:Sandia National Labs:IFPUG:9/2004
BIZ Case
Subsequent “events” used a similar process
31
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
Frameworks
The team established LSS at a process improvement within the
hosting organization.
More To Do
Path To
The team demonstrated that use of the LSS processes could
lead to real insights and needed improvements.
Significance
Meta Data
The results of this work identified a number of areas where
practice and policy were not the same.
Reporting
CMMI ®
The results of this work drove the organization to capturing
better measures around the creation of software artifacts.
Examples
Vocab
While LSS has reportedly been performed for pieces of the
software lifecycle (like coding), similar work across the lifecycle
is rare.
DMAIC Use
Savings
Conducting events for underlying business process change is
easy to suggest for our customers; it should be easy for us as
software engineers.
vs. JADs
Elements
Another (one of many) self-assessments of software process.
Applying 6S
BIZ Case
Significance of this Event
Leaning
32
Closing
Frameworks
Schofield:Sandia National Labs:IFPUG:9/2004
More To Do
Path To
EZ stuff is already done, less well projects await us.
Significance
Meta Data
Standards (ISO 9001:2001) are vague; frameworks for
assessment may not tailor to real organizations.
Reporting
CMMI ®
Misdirected attention at life cycle approaches (agile,
Extreme Programming, cyclic development) instead of
required software engineering skill sets.
Examples
Vocab
Misplaced and erroneous focus on KLOCs (FPs is an ISO
standard 20926:2003).
DMAIC Use
Savings
Unrealistic expectations from estimation tools (what you
need to know upfront).
vs. JADs
Elements
Too few objective reports on project completions, too
much publicity around failures.
Applying 6S
BIZ Case
General Software Engineering Problems –
addressed or not by LSS?
33
Leaning
Schofield:Sandia National Labs:IFPUG:9/2004
Closing
Frameworks
Processes for leaning are themselves “candidates for
leaning.” Remember the turtle!
More To Do
Path To
Which of your own personal processes have you leaned
lately?
Significance
Meta Data
Understand the denotation and connotation of the phrase
“six sigma.” Watch for the effects of “sigma shifting.”
Reporting
CMMI ®
Software engineering processes are candidates for increased
measurement and quantitative self-assessment.
Examples
Vocab
Software engineers are not as unique as they’ve historically
asserted.
DMAIC Use
Savings
Software engineering processes are candidates for leaning.
vs. JADs
Elements
Defects persist in software; most of these come from
executing the software development process.
Applying 6S
BIZ Case
Closing