Slides - U

Why and how are agile
teams using metrics?
Eetu Kupiainen, Mika Mäntylä & Juha
Itkonen
Research aim and -method
● Why are industrial agile teams using metrics?
● What metrics do they use?
● What actions does the use of metrics trigger?
Systematic literature review
● Automatic search from a research search engine
(Scopus)
● From 774 papers to 29 primary studies
Context information
Categories for metric usage
● Iteration planning
● Iteration tracking
● Motivating and
improving
● Identifying process
problems
● Pre-release quality
● Post-release quality
● Changes in
processes or tools
Iteration planning
● Why
○ Prioritization tasks
○ Scoping the iteration
○ Effort prediction
● Metrics
○ E.g Effort estimate, actual velocity of previous
iteration, team’s available hours
Iteration planning, case example
● VeriSign Managed Security Services
○ Complex multi-product development organization
○ Need for prioritization of backlog
○ One attribute for prioritization is customer
commitments in dollars
P. Hodgkins and L. Hohmann. Agile program management: Lessons learned from the verisign managed security
services team. In Proceedings - AGILE 2007, pages 194{199, Washington, DC, 2007.
Iteration tracking
Why
● Monitoring (Are we going to make it?)
○ Reduce scope, add more resources
● Visibility for all stakeholders
● Balance workflow
Metrics
● Velocity / Burndown, completed work items
Iteration tracking, case example
● Mamdas, Israel Air Force
● Large-scale information system
Y. Dubinsky, D. Talby, O.
Hazzan, and A. Keren.
Agile metrics at the israeli air
force. In Proceedings - AGILE
Confernce 2005, volume 2005,
pages 12-19, Denver, CO,
2005.
Motivating and improving
Why
● Motivate team to act and improve
○ E.g Fix build faster, fix bugs faster, increase unit
testing
Metrics
● Build status, defect amounts, # of unit tests
Motivating and improving, example
● Systematic, complex and critical IT solutions,
ScrumFDD
● Unanticipated integration problems -> fixlater mentality -> measure fix time for
failed builds -> show fix times next
to the coffee machine -> create
discussion and improvement
C. Jakobsen and T. Poppendieck. Lean as a scrum troubleshooter. In Proceedings - 2011 Agile Conference, Agile 2011, pages 168-174, Salt
Lake City, UT, 2011.
Identifying process problems
Why?
● Identify and predict problems
● Identify phases where no value is added
Metrics
● Lead time, common compo time, defect
trend indicator
Identifying process problems: case
● Timberline, now part of Sage
● Software for construction industry, Lean
P. Middleton, P. Taylor, A.
Flaxel, and A. Cookson. Lean
principles and techniques for
improving the quality and
productivity of software
development projects: A case
study. International Journal of
Productivity and Quality
Management, 2(4):387-403,
2007.
Pre-release quality
Why
● Prevent defects reaching customer,
understand quality of the product, allow
further development
Metrics
● Technical debt board and -costs, static code
check metrics, faults per iteration
Pre-release quality, case example
● Petrobras, software for oil and gas industry
● Scrum
P. S. M. dos Santos, A. Varella,
C. R. Dantas, and D. B. ao
Borges. Visualizing and
managing technical debt in agile
development: An experience
report. In H. Baumeister and B.
Weber, editors, Agile Processes
in Software Engineering and
Extreme Programming, volume
149 of Lecture Notes in
Business Information
Processing, pages 121-134.
Springer, 2013.
Post-release quality
Why
● Evaluate quality of product after release
● Predict post release quality
Metrics
● defects sent by customer, Net-PromoterScore, change requests from customers,
defect counts, deferred defects
Changes in processes or tools
Why
● Improve development process, improve tool
configuration
Metrics
● Lead time, time to fix the build, throughput
How
● Parallel test phases, visualize build status
Changes in processes or tools, case
● Ericsson AB, telecom and multimedia,Scrum
● Competitive advantage through customer
responsiveness -> increase throughput ->
measure rate of requirements
over phases -> identify bottlenecks ->improve process ->
integrate early and often
K. Petersen and C. Wohlin. Measuring the flow in lean software development. Software - Practice and Experience, 41(9):975-996, 2011.
Other findings
● Effort estimation metrics used a lot
● Lack of code metrics
Recap - why and how
● Iteration planning
● Iteration tracking
● Motivating and
improving
● Identifying process
problems
● Pre-release quality
● Post-release quality
● Changes in
processes or tools
Thank you!
Questions, comments?