slides - Neal Ford

Continuous Delivery Workshop
deployment
pipelines
Workshop materials created by Jez Humble, Martin Fowler, Tom Sulston, & Neal Ford
deployment pipelines
tests, synergistic practices,
incremental deployment
data & infrastructure
Who are You?
Manager
DBA
BA
Developer
UX
Operations
QA
Agile Transformation
Manager
DBA
BA
Developer
UX
Operations
QA
Continuous Delivery
Manager
DBA
BA
Developer
UX
Operations
QA
It’s a people problem.
The Dangers of Silos
The Tragedy of the Commons
Conway’s Law
“organizations which design systems ... are
constrained to produce designs which are
copies of the communication structures of
these organizations”
—Melvin Conway
Release Cadence
Mary Poppendieck
Tom Poppendieck
“How long would it take your organization to
deploy a change that involves just one single
line of code?”
devopsy.com/blog/2013/08/16/devops-kata-single-line-of-code/
“Can you do this on a repeatable, reliable basis?”
Continuous Delivery Metrics
lead time
the time between the initiation and completion
of a production process.
cycle time
the total elapsed time to move a unit of work from
the beginning to the end of a physical process
Continuous Integration
Integration early and often.
Everyone checks into trunk at least once a day.
1200
Bring the pain forward.
900
pain
eager vs. late
600
300
0
time
Continuous Integration
Integration early and often.
Everyone checks into trunk at least once a day.
Integration
Integration early and often.
Everyone checks into trunk at least once a day.
Continuous Deployment
Deploy as the final stage of continuous integration.
Integration
Integration early and often.
Everyone checks into trunk at least once a day.
Deployment
Deploy as the final stage of continuous integration.
Continuous Delivery
Software is always in a deployable state.
User Interface
(Web, Swing)
Business Analysts
QA
Domain Logic
(Model)
Application Server / Spring
Persistance
(OR Mapper)
Firewall
External
Stuff
Storage
Application
External Endpoints
Mainframe Computer
Modern software
is complex!
Agile 101
"Agile" team
Analysis + Design
Centralized QA
IT Operations
Development
Integration + QA
Release and operation
Testing + Showcase
Customer
Iteration
0
1
2
3
4
The "last mile"
Continuous Delivery
"Agile" team
Analysis + Design
Centralized QA
IT Operations
Development
Integration + QA
Release and operation
Testing + Showcase
Customer
Iteration
0
1
2
3
4
The "last mile"
Customer
Delivery team
Constant flow of new features into production
business needs > operational constraints
always
production
ready
Potential Hindrances
Lead time is too long
Last mile is too painful
Poor collaboration
Identify & remove friction
Continuous Integration
Fast, automated feedback
on the correctness of your
application every time there
is a change to code
Deployment Pipeline
Fast, automated feedback
production readiness
readiness
on the production
of your application every
time there is a change — to
code infrastructure
code,
infrastructure, or
configuration
configuration
22
Deployment Pipelines
Prerequisites
continuous integration
comprehensive
configuration management
excellent automated testing at all levels
commit Stage
Commit stage
Version
control
source code
commit tests
build scripts
Compile
Unit test
Assemble
Code analysis
deployable binaries
test reports
metadata
Run against each check-in (continual
integration)
Starts building a release candidate
If it fails, fix it immediately
Artifact
repository
Pipeline Construction
commit
functional
test
artifact
respository
user
acceptance test
artifact
respository
artifact
respository
staging
…
artifact
respository
increasing confidence in production readiness
Pipeline stages = feedback
opportunities
UAT Stage
Version
control
Artifact
repository
acceptance tests
deployment scripts
configuration data
binaries
Acceptance test stage
Configure environment
Deploy and smoke test
Acceptance test
Tear down
test reports
metadata
Artifact
repository
End-to-end tests in production-like
environment
Triggered when upstream stage passes
First DevOps-centric build
27
Manual Stage
Version
control
Artifact
repository
deployment scripts
configuration data
binaries
Later stages
Configure environment
Deploy and smoke test
Tear down on request
test reports
metadata
Artifact
repository
UAT, staging, integration, production, …
Push versus Pull model
Deployments self-serviced through pushbutton process
28
Env &
app
config
Source
code
Env &
app
config
Version control
UAT
Testers
Self-service
deployments
Developers
See code metrics
and test failures
Configure environment
Deploy binaries
Smoke test
Commit stage
Acceptance stage
Capacity stage
Compile
Commit tests
Assemble
Code analysis
Configure environment
Deploy binaries
Smoke test
Acceptance tests
Configure environment
Deploy binaries
Smoke test
Run capacity tests
Production
reports
binaries
metadata
Operations
perform
push-button
releases
binaries
reports
metadata
binaries
Configure environment
Deploy binaries
Smoke test
reports
metadata
Artifact repository
29
Machinery
continuous integration ++
www.thoughtworks.com/products/go-continuous-delivery
www.go.cd
Increasing confidence in build's production readiness
Environments become more production-like
Commit stage
Compile
Unit test
Analysis
Build installers
User
acceptance
testing
Acceptance
test stage
Production
Performance
testing
Faster feedback
“chicken
counting”
servers
0 1∞
Increasing confidence in build's production readiness
Environments become more production-like
Commit stage
Compile
Unit test
Analysis
Build installers
User
acceptance
testing
Acceptance
test stage
Production
Performance
testing
Faster feedback
pipeline anti-patterns
Increasing confidence in build's production readiness
insufficient
parallelization
Environments
become more production-like
Commit stage
Compile
Unit test
Analysis
Build installers
User
acceptance
testing
Acceptance
test stage
Production
Performance
testing
Faster feedback
ideal time: < 10 minutes
pipeline anti-patterns
Increasing confidence in build's production readiness
insufficient
parallelization
Environments
become more production-like
Commit stage
Compile
Unit test
Analysis
Build installers
User
acceptance
testing
Acceptance
test stage
Production
Performance
testing
Faster feedback
Mingle:
3,282 test / 53
computers = ~1 hour
insufficient parallelization
heuristic:
make your pipeline wide, not long
reduce the number of stages as much
as possible
parallelize each stage as much as you
can
create more stages if necessary to
optimize feedback
pipeline anti-patterns
inflexible workflow
pipeline fans out
as soon as it
makes sense to
do so
The Deming Cycle
The Scientific Method
hypothesize
experiment
evaluate
principles of software
delivery
principles
automate almost everything
build, deploy, test, release
manual testing, approvals
automatable
high-value,
humans only
principles
keep everything you need to build,
deploy, test, & release in version control
• requirements documents
• test scripts
• automated test cases
• network configuration
scripts
• technical documentation
• database creation,
upgrade, downgrade, and
initialization scripts
• application stack
configuration scripts
• libraries
• deployment scripts
• tool chains
When You Hire a New Developer…
Infrastructure Consistency
boxen.github.com
pain
If it hurts...
... do it more often
bring
the pain
forward
time
principles
principles
“Cease dependence on mass
inspection to achieve quality.
Improve the process and
build quality into the product
in the first place”
W. Edwards Deming
build quality
in
continuous
improvement
Deming cycle
principles
practices
only build your
binaries once
practices
deploy the same
way to every
environment
practices
smoke test
your
deployments
practices
keep your
environments
similar
if
anything
fails, stop
the line
practices
Metrics & Monitoring
the driver for agile in the enterprise
21st century agility (business + IT)
selling continuous
delivery
visibility
risk mitigation
responsiveness (cycle time)
nealford.com
@neal4d