Evolution of Software Development Standards in the Military Domain

Evolution of Software
Development Standards in the
Military Domain and Effects on
Software Applications
K. Burak Codur
Ali H. Doğru
Turkish Military Academy
Defense Sciences Institute
Middle East Technical University
Computer Engineering Department
Outline
• 
• 
• 
• 
Military standards for software development
Research method
Findings
Impacts on the evolution of software
The Standards
Introduction of
MIL-STD-1679
Introduction of
MIL-STD-498
1978
1994
1968
NATO
Conference –
First use of the
term “Software
Engineering”
1985
Introduction of
DOD-STD-2167
1998
Introduction of
IEEE 12207 (to the
military service)
Why These Standards Make Up
A Unique Case Study?
•  Military domain is unique:
–  Whole spectrum of applications.
–  Single type of user and acquirer.
•  The standards utilized are well known.
•  One can be sure these standards were applied.
Motivation
To understand the
evolution of software in
the military domain
How?
We have texts of the standards.
+
Out of the 4 standards, 3 of them are not
in use, 2 of them have not been in use for
almost 15 years.
-
We do not have comparable data.
-
We do not have comparable applications.
-
How? (Cont.)
Content Analysis
Software Development
Experiment
Content Analysis
•  An Ontology for Software Development Standards
•  Analysis and Comparison of the Contents
•  Definitions
•  Requirements
•  BPMN Diagrams of Standards
Software Development Experiment
•  Goal: To Implement Same Projects in
Accordance to Each Standard and Collect
Comparable Data
•  Implementation: A Graduate Course at Software
Engineering Program of Computer Engineering
Department, METU
•  Projects: Planning, Requirement Analysis and
Preliminary Design of Software
General Design of the Experiment
OCD,
SSS,
SSDD
+
Standards
and
Templates
1. Phase – Planning
2. Phase – Requirements Analysis
3. Phase – Preliminary Design
Estimates for
schedule and effort
Requirements
Estimate for
Lines of Code
6 Projects, 6 Groups, 3 Iterations
Sample Data – Schedule Estimate
 Projects
Groups 
Major Findings - 1
1.  The processes defined by the standards became
more detailed and complex in time
(Content analysis).
Major Findings - 2
2.  There are specific subjects focused by the
standards, such as:
•  coding (MIL-STD-1679) or
•  safety,
•  security and
•  privacy requirements (MIL-STD-498).
These focused subjects lose attention in time
(Content analysis).
Major Findings - 3
3.  Scopes of the standards have spread in
time:
•  More concepts have entered in the scope of the
standards,
•  while a few diminished in time
(Content analysis).
Major Findings – 3 (Cont.)
Concept
Undelivered software
Incremental and
evolutionary software
development
Tailoring
Reuse
Independence in tests
Traceability
Versioning
Software transition
Testability
Validation
Service
Stress tests
User training
Document templates
MILSTD-1679
DODSTD-2167
MILSTD-498
IEEE/EIA 12207
X
X
√
√
√
√
√
√
X
X
X
X
X
≈
X
X
X
√
X
√
√
√
√
√
√
≈
≈
X
X
X
≈
√
√
√
√
√
√
√
√
X
√
X
√
√
√
√
√
√
√
√
√
√
√
√
√
X
√ means “contains”, X means “does not contain”, ≈ means “partially contains”
Major Findings - 4
4.  Evolution of the standards did not progress
in parallel with developing industry in all
aspects:
•  There are some cases, albeit rare, where the
concepts that diminished from scope of the
military standards were later employed by
modern standards (Content analysis).
Major Findings - 5
5.  Schedule and effort estimates increased as
standards evolved. (software development
experiment).
Major Findings - 6
6.  Number of requirements defined increased as
standards evolved (content analysis and
software development experiment).
Major Findings - 7
7.  There is not a relation between evolution of the
standards and lines of source code estimation
(software development experiment).
Major Findings - 8
8.  Evolution of the standards resulted with
increase in the variety of software
development models applied (content
analysis and software development
experiment).
Major Findings - 9
9.  Using standards tailored to application type
makes sense (software development
experiment).
Conclusions and Future Work
•  Evolutionary progress until our decade.
•  If life cycle of software spreads over jurisdiction of
multiple standards, a transition work is needed.
–  Schedule and effort estimates and the number of software
requirements would increase
•  If evolutionary progress continues - with the help of
research like this - predictable transitions can be
managed in the future.
•  Future Work: Assessment of change in software quality
Questions
•  About the presentation
–  In the limited context of our experiment, how one can
study the individual effects of different parameters?
•  Debate questions
1. Do we need military defined software development
standards in the military domain?
2. How can we collect and keep data (or clues) related to
effects of regulations (so that later we can utilize
these to assess evolution of software)?