1521B - Product Lifecycle Management

Transcription

I
MILSTD-6
4 JUNE
●
b (USAF
l=
SUPERSEDING
MIL-SD-l~A
)’
,
1 JUNE
lUSAm
1976
STANDARD
MILITARY
TECHNICAL
REVIEWS AND AUDITS
EQUIPMENTS,
AND
mR SYSTEMS,
COMPUTER
SOFIWARE
}
AM:C
IuO.!=?637
— iA.--~!~~,i~’H;C?i u
,ewifi~T
A.
~pp~veo
AaEA CMAN
torpUIJtIC
r’eleaae:
Clisvlcunon
ISUnllmltM.
MIL-STD-1521B
D&PAR’34ENT OF DEFZNSZ
WAS.HIWTON,
Technical
Software
itevie”~san~ Audits
D.C.
203C!
for Syster.s, Equipments,
1. This Milltary Standard is approved for use by all
and Agencies of the’ Department of Defense.
and
Computer
Departments
deletions)
2. Beneficial conmients (recommendat ions, additions,
any pertinent
data
which
may
and
be of usa in improving this
Hanscom
document should be aedressed to: Commander, liq tXD/ALET,
AFB, MA 01731 by using the se~f-addressed Standardization Document
Improvement Proposal (DD Form 1426) appearing at the ●nd of
this
document or by letter.
1-
.
●
MIL-STD-1521B
FOREWORD
ts
take
advantage
This standard has been designed
advanceme!?t and mafiage!ne?tprocedures
technological
reviews and autiits.
‘o
i .’
i;;/~,,
.—-,.
of
Cur::n:
:n conducting
I
: 141L-STD-1521B
CONTENTS
Paragraph
Page
SCOPE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
::1
1.2
1.3
Purpose .......................................
Classification . ...............................
Application ............. ......................
1
1
1
1
2.
RSFERSNCED
DOCU?4EW”
*S.................... ......
3
3.
3.1
3.2
3.3
3.4
3.5
3.6
5
H
3.9
3.10
DEFINITIONS ...................................
System Requirements Review (SIR) ..............
System Design Review (SIR) ....................
Software Specification Review (SIR) ... ........
Preliminary Design Review (PDRI ...............
Critical Design Review (cOR) ..................
Test Readiness Review (TM) ............... ....
Functional Configuration Audit (FCA) ..........
Physical Configuration Audit (PA) ............
Formal Qualification Review (FAR) .............
Production Readiness Review (PAR) .............
4. 4.1
4.1.1
4.1.2
4.1.3
4.2
4.3
GEIJSRAL REQUI RsFIENTs . . . . . . . . . . . . . . . . . . . . . . . . . .
Contractor Participation and Responsibilities.
Subcontrnc%ors and Suppliers ..................
Location ......................................
Contractor Requirements. ......................
Contracting Agency Participation ..............
Sample Forms ..................................
:
10
11
5.
DSTAILED REQUIRSMSNTS.
13
. . . . . . . . . . . . . . . . . . . . . . .
NOTES ................. . . . . . . . . . . . . . . . . . . . . . . .
Intended use .......... . . . . . . . . . . . . , . . . . . . . . . .
Data requirements list and cross reference. ...
Changes from previous
sue . ............. .....
v
:
5
5
:
:
6
-i
9
;
15
15
15
1s
1
HIL-StD-lS21B
F IGURSS
Page
Section
Figure
1
Figure 2
Figure 3
Figure 4
i%qlneerlng and .est Flow.... .................
Sample Action Icem Fem .......................
Sample Certification Attachment (?O.) .........
Sample Certification Attachment (Pm) .........
16
;;
96
APPENDIXES
Page
Appendix
A.
System Requirements
Review
B.
System Design
c.
Software Specification
D.
Preliminary
(SRR) ..............
19
. . . . . . . . . . . . . . .
23
SIR) ...........
31
Design Review (PDR ...............
33
E.
Critical Design Review (CDR).. ...............
53
F.
Test Readiness
69
G.
Functional Configuration
H.
Physical Configuration
I.
Formal Qualification
J.
Application Guide
K.
Production
Review (SDR)
Review
Reviev (TAR) ...................
Audit (FCA) ..........
Audit
Review
71
(PA).............
75
(FAR) .............
83
for Tailoring MIL-sTD-1521. . 117
Readiness Review
vi
(P!7R)........... ... 123
●
MI L-STD-J5Z1S
10
SE~lO?J
1
In
SCOPE
for :he
This standard prescribes the requirements
1.1 Purpose.
Re%iews and Audits on Systens, Equipments,
conduct
or Technical
and Computer Software.
reviews
The following technical
and audits
1.2 Classification.
be selected by the program manager a: the appropriate phase
shall
described
Zach review/audit is generally
of program development.
more
specifically
defined
in a
Definitions,
and
in Section
3,
separate appendix.
Systam Requirements Revie-~ (S2R)
System Design Review (SDR)
Software Specification Review (SSR)
Preliminary Design Review (?DR]
Critical Design Revie’4 (CiIR)
Test Readiness Review (TRR)
Functional Configuration Audit (FCA)
?hysical Configuration Audit (PCA)
Formal Qualification Review (FQR)
Production Readiness Review (PRR)
NCITE: A typicaI ●ngineering end test flow relative
activities is illustrated in Figure 1.
to
program
herein
Rev iews and
Audits defined
1.3 ADV lication.
Technical
shalr
be conducced in accordance with this standard to the extent
(sow) , and
specified in the contract clauses, Statement of Work
the Contract
Data Requirements List.
Guidance in applying this
standard is provided in Appendix J. The contracting agency shall
this standard
to requi re only vhat is needed for each
railer
individual acquisition.
10
1/2
I
141L-srD-ls21n
REFERENCED
DOCUMENTS
The
2.1 Reference documents are not included in this document.
Statement of Work shall be referenced for applicable documents.
and
standards,
dravings,
(Copies
of
specif.icacions,
required
by contractors
in connection with
publications
f?om the
specific procurement functions should be obtained
or .as directed
by the contracting
contracting
agency
officer).
‘o
3/4
l’llLSllJ
Q
SECTION
1“
1
,0
3
DEFINITIONS
TSCHNICAL
I
15215
REVISW
AND AUDiTS
The objective
of this
3.1 System Requirements
iteview (sRR) .
review IS to ascertain th~qua~the
contractor’s efforts in
requirements.
1:
will
be conducted
when
a
defining
system
significant portion of the system functional requirements has been
●stablished.
3.2 System Design Reyiew. (SDR). This review shall be conducted to
evaluate
optlmlzacion,
correlation, Completeness,
the
and risks
with
Also
the al~o~ated
technical
requirements.
associated
is a summa ry review of the system engineering process
included
which produced the allocated technical
requircmencs
and of the
for che nmxt phase
of effort.
engineering
planning
9asic
and planning
for
manufacturing consi~erations will be reviewe~
phases will be addressed.
in subsequent
production
engineering
●ffort
This review will be conducted when the system definition
point vhe re system
characteristics are
has proceeded
to the
defined an~ the configuration items are identified.
Specification
Review
the
(SSR).
A review of
3.3 Softwere
finalized computer software Co-a:~a
(CSCI ) require.meats
concept.
CSC I
The SSR
is conducted
and operational
when
have been
requirements
sufficiently
defined
to ●valuate. the
contractor’s responsiveness to and interpretation of the system,
or prime
A successful SSR is
item Level requirements.
segment,
predicated upon the contracting agency’s
determination
that the
Requireme!?ts
Interface
Requirements
Specification,
Software
and Operational
Concept
Document
form
Specification(S),
for proceeding
basis
into preliminary
softvar~
satisfactory
design.
.
;iidu%+%?
WcxraiK~~Lso~e”’&r%~~
S;
technical
configurateion itema to (1) ●valuate
the progress,
adequacy,
and risk resolution (on a technical, cost, and scheduie
basis)
of the
with
‘elect$rf%Xe
ap~;~chGn&;!eZ~~eZ~~ia2;~
compatibility
requirements of the Hardware Configuration Item (HwCI) development
(3) -evaluate the degree of definition and assess
specification,
with tI.e selected manufacturing
the technical risk associated
existence
and
(4)
establish
methods/processes,
and
the
Compatibility of the physical and functional interfaces amcr.g :he
items of equipment, facilities,
con~igurat ioii item and other
For CSCIS,
this review will
c~mputer so:tware, and personnel.
(~) the evaluation of the progress, consistency, and
focus on:
top-level
design
and test
technical adeqtiacy of the selected
approach,
(2) comparability
b:::een
software
requ~reme:;s and
the
preliminary design. ant? (3) on
..
preliminary
version
,0
5
MIL-STD-1521B
operation
anti support documents.
Critical DeSicn a:view, lCDR). This review shall be conducted
Ion item when detail design is ●ssentially
for each co-at
The purpose of this review will be to . (1) detemaine
complete.
that the detail
design
of the configuration item under review
satisfies the performance and engineering
specialty
requirements
of the HWCI development specifications, (2) ●stablish the detail
design compatibility among the configuration item and other
it ems
of equipment,
facilities,
computer
software and personnel. (3)
assess configuration item risk areas (on a technical,
cost , and
schedule
basis).
(4) assess
the results of the producibility
hardware,
analyses
and
conducted
on system
(5) review
the
hardware
product
preliminary
specifications.
For CSCIS, this
review will focus on the determination of the acceptability of the
and test characteristics of the
detailed
design,
performance,
design solution, and on the adequacy of the operation and support
documents.
3.5
I
3.6 Test Readiness Review (TRR). A review conducted for ●ach CSCI
to d=mlne
whether the s~e
test procedures are complete and
to assure that the contractor is prepared for fonrial CSCI testing.
are evaluated
test procedures
with
Software
for compliance
software
test plans and descriptions,
and for adequacy
in
test requirements.
accomplishing
At TRU, the contracting agency
informal software
also reviews the results of
zest ing and any
updates
to the operation and support documents.
A successful TRR
is predicated
on the contracting agency’s determination
that the
and
informal test results
software
test procedures
form a.
satisfactory basis for proceeding into formal CSCI testing.
●
Audi t (FCA).
A formal audit to
3.7 Functional Configuration
valiaate
~f
a configuration item has been
that the development
completed satisfactorily
and that the configuration
item has
che performance and functional characteristics specified
achieved
in the functional or allocated configuration
identification.
In
completed operation and support documents shall be
addition,
the
reviewed.
3.8 Ph sical Confi uration Audit (pCA)
of +ignat~
conf;-i~td
‘~Zhn;z~~;/xZ~!a?%
configuration
item “As
conforms
to
technical
documentation which defines the configuration item.
3.9 Formal Qualification Review (FQR). The test,
inspection,
or
anal=
process
by ~
a group of configuration
items
to
comprising
the system are verified
met
have
specific
contractual
contracting
agency
performance
requirements
(specifications or equivalent).
This review does not
~cA a;:;y to
hardware
or software
requirements
verified
at
che
individual configuration item,
6
●
+fIL-STD-1521B
3.10 Production Readiness Review (PRR). This rev ● v is intended
to d~the
stacus~c~ion
of the specific actions
which must be satisfactorily aceompiis;ed
prior
to executing
8
production
go-ahead
decision.
The review is accomplished in an
Development
incremental
f;;~~;;l during
the
Full-Scale
phase.
t.Vo
rev ievs and one final review to assess the
usually
risk in exercising
the production
go-ahead
decision.
In its
the
PRR
concerns
earlier
stages
itself with
gross level
manufacturing concerns such
as the need
for identifying
high
risk/low
yield
manufacturing
processes
or materials
ar zhe
requirement for manufacturing developmentefforz
to,satisfy aesign
requirements.
‘The
reviews
become
more
refined
as the design
matures,
dealing
With
such concerns
as ‘production planning,
incorporation
of producibility-oriented
facilities. allocation,
changes, identification and fabrication of tools/test
equipment .
long lead item acquisition etc. Timing of the increment&L PRRs is
a function of program posture and is not specifically locked in to
other reviews.
OTHER DEFINITIONS
3.11 For further guidance
on
DOD I
5000.33,
edition
of
Definitions.
●
1
terminology
see
cost
Uniform
Budget/Cosz
the latest
Terms
and
3.12 New titles are being phased in for Eke levels of maintenance.
(with
terms) :
On
They
are
their
former
Equipment
(Organizational), Off Equipment
- On Site
off
(Intermediate),
the latest edition of AFR
*~~nt
- Off Site
(Depot).
See
Equipment
14nincenance
Policies,
and
Objectives,
Responsibilities.
3.13 For definitions of the various
levels of repair,
see
the
latest
edition
of MIL-STD-280A, Definition of item Levels, Item
Exchangeability, Models, and Related Terms.
3.14 Configuration item. Hardware or software, or an aggregation
which
is designated
by the contracting
agency for
of both.
configuration management.
‘o.
7/8
,1
MIL-STD-1521B
SKTION
4
GE.NSRAL R5QUIREMWTS
The contractor
4.1 Contractor ?artici~ation and Responsibilities.
the Technical Reviews and
for ~ductlng
be responslDle
shall
requi-rements except :as
Audits in acco~dance with the followi;g
amended by the contract.
I
I
The contractor
shall
be
4.1.1 Subcontractors
anC Suppliers.
vendors,
that subcontractors,
and
in~ing
responsible
for
suppliers participate in formal Reviews/Audits, as appropriate.
.
4.1.2 Location.
Unless otherwise specified in” the Statement
of
Work,
the Reviews/Audits
shall be conducted at the contractor’s
facility or at a designated subcontractor facility, if approved by
Accordingly,
the contractor shall be
agency.
the contracting
to
and material
the necessary
resources
required to provide
This includesa~~e following
perform the Review/Audit effec~ively.
for the
scope of
appropriate
type
items to zhe ● xtent
Review/Audit required by the contract:
a.
Meeting agenda/plahs
b.
Conference
c.
data,
specifications.
system
engineering
Applicable
drawings, manuals, schedules, and design and test data
d.
Specialty
e.
Trade study results
f.
Risk analysis
9.
Mockups,
hardware
h.
Test methods
i.
Meeting minutes
room(s)
study results
results
breadboards.
in-process
hartivare,
and
finished
and data
shall
be
The
contractor
Requirements.
4.1.3
Contractor
for eszabi:snlng :fie time, place and acjenda for each
responsible
Revicv/Aud:: in consonance with the master .milestone schedule,
Thls should
subject co coordiadation with the contracting agency.
to
be accomplished sufficiently in qdvance of each Review/Audit
a~lo”
adequaxe preparation for the nee:in~ by both the contractor
In addition, che contractor
and the contracting agency (see 6.2).
shall:
●
with,
4.1.3.1 Insure that each Fteviev/A,udiEscheduie is compatible
information
contract
necessary
and
the
availability
of the
141L-sTD-1521B
study results,
engineering
data,
trade
articles, e.g., system
results.
results,
risk
analysis
producibility
analysis
specificati~ns. cenzals, t!rawicqs, reports,
hard=a:e. soft-~sre, or
mockups.
detail
Prepare
for ●ach ReviewlAudit
in sufficient
consistent with the scope and magnitude of the Review/Audit.
4.1.3.2
Review/Audit.
for
each
4.1.3.3
Designate
a Co-Chairperson
contractor
personnel
or chose
and subcontractor
Participating
to discuss
in
chosen to make presentations
shall be prepared
technical detail any of the presented material within the scope of
the review.
acceptable
method
to
4.1.3;4 Provide a stenographer
or other
Minutes shall be
meeting
minutes.
record
inputs to official
Co-Chairperson
and shall
recorded only as dictated
by either
and answers.
action
items,
consist
of signif icant questions
deviations, conclusions, recommended courses of act ion resulting
from presentations
or discussions.
Conclusions from discussions
in the main
conducted auring side meetings shall be summarized
meeting
at an appointed time, and appropriate comments shall be
minutes.
read into the offlc~al
Recommendac ions not accepted
with
for
together
the reason
recorded
should
also
be
The minutes
non-acceptance.
of each daily seszion
shall be
available for review by both the contractor.a.nd contracting agent
personnel at the conclusion of each day’s session (see 6.2).
*
and
record all ection
items in che minutes
4.1.3.5 Clearly
agency and/or contractor action is
identify whether
contracting
(See Figure 2 fur Sample Action Item
required for its resolution.
Form) .
4.1.3.6 Publish and dis:rihu:e
4.2 con:ractin~
~
official minuzes.
ParKici~ation.
4.2.1 Serves as CO-Chairperson.
4.2.2 Provides the name, organization, and security
clearance
of
each participating
individual
to the Contractor prior to each
Review/Audi C.
4.2.3 Reviews the daily minutes and ensures
sicjnificant con:Yacting agency inputs.
that they reflect
all
of the
4.2.4 Provides formal acknowledgement to the contractor
of each Review/Audit after receipt of Review/Audit
accomplishment
the
The contracting
agency establishes
minutes
(see 6.1).
adequacy
of the contractor’s review performance by notification
of:
a.
Approval ‘- to indicste that the review
10
was
satisfactory
a
●
ZMIL-STD-1.521B
completed.
t
b.
contingent approval -- co indicate that the rev iaw is not
considered accomplished until the satisfactory ccmple:ion of
resultant ac:ion items.
c.
Disapproval
inadequate.
‘- to indicate
that
the
review
4.3 .samDle Forms.
A sample
item form
action
certi~on
attachment
are provided ‘for guida%e
Figures 2, 3 and 4).
I
11/12
.4
as
seriously
and
sample
‘purposes (;see
..
)41L-STD-1521B
*
SECTICN
DETAILED
5
RSQUIWMENTS
Audits will
Reviews
or
be conducted
as
appropriate
5.1 The
specified in the following appendices (as selec:ed and/or modified
in the contract):
5.1.1
system !7equirenents aeview.
5.1.2
System Design aeview.
See Appendix
5.1.3
Software Specification
Review.
5.1.4
Preliminary
5.1.5
Critical
5.1.6
Test Readiness Review.
5.1.7
5.1.s
Physical Configuration
5.~.9
Formal Qualification
Design Review.
Application Guide
Appendix J.
,5.~.11
Production
See Appendix C.
See Appendix
D.
See Appendix F.
Audit.
Audit.
Review.
For
Readiness
B.
See Appendix E.
Design Review.
5.1:10
See Appendix A.
See Appendix H.
See Appendix
Tailoring
Review.
13/14
See Appendix G.
I.
)41L-ST9-1521. . See
See Appendix K.
MIL-STD-1521B
●✿
SECf’ION 6
!
No’!’”ss
for
6.1
Intended use. This standard prescribes the requirements
Reviews
Tecnnlca~
contlu~
—
and Audits on Sys:ems, Equipments,
Software.
Official
acknowledgement
by
and Computer
the
contracting
agency of the accomplishment of a Fteview/Audi: is =ot
to be incerpreced as approval of statements made in the minutes or
of matters discussed at the aeview/Audit and does not relieve the
contractor fro= requirements which are -a part ‘of the contract.
●
I
Whea
list and cross
6.2 Data
requirements
reference.
this
In an~ui~io~ch
Incorporates a DD Form
stand=
is used
Data
Requirements
1423: Contract
List
ICDRL),
the
data
below shall be developed as specified by
requirements
identified
an approved Data Item Description (DD Form 1664) and delivered
in
with the approved CDRL incorpora:cd iato the contract.
accordance
When the provisions of the DOD FAR c labse on data
requireme~ts
DOD FAR Supplemen: 52.227-7031) are invoked and the DD
(currently
Form 1423 is not ur,ed. the data snecified below shall be delivered
in accordance
bv the contractor
with the contract or ourchase
o;tler requirements.
Deliverable data reauired bv this stahdard is
cited in-the following paragraphs.
Paraqraph
No
-—
Data Requirement Ti:le
No
AP plicable —
DID -
4.1.3
Conference
Agenda
i)I-A-7088
4.1.3.4
Conference
Minutes
DI-A-7089
and
identi i ed
(Data item descriptions related to this standard,
listed as such
in DOD
section
6 wiil be approved end
II,
AMSDL. Cop”ies of data
item descript ons
i~O0.19-L. , vol.
the
required
by
contractors
in connection
with spec fic
should be
acquisition ‘functions
obtained
from
the - Naval
Publications
and Fo rMS Center or as directed by the contracting
officer. )
6.3 Chanoes from previous issue. Asterisks or verzica: lines%re
not ~
~
revision ~ntify
changes wi:h respect to the
previous issue due to the expensiveness of :he chazses.
15
●
1
●
(
“o
,.
MIL-STD-1521 B
Camsa. !lmlBm.—.
IaA= or .-X
@UT:ON
I SCW7:
I
I
.XIG2YATC!R
USIGND ‘lw:
b--
1------
-
““
‘-”
‘-:
~
““-’
17/18
I
I
MI L-STD-1521B
<APPENDIX A
Review
. .
(SRR).
The SiGts ● re normally conducted during
the system
10.1 Generai.
or Demonstration and validation phase.
5xp10ration
Conceut
Such
vill be
revieifs say” be conducted
at any
time M t normaljy
the accomplishment
of functional analysis and
af:er
conducted
requirements
preliminary
allocation
(to
Hardware
Configuration
operational/maintenance/training
Items
(CSCIS),
facility
[HWCIS), Computer Software Configuration Items
items, +uanuf-acturing considerations, -personnel and
config~ration
human factors) to determine initi-al direction and progress of the
Engineering
System
Management
effort and
contractor’s
his
convergence upon an optimum and complete configuration.
.
The total System
Management
Engineering
activity
10.2 Purpose.
and lts output
shall be reviewed
for responsiveness
to the
requirements.
Contracting
Statement of Work an~ system/segment
agency direction to the contractor vill be provided: as necessary,
for continuing the technical program and system optimization.
Representative items
to
10.3 Items to be Revieved.
inclu-e=e=lcs
of the following, as appropriate:
●
a.
Mission
b.
Functional
F1OW Analysis
c.
Preliminary
Requirements
d.
System/Cost
Effectiveness
e.
Trade
(e.g., addressing
Studies
hardware/firmware/software)
f.
Synthesis
9.
Logistics
}
I
I
and Requirements
be
revieved
Analysis
Allocation
Analysis
system
functions
in
Support fialysis
Specialty Discipline Studies (i.e., hardware
and Softvare
,reliability analysis, maintainability
aiIialysis, anaament
integration,
electromagnetic
compatibility,
.survivability/vul.ner.abilizy (including nuclear), inspection
management,
methods/teck3iques
analysis,
energy
-environmental considerations).
.
i. System Interface Studies
h.
o
.
].
Generation
of Specifications
k.
Program
1
..
Integrated Te.Si ?ianning
Risk Analysis
19
MI L-STD-1521B
APPENDIX A
m.
Producibility
Analysis Plans
n.
Technical
o.
Engineering
P.
Data Management
q.
Configuration
r.
System Safety
s.
Human Faccors Analysis
t.
Value Engineering
Performance Measurement
Planning
Integration
Plans
uanaqemenc
?lans
Studies
u .. Life Cycle Cost Analysis
v.
Preliminary
w.
Manpower Requirements/Personnel
x.
Milestone Schedules
10.3.:
Manufacturing
Plans
Analysis
The contractor shall describe his progress and problems
in:
,ranking
and
risk
Risk
(the
identification
10.3.1.1
among system effectiveness analysis, technical
interrelationship
performance measurement, intended manufacturing methods, and costs
shsll be discussed, as appropriate) .
a
avoida>ce/reduction
control
Risk
and
(the
10.3.1.2
interrelationships with trade-off studies, test planning, hardware
measurement
shall
be
and technical
performance
proofing,
discusses, es appropriate).
trade-offs
stated
system/segmenz
Significant
among
10.3.1.3
requirements/cons:ra ints and resulting engineering
specification
methods/process
requirements/constraints ! manufacturing
design
ownership
loglstic/cost
of
and
constraints,
cost/design-to-cost
requirements/constraints and unit production
objectives.
Identifying
computer
and
10.3.1.4
resources
of the
syszem
in:o hwC IS and CSCIS.
Include any
t?e system
partitioning
:rade-off studies concucted co eva:.~are alternative approaches and
methcds f~r meeting operational needs and to determine the ●ffects
of
include any evaluations
of constraints on the system. Also
resource
limitations,
logistics,
cost , schedule,
technology.
inpact on
ir.teliigence ●stimates, ● ::., made to de:ermi.ne tneir
:he system.
Ia aadition, address the following specific tradeoffs
:e~ated to :omputer reSCurCes:
20
I
●
●✎
Candidate programming languages and computer
architectures
●valuated
in light of DoD requirements for approved higher
order languages and stantiard instruction set architectures.
b.
Alternative approaches ●valuated for iriplementing secxrity
If an approach has been selected, discuss how
requirements.
it is the most economical balance of ●lements which meet the
total system requirements.
c.
Alternative
approaches
achieving
the
identified
for
and, for joint service
operational
and support
concepts,
‘programs, opportunities for interser~ice support.
which
10.3.1.5
Producibility
and ‘manufacturing
considerations
could
the program
decision such as critical components,
impact
materials and processes, tooling and test ●quipment
development.
production
methods !
long
and
testing
lead
items,
facilities/personnel/ski 11s requirements.
10.3.1.6 Significant hazard consideration should be made
here
to
develop requirements and constraints ro eliminate or control these
system associated hazards.
●
\
10.3.2 Information ‘tihichthe contractor identifies as being useful
to his analysis and available through the contracting agency shall
review
prior
to this
be
requested
(e.g.,, prior
studies,
cost
factors,
factors,
safety
data,
test
operational/suppOrt
A separate SXR may be conducted for each of the
plan(s), etc.).
the nature and
operational
support
subsystems
depending
upon
complexity of the program.
After completing the SRR, the contractor
10.4 Post Review Action.
shall~~~and
distribute
copies
of Review
The
minutes.
contracting agency officially acknowledges completion of the SRR
as indicated in paragraph 4.2.4.
!
o
21/22
—
MIL-STD-1521B
APPSNDIX B
20.
System Desiqn Review
(SDR).
20.1 Generai.
The SDR shall
be conducted
to evaluate
the
“—
traceability,
correlation,
~ompleteness,
and the
oqcimlzatlon,
including
risk of the allocated requirements,
the corresponding
requirements
in fulfilling the system/segment requirements
test
The review ●ncompasses
the total
(the functional
baseline).
systaa
requirements,
i.a., operations/maintenance/test/traininq
preliminary
hardware, computer software,
facilities,
personnel,
AISO included shall be a summary
logistic support considerations.
review of the System Sn’ginee’r
Activities
(e.g.,
ing Management
land
analysis,
mission
requirements
analysis,
functional
selection,
requirements allocation, manufacturing merhods/process
analysis,
prug ram
risk
system/cosc
analysis,
●ffectiveness
trade studies,
logistics support
analysis,
intra- and interintarface
test planning, specialty
syatem
integrated
studies,
which produced
discipline studies, and Configuration Management)
the above
system definition products.
A technicai understanding
shall be reached on the validity and the dearee of completeness of
the following information
Spec fication
a.
System/Segment
b.
The engineering des gn/cost of the system
Definitions) .
c.
Preliminary
Dperetional
d.
Preliminary
Software Requirements
●.
Preliminary
Interface Requirements
$.
.
Aa appropriate:
Section
3,
Concept Document
(1) Prime Item Development
(2) Critical
(see
Specification>
Specification(s)
Specification
Item Development
Specification
An SDR shall be conducted as “the final review prior
%“2t%%%%ittal
of the ~ns~~;;~on
and valida;~ye;ph;~;
products or as the initial
Development
sys t●ms noc requiring a formal Demonstration and Validation Phase
but sufficiently complex to warrant the formal assessment
of the
(and the basis of these requirements)
allocated
requirements
before proceeding vish tune preliminary
design
of :3WCIS or the
detailed
requirements
analysis
for CSCIS. The SDR is primarily
concerned vi:h
review of the operational/support
the overall
requirements
(i.e., the mission requirements), updated/completeti
System/Segment Specification requirements,
allocated
performance
manufacturing
requirements,
programming
and
methods/processes/planning,
and the accomplishment of the System
Management
activities
Zngi.qeering
ta insure :hat the definition
23
1,
MIL-STD-1521B
APPENDIX B
●ffort products
SDR are to:
are necessary
●
The ;ar?oses of the
a!td sufficient.
that
the
updatedlcompleted
Insure
20.2.1
and Cost ,eff~ctive
Specification
is adequate
validated mission requirements.
Sysrem/Segment
j~
satisfying
20.2.2 Insure that the allocated requirements represent
and optimal synthesis of the system requirements.
program
20.2.3 Insure that the technical
ranked, avoided, and reduced through:
risks
are
a complete
identified,
a.
(particularly
mission
for sensitive
trade-offs
Adequate
versus
engineering
realism and manufacturing
requirements
feasibility to satisfy the anticipated production quantities
of corresponding performance requirements):
b.
Subsysrem/cOmponent
C.
A
d.
disciplines
engineering
Implementation
of comprehensive
vors t case analysis,
failure mode
and effects
“(e.~.,
analysis
producibility
analysis, maintainability analysis,
and standardization. )
responsive
hardware proofing;
test program;
and
●
operations,
how the final
combination
of
Identify
20.2.4
maintenance,
logistics
manufacturing,
and test and activation
progrem
concepts:
quantities
requirements have affected overall
3,
(see Section
cost
equipment , unit product
and
anti
personnel,
software,
Defin;%~s,O~a:ag’raoh
3.12), compu:er
facilities.
20.2.5 Insure tha;n~ technical understanding
direction
technical
reached
been
contractor.
of
is
requirements
provided
to
The SDR shall include a review of
20.3 Items to be Reviewed.
follo~i=m~
as appropriate:
20.3.1 System Engineering Management
a.
t4ission and Requirements
b.
Functional
c.
Requirements
d.
5ystem/Cost
e.
Synthesis
Activities,
Analyzis
Analysis
Allocation
Effectiveness
2&
e.rj.:
has
the
the
●✍✎
MIL-STD-i5ZlB
APPENDIX B
f.
Survivabilisy~J’alterability
9.
Reliability/Maintainabi
h.
Electromagnetic
i.
appropriate,
Analysis
(to address
Logistics Supporx
including
l~istlcs support
integrate~
logistics
support
ftici.lities,
concept, maintenance, zsupply, :softvare su~port
etc.)-
j.
hazard
on system
System Safety (emphasis shall be placed
analysis and itienzification of safety test requirements)
k.
Security
1.
Human Factors
m.
Transportability
n.
system Mass Properties
o.
Standardization.
P-
Electronic
q.
Value Engineering
r.
System Grovth
s.
Program Risk Analysis
t.
Technical
u.
Protiucibility Analysis
and Manufacturing
v.
Life Cycle Cost/Design
to Cost Goals
w.
Quality Assurance’Program
x.
Environmental
Conditions
Humidity, etc.)
Y.
Training
z.
Milestone
lity/Availability
R/M/A )
Co~patibiLity
[including Packaging
and Har.dling)
Warfare
Capability
Performance
and Training
Measurement
Planning
(Temperature,
vibration,
“Shock,
Support
Schedules
.s8. Software Development
Procedures
20.3.2 Results of significant
,a.
(including nuclear)
Sensitivity
of
trade studies,
miss
selected
25
on
for example:
:equiremerts
versus
MIL-STD-1521B
APPSNDIX B
realistic performance
parameters
and cost estimates.
b.
Operations
desi5n versus maintenance
design
c.
System centralization
d.
Automated
e.
Reliability/Maintainability/Avai
f.
Commercially
9.
National Stock Number
h.
studies
(Allocation
of
Testability
trade
fault
detection/isolat ion
capabilities
between
●lements
of
built-in test, on boarWon-site
fault dezection/isolation
and
ma~ual
subsystems,
separate
support
equipment,
procedures)
i.
Size and weight
j.
versus
Desired
propagation
characteristics
systems
(optimum
interference
to
other
frequencies)
versus decentralization
versus manual operation
available
lability
items versus new developments
(NSN) ite!es versus new development
retiuction in
selection
of
k,
Performance/logiszics
1.
Life cycle cost reducticn for different
languages
m.
Functional allocation between hardware,
and personnel,’procedures
n.
Life Cycle Cost/system performance trade studies
sensitivity of performance parameters to cost.
o.
Sensitivity of performance
P.
Cost versus performance
~.
Design versus manufacturing
r.
Hake versus buy
s.
Software development
20.3.3 Up~ated
design
functions and item.
20.3.+ Updated
processes.
●
trade studies
parameters
computer programming
software,
to
firmware
include
versus cost
consi~eration
schedule
requirements
requirements
for
26
for
9peracions/maintenance
manufacturing
sethods
ant!
MI L-sTD-1521B
APPENDIX B
20.3.5 Updated operations/maintenance
‘(
20.3.6 Updated requirements
and training.
20.3.7 Specific actions
I
for facilities.
Operationslmaintenance
to be performed
personnel
include evaluations
of:
a.
System design
b.
selected
configuration
the
Capability
o’f
requis.ements of the System/Segment Specification
c.
Allocations
of
subsystems/configuration
d.
Use of commercially
e,
Allocated
f.
HWCIS
to
permit
of
Size, weight,
and configuration
packaging,
and ●ffective transportation,
and
●conomical
with applicable
specifications
and
hsndling
consistent
standards
9.
Specific “design concepts whic”h may
toward ●dvancing the state-of-the-art
h.
require
Specific subsystems/components which may
proofing” and high-risk long-lead time items
i.
items
to meet overall
inventory
system
The ability
of
vith configuration
and their compatibility
requirements.
item interfaces
.
1,
I
for
requirements
k.
feasibility
and systemlcost
to
tmeet
requirements
System
items
to
available and standard parts
inter- and intra- system
The planned system design in view
functions, as applicable
Cansideraticns
effectiveness
interface requirements
of
require
provitiing
development
‘hardware
multi-mode
given to:
(1) Interference caused by the external environment
to
system and the system to the external ●nvironment.
the
(2) Allocated
performance
characteristics
of all Sys:em
:0 identify potential
receivers
transmitters
and
intra-system electromagnetic (SM) incompatibilities.
(3) Non-design, spurious
and ha;ym~;
system Performance
on electromagnetic
and their
characteristics
environments of operational deployments.
●✎
1.
Value Engineering
studies, preliminary
Value Engineering
Change P=oposals (VECPS) and VECPS (as applicable).
27’
..
MIL-STD-1521B
APPENDIX B
●
20.3.8 Review the Preliminary Operational
Concept
Document,
azd
sections
1.0, 2.0, 3.0, 5.0, 6.0, and 10.0 of the Sys:em/Segment
HUC I Development
Specification,
Specifications,
all avaiiable
preliminary
Software
Requirements,
Requirements
and Interface
content,
Specifications
format,
adequacy,
for
technical
completeness
and
traceability/correlation
to the validated
mission/support requirements.
All entries marked “not applicable
(N/A)” or “to be determined (TBD)- are identified and ex?lained by
the contractor.
20.3.9 Review section 4.0 of the System/Segment
Specification,
all
available
hardware
Development
Specifications,
and preliminary
Requirements
Specifications
Software Requirements and Interface
All
for format,
technical adequacy, and completeness.
content,
and system
available test documentation: including NCI/subsystem
test plans,
shall be revleved to insure that the proposed test
program satisfies the test requirements
of section
4.0 of all
entries labeled ‘not applicable
All
specifications.
~~~;;$able
in sect ion 4.0
of
any
(T3D)”
or “to be determined
and explained
by the
applicable
speci fication are identified
contractor.
for interactim.
20.3.10 Review the system, HWCI, and CSCI design
with the natural environment.
If any effect or interaction is no
or
it is
ch~w~et;~~ understock and further study
is required,
for in the design, the
not completely
compensated
All
resolution
~lSO b= reviewed.
proposed
method
of
shall
proposed
environmental
tests shall be reviewed for compatibility
with the specified natural environmental conditions.
to
by the contractor
20.3.11 Maintenance
funct ions developed
determine
that support concepts are valid, technically feasible,
and understood.
In particular, attention is given to:
a.
R/H/A
considerations
Specification
b.
Maintenance
c.
Corrective
in
the
design characteristics
of the system
and preventive maintenance
.S. Special eqaipment,
System/Segment
updated
tools, or material
requirements
required
or planning for automated.maintenance
●.
Requirements
f.
Analysis
compatibility
Item Maintenance
maintenance program when weapon is deployed
9.
Specific configuration
h.
Forms, procedural,
item support
and techniques
28
with
analysis
required
requirements
for maintenance
analysis
●
FfIL-sTD-1521B
APPENDIX B
i.
Maintenance related trade-off studies and findinas (ineluties
commercially
available equipment, software faul~ diagnostic
techniques)
].
~ogiscic
K.
Support procedures and tools
for
facilitate
software modification,
and updates
1.
Hardness
cost impacts
critical
which
computer
software
improvements, corrections
items/processes
non-nuclear
and laser
20.3.12 System compliance
with nuclear,
requirements.
High
risk areas or design
concepts
hardening
requiring possible advances of the s:ate-of-:he-art as a result of
identified,
survivability
critieria
shall
be
and prepared
Prec.ared test nroarams
reviewed.
a9DrOaCh(tS) to the oroblem
ancl compatibility ~it~ the
shill
be reviewed
ior sufficiency
threat
and
existing
test
specified
environment
sinulacion
facilities.
1
●
,
and risks
completeness,
20.3.13 The optimization, traceability,
with
the allocated
technical
:.equirements, and the
associated
requirements
as a
basis
for
adequacy
of allocated
system
with
proceeding
development
of hardware
the
and software
preliminary
Software
configuration items. Include any available
Requirements and Interface Requirements Specifications.
20.3.14 Manufacturing
(HUCIS onlv)
—-
20.3.14.1 Production feasibility and risk analyses
addressed
at
:he S.?..shall be updated and expardeti. This efforz should review
the
risk and evaluate
the progress made in reducing production
in the Full Scale Development
risk. remaining
for consideration
Phase.
Estimates of cost and schedu:e inpacts shall be updated.
20.3.14.2 ReVieW of the
include:
Production
Capability
Assessment
shall
production
capability
shall
be
of
20.3.14.2.i
A review
which
will
constitute
an assessment
of the
accomplished
equipment
and skills
facilities, ma:erials, methods, processes,
full scale deveiopmen: acd procuc:ion
necessary
to perform
the
or develop
efforts.
Identification of requirements
to upgrade
Requirements
for
manufacturing
capabilities
shaii be made.
will
Technology
Drocrams
also
be
Manufac?urinq
(MANTXH
1
.
.
. .
identified a= an element of this production assessment.
Present
manasemenc
“and
the
controls
20.3.i4.3
design/manufacturing
●ngineering approach :0 assure chat
equipment is producible.
29
the
the
MIL-STD-1521B
APPENDIX B
I
20.3:14.4
Present
●
review of trade-off
studier
for design
producibility,
reqylrements
against
requirement
the
for
facilities, toolin~, production test ●quipment,
inspection,
and
capital equipment for intended production rates and volume.
20.3.14.5 The analysis, assessments and trade-off
studies should
recommend any additional special studies or development efforts as
needed.
20.4
Post Review Action.
After completing the SDR, the contractor
shal 1~ti~a~tribute
copies of Review Minutes.
The
contracting agency officially acknowledges completion of the SbR
.
30
●
MIL-STD-1521B
APPENDIX c
*
30.
I
Softvare
Specification
Review
(SSR).
reviev of a CSCi”S
The SS!? shall
be a formal
30.1 General.
Requirements
specified
the
Software
requiSpecification(s).
Interfac~n Requirements
the
Specification a%
Normally, it shall be held after System Design Reviev but prior to
A collective
SSa
for a
the start of CSCI preliminary design.
items, treating each configuration item
qro~p
of configuration
lndavidually, may be held when such an approach is advantageous to
Its purpose is to establish the allocated
the c?ntractinq agency.
basellne for preliminary
‘CSCI design
~;e demonstrating
‘to the
agency
the
adequacy
of
contracting
Requirements
Interface
Specification(s)
(SRS),
~~R;~fication
, and Operational Concept Document (oCD).
The contractor
30.2 Iteats to be reviewed.
or review by the contracting
follo~i=m=
●
the
inputs.
the
CSCI,
including
overview
of
and OUCPU:S of each function.
a.
Functional
processing
b.
those
for
Overall CSCI performance requirements, including
and
similar
requirements,
execution
time,
storage
constraints.
c.
Control
‘1
“
flow
functions
and data
flow between
that comprise tne CSCI.
each
of
the
software
d.
and all other
All interface requirements between the CSCI
to the
configuration
internal
and external
iteras both
system.
e.
levels
applicable
Qualification requirements that identify
of testing for the software requirements zhat
ana methods
comprise the CSC1.
f.
-y
9.
Quality factor requirements: i.e., Correctness, Reliab~l+ty,
Haintainablllty.
Integrity,
Efficiency,
Usability,
Reusability,
and
Portability,
Testability,
Flexibility,
InterOperability.
h.
.Mission requirements
of the system
operational and suppor~ environments.
i.
Functions and characteristics
the overall system.
j.
Milestone
k.
Updates
~
I
shall present
agency:
special delivery
requirements
for the CSCI.
and
its
associated
of the computer system
within
schedules.
since the last review to
31
ali
previously
delivered
1
MIL-sTD-1521B
APPENDIX c
!
I
I‘
software
1.
related CDRL
items.
tiy actions or procedures
&eviating
from approved plans.
I
After completing the SSR, the contractor
Post Review Action.
Review
Minutes.
The
copies
of
shall~ti=a~tribute
contracting agency officially acknowledges completion of the SSR
1!
30.3.L
The accomplishment of the SSR
shall be recorded
on the
Record
by the contractor (see
configuration
item Development
UIL-STD-483, Appendix VII).
30.3
.
J
I
32
●
PIIL-STD-15Z19
APPENDIX D
40.
Preliminaryy Design Review
(PDR)
The PDR shall be a formal technical review of the
40.1 General.
for a configuration
for a
approach
item or
design
basic
It shall
be
functionally related group of configuration item+.
heid after the hardware Development Specification(s), the Software
(STP),
Top Level Design Document (STLDD), the Software Test Plan
and preliminary
Plan,
versions of the Computer
the HUCI Test
(SUM),
System Operator’s Manual (CS2M), Soft”*are User’s Manual
Computer
System
Diagnostic Manual (CSDM). and Computer Resources
but prior
to
Integrated “Support Document (CRISD) zare available,
design.
For each configuration item the
the
start of detailed
actions described below may be accomplished as a single event.
or
be sFread over several events, depending on the nature
they may
item. and
and the extent of the development of the configuration
A
specified
in the contract Statement of Work.
on provisions
each
collective PDR for a group of configuration items, treating
configuration izem individually, may be held when such an approach
PDR
is advantageous to the contracting agency: such a collective
over several
events,
as for a single
also be spread
may
overai 1
The
program
r isks
cor.figuration item.
technical
with each configuration item shall also be reviewed on
associated
For software, a technical
a technical, cost, and schedule basis.
shall be reached on the validity and the degree of
understanding
versions
of
completeness of the STLDD, STP. and the preliminary
the CSOM, SLE4, CSDt.1,and CRISD.
shall
The contractor
40.2 Items to be Reviewed.
by the contracting agency:
follo~f~
Gview
present
the
40.2.1 HUCIS:
1
a.
Preliminary design synthesis
of
the hardware
Specification for the iteE being reviewed.
b,
Trade-studies and design
studies
20.3.2 of SDR for a representative
c.
Functional
diagrams.
d.
dmwings
preliminary
Equipment
layout
and
any
proprietary
including
or
design/process/components and information.
●.
Environment
f.
Electromagnetic
9.
Power distribution
h.
Preliminary
flow, requirements
control
resuits
listing).
allocation
Development
(see
paragraph
data, and schematic
drawings,
restricted
and thermal design aspects
compatibility
of the preliminary
design
and grounding design aspects
mechanical
and
packaging
33
design
of
consoles,
I
M! L-STD-1S2LB
APPENDIx D
a
I
racks, drawers, printed circuit
boards, connectors,
● tc.
i.
Safety
j.
Security
k.
Survivability/Vulnerabi
considerateions
1.
preliminsrg
m.
Pertinent
n.
Preliminary
weight data
o.
Development
test data
P.
requireutencs contained
item
Lnterface
in configuration
Development Specifications and interface control data (e.g.,
interface control drawings) derived from these requirements
q.
Configuration
r.
Mock-ups, models, breadboards,
●ppropriate
s.
Producibility
and
Manufacturing
Considerations
(e.9.7
materials,
test ●quipment, processes, facilities,
toolxng,
skills, and inspection techniques).
Identify single source,
sole .source, diminishing source.
t.
Value
VECPS
u.
Transportability,
v.
Human Engineering and Biomedical
considerateions
life support and Crew Station Requirements).
v.
Stan&ardization
I
x.
Description and characteristics
of commercially
available
equipment,
any optional
including
capabilities
such aS
interface
special features,
units, spec ia 1 instructions
o:
etc. ,
controls,
formats,
(include
limitations
commercially available equipment such as
failure
to mee~
human
end maintainability requirements
engineering,
safety,
of the specification and identify deficiencies).
I
r.
Existing
documentation
(technical
orders ,
commercial
available eqq~ipment an
manuals,
etc.,)
for commercially
copies
of contractor
specifications
used
to
procur
b
34
engineering
considerations
engineering
considerations
lity
(including
lists of materials,
nuclear)
parts. and processes
relability/maintainabi lity/availability
item development
scheaule
or
Engineering
Considerat ions,
(if applicable).
packaginq,
data
prototype
hardware” when
Preliminary
VECPS
and
and handling considerations
(including
considerations
,
1’
!
I
MI L-STD-15219
APPENDIX D
iO,,-
equipment
contracting
1,
z.
shall be
agency.
made
available
for
review
by
the
Firmware to be provided wi:h the system:
microprogram logic
diagrams and reprogrsuming/instruction
translation al~orithm
descriptions, f“;;;ca;;~;~epackaging
(integration technology
LSI,
types
(e.g., 040S, PSIOS))r and
(e.g.,
and
special
equipment
support
soft.aare
needed
for
developing, testing, and supporting the firmware.
.aa.
Life Cycle Cost Analysis
ab.
Armament
ac.
Corrosion
ad.
Findings/Status
compatibility
preventionlcontrol
considerations
I
I
of Quality
Assurance
Program
40.2.2 CSCIS:
a.
computer
functional
flow
Functional flow.
The
software
all of che requirements
embed y ing
allocated
fram the
software
Requirements
Specification
and
Interface
Retirements
Specification(s)
to the individual Top-Level
Computer Software
Components (TLCSCS) of the CSCI.
b.
This
information
shall
be
allocation
data.
Storage
presented for each CSCI as a whole, describing the manner in
which available storage is allocated to individual
TLcscs .
Timing,
requirements,
an~ relevant
equipment
sequencing
are to be
constraints used in determining the allocation
included.
c.
functions
the
Centrol
description.
A description
of
executive
control
and start/recovery features for the CSCI
initiating
system
shall be available, including method
of
operation
and
enabling
from system
features
recovery
malfunction.
d.
cSCI structure.
The contractor shall describe the top-level
of the CSCI , the reasons
for choosing
structure
the
components described, the development methodology which will
be used within
the constraints of the available compu:er
resources, and any support programs vhich “~ill be required
order
in
to develop[maintain
the CSCI
structure .and
allocation of data storage.
e.
An identification of unique security requirements
Security.
and
a description
of the techniques
to be used for
implementing and maintaining securi:y
within the CSCI shall
be provided.
I
a
.
35
MIL-sTD-1521B
APPSNDIX D
identification
of
An
reent~ancy
and 8 description
of thean~echniques
for
reentrant routines shall be available.
f.
Reentrance.
requirements
implementing
9.
Computer softvare de~elopment facilities.
The availability,
and planned utilization of the computer soft-dare
adequacy,
development facilities shall be addressed.
h.
development
software
Computer
facility
versus
the
operational
The
system.
contractor
shall
provide
information relative to uziGue design
features
which may
exist
in a TLCSC in order to allow use within the computer
software development facility, but which will not ●xist
in
TLCSC
ifi the operational
the
installed
system.
The
contractor
shall provide
information
on the design
of
support programs not explicitly required for the operational
to aasist
system
but which will be generated
in the
development
of the CSCI(S).
The contractor
shall also
Development
Software
provide
details
of the
Library
control=.
i.
The contractor
Development
took .
shall describe
any
data
special
simulation,
reduction, or utility tools that
are not deliverable under the tenma of the
contract,
but
which are planned for use during software development.
j.
Test tools. The contra:;;: shall describe any special
test
systems,
reduction
tools, test computer
test data,
software, or calibration and diagnostic
software
that are
not deliverable
under terms Of the contract. but which are
planned for use during product development.
k.
Description and characteristics
of commercially
available
including
any
optional capabilities such
computer resources,
as spatial features, interface units, special
instructions,
controls, formats, etc.
Include limitations of commercially
available
such as
equipment
meet
human
failure
to
engineering,
safe~y and maintainability requirements of the
specification and Identify deficiencies.
1.
Existing
documentation
(technical
orders,
commercial
manuals, etc. ) for commercially available computer resources
and copies of contractor
specifications
used
to procure
computer resourcaa shall be made available for review by the
contracting agency.
m.
The contractor
Support resources.
shall describe
those
resources
necessary
to support
the software and firmware
during
operational
deployment
of the system,
such
operational
and support
har~vare and software, personne?~
special skills,
human factors, configuration
management
,
test , and facilities/space.
I
36
e
●
.
●
1’
ML L-s”l’u-AaLLn
APPENDIX
D
e.
I
\
n.
Opera:ion and support documents.
The preliminary
versions
of the CSO!4, SUM, CSDU , and CRISD shall be reviewed for
with
czmpatability
the top-level
technical
content and
design documentation.
o.
Updates since
the last review to
software related CDRL items.
P.
Review considerations
applicable
all
previously
delivered
to :c.2.l as appropriate.
ent (SE):
40.2.3 .—
‘Sutmort-Eauismo
.
0
10
I
I
I-_
40.2.1
a.
Review considerations .app.licable to
40.2.2 as appropriate.
b.
‘Ierify testability
analysis
resul:s.
For example,
circuit
boards
are test po in~~
repairable
integrated
isolated
to the lowest
available so that failures can be
level of
repair (See Section 3 Definitions, for ‘Levels of
repair-) .
c.
Verify that the Government furnished SE
used to the maximum extent possible.
d.
Review progress
of long-lead
t ime SE
items,
identified
through
interim release and SE Requ’ rements Document (SERD)
procedures.
e.
Review progress toward tietermining total SE requirements
installation, checkout, and test support requirements.
f.
Review
reliability/maintainabi lity/availability
the
support equipment items.
9.
Id&ntify logistic support requirements for support equipment
items and rationale for their selection.
h.
Review calibration
requirements.
i.
Describe technical
equipment.
manuals and data availability
j.
Verify compatibility of proposed support ecpipmen: ..-’i:> the
syszam maintenance concept.
k.
Analysis
(LSA) is not done , then
If a Logistic SUppOrt
review
results 0: SE trade-off
for each
the
studies
alternative support concept.
For existing
SE and printed
circuit board testers, review Maintainability data resulting
Review
the cost
frmn the field use of these equipments.
difference
between
systems using single or multipurpose SE
Vs . proposed new SE.
Examine
feasibility
in
technical
37
pacagriaph
is
planned
to
and
be
for
of
for support
MI L-STn-1521B
APPSNDIX D
using
●zisting, developmental,
and proposed new SE. For
mobile systems, review the mobility requirements of scpport
equipment.
1.
Review the relationship
of the computer
the
resources
in
those
in Automatic
Test Equipment
(ATE), Relate this to the development
In Test
of Built
Equipment
(BITE) and try to reduce the need for complex
supporting SE.
system/subsystem
40.3 Evaluation ~
40.3.1 HUCIS.
evaluat~
with
Electrical,
MechanicalL
The material of paragraph
@
Loqical Desiqns
40.2.1
above
shall
be
a.
Determine that the preliminary detail
design
provides
the
capability
of satisfying
the performance characteristics
paragraph of the HWCI Development specifications.
b.
of
H’UCI
Establish
compatibility
the
operating
with overall system design
characteristics
in each mode
is
multi-mode
requirements
if the HWCI
involved
in
functions.
c.
physical
Establish
and nature
of
and
the ●xistence
between
functional
interfaces
t!teRUCI and other items of.0
equipment, computer s{ ftwar&, and facilities.
40.3.2 CSCIS.
evaluat~
The material of paragraph
40.2.2
above
shall
be
a.
Determine whether all interfaces between the CSC I and all
other configuration
terns both
internal and exrerna; co the
Requirements
system meet the requi ements of the Software
Specification and 5nterface Requirements Specification(s).
b.
all the
Determine whether the top-level
design embodies
and Interface
Requirements
requirements
of the Software
Requirements Specifications.
c.
Determine whether the approved design methodology
used for the top-level design.
d.
Engineering
Determine whether the appropriate Human Factors
(HFE) principals have been incorporated in the design.
●.
Determine vhether timing and sizing
constraints
met throughout the :op-level design.
f.
Determine whether logic affecting system and nuclear
has been incorporated in the design.
has
have
been
been
safety
..
)41L-STD-1521B
APPENDIX D
Review HUCI design
40.4 Electromagnetic
Comnatibiliiy.
for
Wl:.n
compatibility/electromagnetic
compliance
electromagneclc
requirements.
Use
interference
(mc/!a41 )
Electromagnet ic
Plan
as
the basis
for this review.
Check
Compatibility
MIL-Specs
MIL-STDS
and
cited
by
application
of
the
system/equipment
specification
to the iiWCI/Subsystem design.
adequacy
to confirm
Raview preliminary ZMI test plans to assess
that E14C requirements have been met.
40.5 Design Reliability.
quantitative
requirements
Identify
the
reliability
40.5.1
specified
in the hardware Development and Software Flequirements
Specification(s) , including design allocations, and the complexity
of the CSCIS.
and
failure
rate sources,
derating
policies,
40.5.2 Review
Review the reliability mathematical models
prediction
methods.
and block diagrams as appropriate.
40.5.3 Describe planned
specified requirements.
o
actions when
predictions
are
less
than
which
have a
40.5.4 Identify and review parts or components
life or require special consideration, and general plan
critical
act ions
for handling.
Agencies so affected shall present planned
CO deal with these components or parts.
Evaluate
40.5.5 Identify applications of redundant HWCI elements.
basis for their use and provisions fbr “on-line” switching of
the
the redundant element.
40.5.6 Rev iev critical
signal paths
to
determine
fail-safe/fail-soft design has been provided.
chat
a
for HUCIS between
functional
40.5.7 Review margins
of safety
requirements
and design provisions for el~m~nts, such as: power
motors,
hydraulic
modules,
pumps .
supplies , transmitter
review
structural .eleinents; i.e. , antenna pedestals,
Similarly,
of safety
dish-s, and radomes to determine that adequate margrns
between operational
stresses
and design
:be
provided
shall
strengzhs.
I
:0 insure
40.5.S Review Reliability Design Guidelines fcr HWCIS
concepts shall be availabia and tised by
reliability
that design
Reliability Design Guidelines shall include,
equipment designers.
as a minimum,
par: application guidelines !elec:rical dera:ing,
thermal derating, part parameter tolerances), part seiec:ion order
reliability
prohibited
parts/mater:als,
of
preference.
procedures
to ensure
apportionments/preaict ions,
and management
compliance with the guidelines.
39
141L-STD-1521B
APPEUDIX D
for fItiCIs preliminary
40.5.9
Review
reliability
demonstration
plan:
failure
counting
ground
rules,
accept-reject criteria,
number of test articles, test location
and environment,
planned
starting date, and test duration.
40.5.10 Review ●lements of reliability program plan
to determine
cowa?d achieving specified
that each task has bee n initiated
requirements.
40.5.11 Review s~contractor/supplier
reliability
controls.
40.6 Desiqn Maintainability
40.6.1
Identify the quantitative
maintainability
requirements
specified
in the hardware Development and software Requirements
preliminary
predictions
Specifications; if applicable,
compare
with specified requirements.
40.6.2 Review HWCI preventive maintenance
frequencies, durations, and compatibility
schedules
in terms of
with system schedules.
40.6.3 Review repair rate sources and prediction
methods.
40.6.4 Review planned
predictions
act ions when
specified requirements will not be attained.
indicate
●
that
and
40.6.5 Review planned designs for accessibility, testability.
(including
provisions for
ease of maintenance
characteristics
recovery
from
automatic
operator-controlled
with specified
failure/malfunc~;ons)
to determine
consistency
requirements.
HWC I design
that parts,
40.6.6 Determine if planned
indicates
and components
assemblies,
will be so placed
that there is
irons, and other
sufficient space to use test probes, soldering
and that they are placed
so -that
tools without
difficulty
structural mambers of units do not prevent access to them or their
ease of removal.
40.6.7 RevieG provisions for diagnosing cause(s) of failure: means
for localizing source to lowest replaceable element: adequacy and
system diagnostics
locations of planned test points: and planned
for isolating
faults
that provide
a means
to and within the
item
encompass
configuration
shall
on-line
off.lin~
d~~~~os~~~l~wand proposed technical orders
diagnostics,
and/or commercial manuals.
40.6.8 Review for HWCIS the Design for Maintainability
Checklist
to insure that listed design principles shall lead to a mature
design
Determine
maintainability
design.
that
contractor
●ngineers are using the checkiist.
40
●
I
MIL-sTD-1521B
APPENDIX D
●
-the
for !lWCIs
preliminary
Evaluate
maintainability
including
number
:hat
of maiczenance casks
demonstracicn
P an,
plans
for
shall be acccmpl sbet!; accep:-reject criteria; general
fau. ts into the _CI and pe:sonnel involved in the
introducing
demortstrat ion.
40.6.9
Rev i●w elements
of maintainability
40.6.10
program
plan
that each task
has been initiated towards achievi~~
determine
specified requirements.
40.6.11 Insure that consideration has been given to optimizing the
sy.steu+/item from a maintainability anti maixrexance vie-+oint and
within
rhe maintenance
concept
as
that
it is supportable
HWCIS
insure
ghat a Repair Level Analysis
Also,
for
developed.
(RLA) has been considered.
40.7 .—
?i~an
?UCtOrS
40.7.1 The contractor shall
allocation
the functional
operational and maintenance
In particular, ● nsure that
the functions: integrity of
a system operation.
evidence
that substantiates
present
The Review shall cover all
decisions.
item.
functions of the configuration
the approach to be followed emphasizes
the man with the machine to accomplish
40.7.2 Review desiqn tia:a, design
and drawings
on
descriptions
system operations. ●quipments, and facilities to insure that human
performance requirements of the hardware Development and SOftvare
9equirernencs Specifications
Zxamples of the types of
are me:.
desisn information to be reviewed are:
a.
Operatifig modes for each c?isplay szation, and for each mode,
the functions performed, the dis?lays and control used, etc.
b.
The exact format and content of eech display, including data
locations,
spaces, abbreviations, :he number of digits, all
(Pictographic) , alert mechanisms
(e.g..
special
symbols
flashing rates). ate.
c.
including
The control and data entry devices
an~
formats
keyboards, special function keys, cursor control, etc.
d.
The format of all opera:or in?uts, zo~ether vith
for error detection and correction.
e.
All
I
●
provisions
and dats printouts - includinq
formats,
status,
error,
headings, &ata units, abbreviations, spacings, coi.umns, ● tc.
in sufficient
?hese
*etail
allov
should
be presented
atiequacy fro;” a human
egency pe:sonnel
to judge
contracting
usability
standpoint,
●nd design
personnel
to know what
is
required, and test personnel to prepare tes:s.
MIL-sTD-1521B
APPENDIX D
●
or
recommendations
to update
the System/Segment,
40.7.3 Make
Interface
Requirements
Requirements
Specification
and
Software
Specification(s) in cases where requirements for human performance
need to be more detailed.
40.7.4
Review
man/machine
are utilized
capabilities
exceeded.
functions
and that
to
his
insure
ma~’s
that
limitations
are not
40.8 Svstem Safety
analyses,
40.8.1 Review resulzs of configuration item safety
quantitative hazard analyses (if applicable).
and
40.8.2 Review results of systen sncl intra-system safety interfaces
and trade-off studies affecting the configuration item.
40.8.3 Review safety requirements
levied on subcontractors.
areas” of safety,
peculiar
to the
40.8.4 Review known special
nature
of the system (e-q., fuel handlinq, fire pro~ection, high
high voltage
protection.
safety
levels of radiated
● nergy.
interlocks, etc.).
Rev i● w
40.8.5
appropriate) .
results
of
preliminary
safety
40.8.6 Generally review adequacy and completeness
item from design safety viewpoint.
tests
(if
●
of configuration
available
40.8.7 Review compliance of commercially
configurateion
iterns or
configuration
item components
with system
safety
requiremen~s and identify modifications
to such
equipment,
if
required.
40.9 Natural Environment
40.9.1 Review contractor’s planned design approach toward meeting
conditions
ranges
for
climatic
(~~~~~ting
and non-operating
temperat.>~e, humidity,
chat are specified
in the Iiwc;
Development Specification.
40.9.2 Insure that the contractor clearly understands
the ●ffect
and
of,
interactions
between,
aerospace
the
cne natural
In cases where
the ● ffect and
environment: and HWCI design.
ln:eraczions
are not known or are ambiguous, insure that studies
are in progress or planned to make these determinations.
40.9.3 Current
environment
and forecast
natural
aerospace
parameters
may be needed for certain configuration i:ems; e.g. ,
display o f airbase conditions in a command
and concrol
system,
etc.
calculation
of
impact
point
:or a missile.
Insure
●
‘141L-Sl!D-1521B
APPENDIX D
lo
I
I
I
I
(
between
the
config=razion
i:em
compatibility
design
and
meteorological
communications
comparing
appropriate
by
link)
characteristics of the source [teletype, facsimiie, or data
with
thet of the configuration item.
Insure that arrangements or
and
that
plans to obtain
needed
information> have been made
environmental information shall be
adequate
clisplay of
natural
provided.
and Part Standardization
40.10 Euui pment .—
40.10.1 —.
Eouinment
and Components:
a.
Reviev current and planned contractor actions
to determine
that
equipment
or components
for which
standards
or
specifications
be used whenever
practi=al.
exist shall
(Standard item with NSN should have first preference).
b.
Review specific trade-offs
or modifications
that may be
required
of existing designs if existing items are, or will
be, incorporated in the fiUCI.
c.
Existing designs will be reviewed for use or
on the potential
impact m
the overall
following areas:
(1)
Performartce
(2)
cost
non-use
program
based
n the
(3) Time
(4) Weight
(5) Size
(6) Reliability
(7) Maintainability
(8) Supportability
(9) Producibility
d.
Reviav HWCI design
to identify
areas .-here a practical
would materially
of
design
change
increase the number
standard items that could be incorporated.
e.
Insure that C~i:ical Item Specifications shall
for hardware
it ems identified as engineering
critical.
43
be prepared
or log:stics
MIL-sTD-1521B
APPE.NDIX D
40.10.2
a.
Parts
Standardization
and Interchanqeabili:y:
—
Review procedures to determine if maximum practical use will
made
of
parts
be
built
to approved
standards
or
The potential impact on the overall program
specifications.
is to be evaluated when a part built :0 approved standards
and specifications cannot be used for any of the following
reasons:
(1) Performance
(2) Weight
(3) Size
(4) Reliability/Mantainabi lity/Availability
(5) Supportability
(6) Survivability
b.
c.
(including nuclear)
Identify potential design changes that will permit a greater
th
parts and evaluate
use
of standard
or preferred
trade-offs.
@
Insure undarstantiing of parts control
program operations for
in new dasign or major
of parLs
and approval
selection
modifications.
d.
Review status of the Program Parts Selection List.
e.
Review status o: ail zoz-standa:d. par:s identified.
:.
Review pending par:s control actions :hac may cause program
slippages, such as non-availability of tested parts.
40.10.3 Assignment —
of
Official Nomenclature:
assignment
a.
Insure understanding of procedure for obtaining
of nomenclature and approvai of nameplates.
b.
Determine that a nomenclature conference has been held and
agreement
has been reached .zith the contracting agency on
i.e., system,
set. central,
the level of nomenclature:
group, componen:, sub-assembly, unit. etc.
40.11 Value Enaineerinq
the
40.11.1
Review
Program,
Engineering
foilowing:
Value
in-house
incentive
which may include but not be limited to th
m
Can:ractor’s
MI L-STD-15Z1S
APPENDIX C
a,
Contractor’s
procedures.
Value Er.q aeeriag
b.
Contractor’s
‘Jalue
c.
?otential
d.
Schedule of.pl-anned Value Engineering
e.
Policies and procedures
Programs.
Erig
Organization,
neering Training
Value Engineering
projec:s,
poiicies
arid
Program.
s:udies and VZC?S.
zasks/evenzs.
for subcan:rac:or
‘Jalue 2n5ineering
40.~2 T~an~poy~abil~~v
●
40.12.1
Review
HwCI
:C determine
if
design meets
contrac:s
requirements
governinq
to permit
size and weigh:
● conomical
handling, loadino. securino.
trans=ortino.
and disassenblv
for
within ‘“existing-capabili~ies o~”military and comm~rcial
shipment
and overweight
items.
carriers.
Identify potential
outsized
system/items
be
ing
defined
as
hazardous.
Ensure
Identify
with
hazardous
items complies
hezardous
packaging
afforded
materials regulations.
)
.’
40.12.2 Identify HWCIS requiring special temperature ●nd
humidity
sensitive and shock susceptibility
control
or those possessing
characteristics.
Determine
zransportat ion requirements
special
and availability for use with these HWCIS.
40.12.3
Review Transportability
Analysis
to
determine
t’ha
t
have been evaluated
transportation
conditions
and thar these
of .crotecti,)e, ski;picq,
condiiicns are reflected in the design
devices.
handling
In
and
addition
size and weight
characteristics, determine that analysis incl~~es provisiaris for
and humidity
temperature
controls, rninitnizationof sensitivity,
susceptibility to shock, and transit damage.
40.13 Test
40.13.1
Review all
changes
to
the
System/Segment,
Hwc I
Development , Software
Requirements,
an~
:n:erface Re.quiremenzs
Specifications s.~bsequent to the established ALiocate~ ‘Jaseline LO
whether
See: ion 4.0 of
determine
ali these specif:ca:ions
.
adequately reflects these changes.
40.13.2 Review
information
to be provided
by the cort:ractor
regarding test concepts for Developmer.t Test an~ Evaluation (DTLE)
testing (both informal and formal) . Information shall include:
●
“.
a.
The organization and responsibilities
be responsible for test.
of the group
that
wiil
MI L-STD-1521B
APPENDIX D
b.
The manege%e~t
provides for:
of
his
in-house
●
development
test
(1)
Test Mezhods
(2)
Test Reports
(3)
Resolution
(4)
Retest procedure
(5)
Change cont:ol and configuration
(6)
Identification of any special test tools that
are not deliverable under the contract.
effort
(plans/?rocedures)
of problems
and erro:s
management
c.
assurance
methodology
to be used to meet quali:y
The
requirements, includinq the test
requirements/qual ification
to regression
repeatability charac:zristics
and approach
testing.
d.
The progress/status of the test effort
reporting milestone.
since
the
previous
such
40.13.3 Review status of all negative or provisional ●ntries
‘not
‘to
be
determined
(TED)”
in
Section
as
applicable (tJ/A)” or
Development,
Software
hardware
4.0 of
the System/Segment,
Review all
Requirements or Interface Requirements Specifications.
that associated
positive entries for technical adequacy.
Insure
test documentation includes zhese changes.
●
in Section
specified
40.13.4 Review interface :est requirements
Requirements,
and
Software
the hardware
Development,
4.0 of
Interface Requirements Specifications for compatibility, currency,
Insure that
of redundant test.
adequacy,
elimination
technical
these
docunents
reflect
interface
all
test
associated
requirements.
has been
40.13.5 Insure that all test planning
documentation
updated
to incluUe nev test s.apport requireciencs and provisions
for long-lead time support requirements.
Rev i●w contractor
40,13.6
test Claza from prior
zestinq
to
determine if such daza negates the need for additional testing.
mock-ups,
or devices
40.13.7 Examine all availabie breadboards,
which
will be us ●d
the test program or which
in implementing
affect the test program. for program impact.
40.13,8 Review plans far software
they:
Unit
testing
to
● nsure
chat
o
46
tiIL-STD-1521B
APPSNDIX D
level
sizing,
a.
Address
Unit
requirements.
b.
anti specific
?resent general
demonstrated by Unit :estin~.
c.
‘Describe
hardware,
items.
d.
Describe how, when, and from .vheee the
items will be obtained.
e.
Provide
and
timing,
accuracy
that
reqcireaents
test scheduies
consistent
test-unique
ensure
:0
test n9
Define the type of testinq required for each
software struc:ure above the unit level
level
b.
and specific
requirements
Pres6nt general
demonstrated by CSC integration testing..
that
Describe how, when, and, from vhere
items will be obtained.
e.
Oescribe CSC integration
(1)
of
the
will
test-unique
the
test management,
Organi: ation and responsibilities
(2) Control procedures
.
..
that
be
test-unique
software,
required
support
the
and facilities.and the interrelationship of these
d.
I
support
with higher level pians.
a,
Describe
hardware,
items.
be
test-unique
sof:.aare,
sup~o rt
the
required
and facilities and the inzerrelatlonshi p of these
40.13.9 Review plans for CSC integration
they:
‘c.
wiii
to include:
of the test team
to be applied during rest
(3)
Test reporting
(4)
Review of CSC integration
(5)
Generation of data to be used in CSC
integration testing.
?rov itie:est schedules
support
test results
consistent
“<ith higher ievel pians.
40.13.10 Review plans for formal CSCI testing to ensure tha: they:
a.
Define tne objective of ● ach CSCI test, and relate Ehe
to the soft.aare requirements being tested.
47
test
I
..
MIL-sTD-1521B
APPENDIX D
b.
Relate
c.
Describe support software. nardware, and facilities required
for CSCI ces:ins; and ho.~, ./hen, ar.d from where they will be
abtainet?.
d.
Describe CSCI test roles and :espcnsibilities.
e.
Describe
hardware,
f.
Provide CSCZ tes:
pla,ns.
9.
Identify sof:.<are requiremer.:s that will be ver fied by each
formal CSC: test.
forma! CSCI tests to other test phases.
requirements
for Government-provided
facilities, data, and doc,umenzation.
40.14 Maintenance
schedules
and Maintenance
—
consistent
Data
——
w: th
software,
higher-level
(HWCIS)
40.14.1 Describe System Maintenance concept for impact on *esign
and SE. Review adequacy of maintenance plans,
Coverage shall be
off
provided for On Squipment (Organizational),
Equipment
- On
(Incefmediate),
off ZauiDment
Site
(DeDOt) level
- off Site
maintenance
of Government
‘Fu;nished
Equipment
(GbE) ,
and
Furnished
(CFE!.
Contractor
Squipme.nt
(See
Section
3,
Definitions, para 3.12 for levels of maintenance. )
●
40.14.2 Determine
degree
@f understantiirtg of
background,
the
requirements,
and
purpose,
usage of Maintenance (failure) Data
(Ref D?ta Item titled,
Collection and Historical/Status Records.
“?.eliability and l.?ainzainability Data Reporting
and Feedback
Failure Summary Report’s”).
40.:4,3
ne:ho~
of ?r~vic?i~g
Maintenance,
Oescr:be
Reliability, !.taincainabiiityData co contracting agency.
?ai lure,
40.14.4 Describe how requirements are submitted to the conz:ac:ing
for =uui~ment Classification lSQ/CL) Codes (formerly Work
agency
Order Number Pref~x/Suffix Codes) when this requirement exis:s.
40.14.5 Review plans for (and stattis of) Work Unit Coding of the
Xcrk Unit codes shall be available for docume.n:i.-.g
equipment .
.#ith configuration
Maintenance
>a:a commencing
i:en/Subsysten
[Ref.
Testing.
Data Item ti:ied
“Technical
Orders” acd the
military specification on wo:’k ur.i: coding).
40.15 ——
Soa?es and Go.lernment
Furnished ?roperty -(GF?)
—
40.15.1 Revie.d lcgis:ics and provisioning planning zc insure :’JIL
understanding
of scope of requirements in tt.ese areas and that a
reasonable time-phased p!an has been deveioped for accomplishment.
Of specific c:rcern are the areas of: provisionifiq requirements,
48
●
MIL-sTD-1521B
APPfzNDIX D
,0
GFP usage. and spare
checkout. and test.
parts.
and
support
during
installation,
40.15.2
Review
provisioning
and
identify
actions
●xisting or
potential
provisioning problems - logistic critical and long-lead
time items are identified and evaluated against use of the interim
release requirements.
40.15.3 Review plans for maximum screening
extent plans have been implemented.
and usage of
G?? ,
and acquiring
40.15.4 Xeviev progress toward determining
installation, checkout, ..andtest support requirements.
40.16 Packagina/SDPE
(SDecial Desiqn Protective
and
total
Scuioment)
40.16.1 Anelyze all available specifications (System/Segment, HWCI
Development:, Soft.Jare Requirem.?nrs, Interface Fle.quirements,a~d
for each
Critical Items) for packaging (Sec%ion 5) requirements
product fabrication and material specification.
I
user/operat ional
support
requirements
and
40.16.2
Evaluate
maintenance concepts for effect and influence on package
design.
I
●
design
time phased
plan
for package
that
40.16.3 Establish
development is in consonance =ith the development of the equipment
design.
equipment
designs
for
40.16.~fReview planned and/or preliminary
packaging and simplicity of package design, and identify
would materially
decrease
areas where a practical design change
cost, weight, or volume of packaging required.
● ase
for Si)?z necessary
:0 effectively
40.16.5 Reviev requirements
transportation, handling and
configuration
item during
support
Insure SDPE is categorized as a configuration
storage processes.
item utilizing specifications conforming to the types and forms as
Review
prescribed
in the contract.
SD?E development/product
specifications for adequacy of performance/interface
requirements.
design
baselines:
initial package
concepts,
40.16.6 De:ennine
constraints, -etc.# to
the txtent Possible at this
parameters,
phase of the configuration item development process.
packaga
design
$0.16.7 :nsure previously cleveiooed and approved
data for like or similar cDnfiguratian items ;s bein~ utilized.
she most
40.16.8 Establish plans for trade studies to Cetermine
packaging
design
approach
needed to
and ( esirable
economical
satisfy the funct onal performance and iogistic requirements.
49
MIL-STD-1S21B
APPENDIX D
●
40.16.9 Verify the adequacy of the prototype package design.
full
40.16.10
Review Section
5 of
Specification
to insure
Identify
understanding by contractor for contractor requirements.
package s?ecificacion used for hazardous materials.
40.17 Technical
—
Manuals
—
Plan”
40.17.1 Review status of the ‘Technical $lanual Publications
to insure :hat all aspects of the plan have been considered to the
extent that all concerned agencies are apprised of the technical
coverage
to be obtained
manual
procurement.
The
under this
and/07
modifications
suitability of available commercial manuals
there:o shall also be determined.
Review
of
technical
manuals
for
40.17.2
the availability
validation/verification during the latter phases of DTCE testing.
40.17.3 If a Guidance Conference was not accomplished or if open
items resulted
from it. then review as applicable provisions for
validation,
verification,
revjews,
accomplishing TO in-process
prepublication. and poscpublicazion reviews.
40;18 System Allocation Document
40.18.1
the
System
Allocation
Document
Draft
and technical adequacy to extent completed.
Reviev
completeness
40.18.2 The
information:
;:
;:
e.
f.
9.
h.
i.
j.
k.
1.
format
shall
provide
the
following
fo
*
minimum
Drawing Number
Issue
Number of Sheets
Location
Configuration Item Number
Title
Part Number
Serial Number
Specification Number
Equipment Nomenclature
Configuration Item Quantity
ASSeMbly Drawing
40.19 Desion Producibility —
and Manufacturicq
40.19.1 The contractor shall demonstrate and present ●violence that
will
be
in:egraced
iato the design
manufacturing
engineering
process.
.
.
50
I
kiIL-sTD-1521B
APPENDIX ‘D
,0
II
a.
The contracto:
●v dence
of
shall provide
perfoming
analyses
oroauci~llity
on development hartlvare trading of:
i:.
g
cost ,
requirements
risk,
aga i=s: mar..cfact.cr
desi5s
production, vol.ae, and existing capability/availa5il i:y.
in zhe contractor” s own
Evidence of such analyses may be
but
must
fornat
conclusively
demonstrate
that in-depth
by
analyses
.+ere
perfo-ed
qualified
organizations/ir.div idu.?is and the results of those analyses
will be incorporated in the design.
●
.b.
●ngineering
an~
?reli~inary
manufacturing
production
shall
material
and
n:annlng
demonstrations
address:
component
preliminary
prociuczion
sequencing,
selection,
methods
and flow concepts,
new processes! manufacturing
risk, equipment and facility utilization for Intended rates
production
in-process and acceptance zest and
and volume,
(Efforts :0 maximize
inspection concepts.
productivity
in
the above areas should be demonstrated. )
c.
Management
systems
to be utilized
producibility
manufacturing
and
integrated throughout the FSD effort.
will
insure
considerations
that
are
40.19.2 The producibility and manufacturing concerns identified
the SRR and the SDR shall be updated and expanded to:
in
a.
evidence
PrOV ide
that
cczcerns
ideriti:ied
in
the
.Manufacturing Feasibility
Assessment
and the Production
Capability Estimate have been addressed and that resolutions
are planned or have been performed.
b.
recommendat ions
technology
Make
including manufacturing
and provide a schedule of necessary actions cc tl?e
efforts
concerns
and
program office to resolve open manufacturing
reduce manufacturing risk.
40.20 ——
Post Review Action
—
40.20.1 After completing the PDR, the contractor shall publish and
covies
of Review minuzes.
Tk,e contracting a~enc~
distribute
indic~ted
officially acknowledges
completion
of a PDR as
paragraph 4.2.4.
40.20.2 The accomplishment of zhe ?DR shail oe recor<ed
con firjuration item Development Xecar6 by :?.e zonzrac:or.
●
✼
51/52
O!l
1!
Ii
MIL-STD-1521B
APPSNDIX z
●
I
50.
I
1:
I
●
)
Critical
Desiqn Review
The CDR shall be conducted
on each configuration
50.1 General.
item prior to fabrication/production/coding
release to insure that
the detail design solutions, as reflected in the Draf: Hardware
Product
Specification,
Soft”*are Detailed Design Document (SDDD),
Data
Base Design
Doc”uent(s)
Interface
(DBDD(s)),
Design
Document(s)
and
(IDD(s)),
●ngineering
drawings
satisfy
r=?uir~ents
established by the hardware Development Specification
and Software Top Level Design Document (STLDD). CDR shall be held
Operator’s
Manual
after the Computer Software
(CSDM) , Software
Manual
User’s
(SUM), Cafuputer System Diagnostic Manual (CSDM),
Uanua 1
Software Programmer’s Manual (SPM), and Firmware
Support
For complex/large
(FSM ) have been
updated
or newly released.
configuration items the CDR may be con~ucted
on an incremental
basis.
i.e., progressive
reviews
are conducted versus a single
with ● ach
CDR. The overall technical program
risks associated
item shall also be reviewed on a technical (design
configuration
and manufacturing), cost and schedule
For software,
basis.
a
technical
understanding
shall. be reached on the validity and the
degree of completeness of the SDDD, IDD(s], D8DD(s),
STD.
CRISD,
SPU , and FSU, and preliminary versions of the CSOM, SUM, and CSDM.
items.
50.1.1
EuuiPment/?acilities
configuration
The
detail
design
Pr~
Specification,
by the hardware
dravings,a~che;;i~;~
mockups, e“tc., shall.be revie~ed against the
Ruc I Development
Specification
performance
requirements.
For
other than facilities, the result of a successful WR shall be the
design
for
establishment
of
the
baseline
detailed
fabrication/production
planning i.e., the contractor is permitted
to use the detail design as presented at CDR and reflected in the
hardware Product Specification for planning for production and, if
for
initial
specifically
authorized,
fabrication/production
efforts.
50.1.2 Corn uter Software configuration items (CSCIS). The CDR for
ew of tfieCSCI detail
a CSCI ~besha
a formal technlca=
design, including data base and i!tterfaces. The CDP. is
normally
accomplished for the purpose of establishing integrity of computer
software design at the level of a Unit’s loa cal desicn .urior to
and ~esting,
coding
CDR may be accomplished at a ;ingie review
meeting
or
in increments
during
process
the
development
corresponding
co periods
at which
components
or groups
of
components reach the completion of logical design.
The primary
product of the CDii is a formal identification of specific software
documentation which will be released for coding and testing.
By
mutual
agreement
the contractor
between
znd the contracting
agency, CDRS may be scheduled concurrently for t“+o or more CSCIS.
●
\
,,
software
development
50.1.2.1 Since
computer
is an iterative
process,
the
completion
of a CDR for a CSC: is not necessarily
sufficient for maintaining adequate visibiii:y into the remaining
53
141L-STD-1521B
APPENDIX E
development
●ffort through testing.
50.1.2.2 Additional
which address:
In-?rcgress Re”:ie=s aay be scheduied
to outstanding
action
post-CDR
a.
Response
items
b.
Modifications to design necessitated
design/program errors
c.
Updating
d.
Updated design
e.
Results obtained during in-house testing, including problems
encountered and solutions implemented or proposed.
by
approved
ECPS
or
sizing and timing data
information,
as applicable
The
contractor
50.2 Items to be Revieweti.
shall
follo~-f=
=view
by the contracting agency:
present
the
50.2.1 ~
a.
reflected
Adequacy
of the detail
design
in the draft
ProduCt
Specification
in
the
hardware
satisfying
for
the
requirements of the HUCI Development Specification
item being reviewed.
b.
Detail engineering
diagrams.
c.
Adequacy
drawings
of the detailed design
Electrical
design
(2) Mechanical
design
(1)
for the HWCI including schematic
(3) Environmental
in the following areas:
control an~ therms: aspeccs
(4) Electromagnetic
compatibility
(5) Power generation anti grounding
(6) Elecr.rical and mechanical
interface compatibility
(7) Mass properties
(8) Reliability/Maintainabi lity/Availability
(9) System Safe:y Engineering
(10 ) Security
Engineering
54
●
MI L-STD-.1521B
APPENDIX E
(11)
Survivabi’..ty/Vulnerability
~
(12) Producibility
(including nuclear)
and Manufacturing
(13) Transportability,
Packaging
and handling
(14) Human Ezgifieering and aiomet!ical Requirements
Life Support and Crew Station Requirements)
(including
(15) Standardization
(16) DesiGn versus Logistics ?rade-o!fs
I
d.
Interface control drawings
●.
Mock-ups,
f.
Design analysis
~-
System
Allocation
Document
scheduled location.
h.
manufacturing
Initial Manufacturing
Readiness (for examDle,
development
demonstrateions,
and
tooling
proofing of new materials, processes, methods, tooling, test
reduction of manufacturing
risks to
equipment , procedures,
acceptable levels).
i.
Preliminary
j.
Life cycle costs
k.
Detail design information on ail
with the system.
1.
Verify corrosion prevention/controi considerations to insure
have been chosen
materials
that will be compatible with
operating environment.
m.
Findings/Status
breadboards,
and/or prototype
hardware
and test data
for
Huc 1
inclusion
at
each
●ngineering,
VECPS andlor formal VECPS
firmware
to
be
provided
of Quality Assurance Program
50.2.2 CSC:S.
a.
anti Interface
Software >etaiieti Designr Data 9ase Design,
Design
Document(s) . In cases where the CDR is =anducced in
increment
increments. complete documents
that
to support
shall
be available.
b.
an~lyses,
describing
:esul:s
of
Supporting documente:ion
testing,
● tc. , as mutually agreeti by the contracting agency
and the contractor.
55
●
HIL-STD-1521B
APPENDIX E
c.
Doc.ament
System
Allocation
scheduled location.
d.
Computer
Resources
e.
Software
Programmer’s
f.
Firmware
Support Manual
9.
Progress
on activities
h.
Updated
i.
Schedules
j.
Updates since the last review to
software related CDRL items.
50.2
CSC I
for
inclusion
at
each
Integrated Support Document.
Manual
required by CSCI PDR (para 40.2.2
operation and support documents
for remaining
.
(CSOM, SUM, CSDM
milestones.
all
previously
delivered
3 Suuuort Equipment ~
a.
aeview
b.
Verify maximum considerations
c.
Identify existing or potential
d.
Determine
and
qualitative
provisioning drawings and data
e.
Review
reliability of SE
‘.
.
Review
logistic su?port
9.
Review
calibration
h.
Review documentation
50.3 Detailed
Desiqns.
requirements
(paragraphs 50.2.1 and 50.2.2) for SE.
●
GFE SE
SE provisioning
quantitative
problams
adequacy
of
req’airements for SE items
requirements
Evaluation ~
for SE.
Electrical,
Mechanical,
—and
Loqical
and log ic
50.3.1 RWCIS.
Detailed
block diagrams,
schematics.
interface control drawings to
diagram~ll
be compared
.ith
and available
determine system compatibility.
Analytical
test
shall
data
be revieved
to insure the hardware
Development
Specification has been satisfied.
information
on firmware
50.3.1.1 The contractor
shall provide
which
is included in commercially available equipment or to be
included in equipment developed under the contract.
Firmware
in
this context
includes the microprocessor and associated sequence
the allocated
of micro-instructions nacessary to perform
tasks.
As a ❑inimum, the information presented during CDR shall provide
56
●
’
●
d41L-sTD-1521B
APPENDIX E
descriptions
1
I
●
J
and status
for the following:
a.
Detailed
logic flow diagrams
b.
Processing
c.
Circuit diagrams
d.
clock
and
t.iming
micro-instructions)
●✎
Memory (e.g., type (RAM, PROM),
spare capacity))
f.
Micro-instruction
list and fomat
9.
Device functional
of firmware.
instruction set obtained
h.
Input/outp~t data width
control. )
(i.e., number of bits for
i.
Self-test
within firmware.
j-
Support software
algorithms
‘data
(diagnostics)
(e;g.,
wore!
:,tim
ing
length,
~char.ts
for
size (total and
by implementation
data
and
for firmware development:
(1) Resident assembler
(2) Loader
(3) Debugging routines
(4) Executive
(monitor)
(5) Non-resident
I
diagnostics
(6) Cross assembler
computer
and
higher
level
language
on
host
(7) Instruction .aimulator
50.3.2 CSCIS. The contractor shall present
the
(inci:d~
rationale) of the CSCI :0 include:
a.
●✌✌
detailed
design
Lower-Level
The assignment of CSCI requirements to specific
Computer
Components
(LLcSCS) and Units,
the
Software
this
criteria
andan~esign
rules
used
to
accomplish
assignment,
the traceability of’ Unit and LLCSC designs
to satisfy CSCI requirements, with ●mphasis on the necessity
of the units for implementing TLCSC design
and sufficiency
requirements.
57
141L-STD-1521B
APPENDIX E
b.
Units,
The overali information flow between
software
by which each Unit
and
method(s)
gains control,
sequencing of Units relative to each other.
c.
The design details of the CSCI, TLCSCS,
LLCSCS , and Units
timing
including ciata definitions,
and sizing, data and
storage requirements and allocations.
d.
The detailed
design
characteristics
of all interfaces,
including their data saurce, tiestination,
interface
name and
the design
for
and, if applicable,
interrelationships:
The contractor shall also give an
direct
memory
access.
overview of the key design issues of the in:erfaca software
and indicate whether data flow formats are fixed or
design,
subject to extensive dynamic changes.
●.
Data be.se
The detailed characteristics of the data base.
structure and detailed design, including all files, records,
be
fields, and items. Access rules, how file sharing will
for data base recovery/regeneration
procedures
controlled,
from a system failure, rules for data base manipulation,
file
integrity,
rules for usage
rules
for maintaining
reporting, and rules governing the types and depth of access
Data management rules and algorithms for
shall be defined.
Details of the
implementing
them shall be described.
lanqu8ge
r-yired by the user to access the data base shall
also be described.
50.4 Electromagnetic
the
the
●
ComDatibiiitv:
a.
Determine
Rev iew contractor
EMC design
of all HWCIS.
requirements
of
with
the Electromagnetic
compliance
Compatibility Plan and h-JCI specifications.
b.
Re,v;ev system EMC including ● ffects on the electromagnetic
EHC )
and intra-system
*C.
● nvironment
(inter-system
toward
and ?rogress
Determine acceptability of EMC design
meeting contractual &UC requirements.
c.
Review SMC test ~lans. Determine adequacy
to con:inn
design charac:erlstics of the system/EWCI/subsystem.
EUC
50.5 Desiqn Xeliabilitv:
50.5.1 Review the mosi. recent predictions of hardwa:e and software
reliability and compare
against requirements specified in hardware
Development Specification and Software Requirements Specification.
predictions
are substantiated by review of parts
For hardware,
application stress data.
50.5.2 Review applications
minimum
life, or those
it ems wi:h
of parts or configuration
which require special consideration to
58
●
..
-MIL-STD-1521B
APPENDIX E
0
I
insure
their effect on system perf~.mance
is minimized.
S0.5.3
Review completeti Reliability
Design
!leviev Checklist
principles
have been satisfactorily
reflected
in
configuration item design.
insure
c~nfiguration
50.5.4
Review
applications
of redundant
t~~~
establish
components
co
expectations
elements
or
materialized since the PDR.
to
the
item
have
reliability
for
demonstration
plan
50.5.5 Reviev detailed WCI
with
requirements.
The number of
test
specified
compatibility
Loca: ions,
and
articles,
schedules.
tes.c conditions,
tesr
involved
are reviewe~v~~a;;sure a %utual understanding
personnel
information
to
to provide
.
planning
of the plan and
activities concerned.
50.5.6 Rev+ev the failure data reporting
for d?termlnation of failure trends.
procedures
and
met hods
circuit
50.5.7 Review the thermal altalysis of components, printed
in.
if these data
Determine
modules,
etc.
are used
cards,
performing the detailed reliability stress predictions.
m
off-line
diagnostic
50.5.8 Review on-line
diagnostic
programs,
and preliminary
technical orders
programs,
equipment,
support
with
system
manuals)
the
commercial
for compliance
(and/or
maintenance concept and specification requirements.
prediction
model
and
its
software
reliability
50.5.9
Review
based upon test data and refined predictions of component
updates
usage rates and complexity factors.
50.6 Design Maintainability
1
the alest rece>t predictions
of
quantitative
50.6.1
Review
maintainability
and compare these against :equiremcn:s s~ecified
Requirements
and Software
in the H’UCi Development Specification
Specification.
I
50.6.2 Review prev.entive ”aaintenance frequencies an? durations for
with overall
requirements
and pianning
syszem
compatibility
criteria.
req~::ed
for the
50.6.3 Identify unique maintenance
procedures
configuration item during operational use and evaluate their roral
effec:s on system maintenance concepzs.
:hac system
is
Asscre
from a maintenance
and main:ainabi~i:y vievuoin:
and
optimized
.....
731s shall Incluce
COnfOrMS vlLh the planned m81ntenanCe concept.
of prov-isions for au:omaric, sem~-automatic, and manual
a review
recovery from hardware/sofr’#are failures and malfunctions.
I
“.
53
I
FIIL-STD-1521B
App~Ix
~
I
50.6.4 Identify design-for-maintainabi lity criteria
provided
by
in the design detail to insure
that criteria have,
the checklist
in fact been incorporated.
50.6.5
placed
solder
are p
access
Detemine
if parts, assemblies,
and other
it ems are so
there
is sufficient
that
space to us ● test probes,
ng irons,
and other tools without difficulty and that
they
aced
so that structural
members of units do not prevent
to them or their ease of removal.
detailed
maintainability
demonstration
plan
50.6.6 Review
for
vith specified
test requirements.
compa t bility
Supplemental
to
inform t ion is provided
and reviewed
insure
mutual
overalla planning
understanding
of the plan and to provide
information to activities concerned.
50.7 ——
Human Paccars
50.7.1 Review detail design” presented
on drawings,
schematics,
hardware
mockups,
or actual
to determine
that it meets human
performance requirements of the HWC1 Development Specification and
Specification,
Requirements
Interface
Software
Requirements
Specification(s), and accepted human ●ngineering practices.
50.7.2 Demonstrate by checklist or other formal means the ●dequac s
of design for human performance.
50.7.3 Review ● ach facet of design for man/machine
Rev ie’+
considerations
time/cost/effectiveness
trade-offs of human engineering design.
S0.7.4 Evaluate
factors:
I
Operator
b.
Operator displays
c.
Maintenance
d.
Anthropometry
e.
Safety
:.
Wo:k space layout
9.
Internal
environmental
ventilation, ● tc. )
h.
Traini=g
i.
persanntl
I
I
I
the following human ●ngineering/biomedical
a.
I
compatibility.
and
forced
design
controls
features
features and emergency cqui?ment
cond
equipment
t
ions
(noise,
ighting,
●
accomunodatiomi
60
I
MIL+TD-1S21B
APPENDIX .S
I
50.8 System Safety
50.8.1 i?evie”~configu:a:ion
safety design requirements.
item detail design
:es: req.uirenents
50.8.2 Reviev acceptance
safety req,~iremeflcsare reflected therein.
_~aluare :adequacy :of detailed
50.0.3
protective equipment/c?evices.
50.8.4 Review Configuration
and procedures.
iiem
for
●
.2
design
ope=aciorfal
compliance
insure
adequate
for safety
maintenance
to
-arid
sa~ety
analyses
50.9 Natural Environment
50.9.1 Review detail design
requirements
environment
Specification.
to
determine
the
of
natural
that
“. meets
hard;;re
Development
concerning
have been accomplished
that studies
50.9.2 Insure
effects
of the natural environment on, or interactions with, the
at
NWCI . Studies which have been in progress sha-ll be complete
this time.
50.9.3 Determine whether arrangements have been made to obtain
●nvironment
natural
anti/or forecast
information. when
current
Assure conrpatibilicy of HUCI and source
needed for certain HWCIS.
of information by ccnparing electrical charac:eriscics and formats
for the source and the NUCI.
5C.1O Squiumer.: ——
a~d Parts Standar5izazion
Determine znat every reasonable
50.10.1 z ui ment and Com onencs.
action Fi%%7TtZnto,u ~
the standardization req,tire;;;;;
for use of standard items (standard item with NS)J should be
app~oval
for use of non-standard or
and to obtain
preference)
non-preferred items. Accordingly, the following criteria shall be
evaluated:
:ha: were reviewed.
a.
Data sources
b.
?ac:ors :nat were considered
similar, exissing cesiqns.
c.
in decisions
considered
:0 acceat
Factors :ha:
were
.~eye
which
i>corpo:.a:ed,
and
exis:i~q
designs
trat!e-offs, if any, that had to be matie.
in the decis:on
:0 rejecz kao=n
any
t?&
50.10.2 Parts
a.
Determine
“-he:her there are any outstar,ding non-standard
or
MIL-sTD-1521B
APPSNDIX E
●
ncn-preferred
par:s
approval requests and action necessary
.
or ~isa;~r:...al
(Sta:us of parts control
for approval
program operations).
b.
Identify non-standar-a.Ron-Prefer:ed parts approvai
proble!ns
and sta:us of actions towazd resolving tha problems.
c.
Review potential fabrication/p?otiuction iine delays
due
co
non-availability
of standard
or preferred parts.
In such
cases, dete=ine
whether it is planned
to request
use of
which may
parts
be replaced
by standard
items during
Assure
subsequent support repair cycles.
that appropriate .
documentation
makes
note of these items and that standard
replacement items shall be provisioned for su>oort and used
for repair.
t.
max imum
practical
that
Require
certification
interchangeability
of
parts
exists among components,
assemblies,
Reservations
concerning
and
~cIs.
interchangeability are identified, particularly for hardness
critical items.
e
Sample preliminary drawings and cross check to insure that
parts
indicated on the draw ngs are compatible with th
Program Parts Selection List.
w
50. 0.3 Assignment ~
I
~
Official
rJornencature.
a.
nomenclature
Determine whether
official
and approval
r.ameplates have been obtained to extenz practical.
b.
Determine whether DD Form 61, Request for Nomenclature,
beefi pracessed to the agreed ievel of inden:ure.
~as
c.
Insure that approved nomenclature has been reflected
Development and Product Specifications.
the
d.
Identify problems
(DD-61S) together
the problems.
e.
Insure that a software invent~r-; nunberino s.~stem has
agreed to and implemented to th~ CSCi lev;i.”
in
of
with nomenclature
associated
requests
with status of actibns towards resolving
been
50.11 Vaiue E’naineerino (VS)
50.11.1 Review status of all VEC?S presented per the terms of
contract .
I
50.~1.2 Review
any new areas of
considered profitable to cha~lenge.
62
potenciai
V.alue
the
Engineering
●
WL-STD-1521B
APPENDIX E
50.lL.3 If required by contract (funded VE
acc~mpliskments
actual Va lae Engineering
program.
prog?am),
review
the
against the planned VE
50.i2 Trans=orcabilitv
50.~2.l Review tran~a~r~ability evaluations accomplished fcz those
over-weight, sensitive,
and/or
ide~tified
items
is outsized,
requiring special cemperatu:e and humidity controls.
50.12.2 Review actions taken ’as a “result of the above evaluation
and military
or commercial
insure
adequate
facilities
to
available
ta
sapport
e~~ipnent
are
system
transporting
requirements during Production and Deplo’ymenc Phases.
handling
50.12.3 Review design of spec ia 1 materials
when required, and accion taken to acquire equi?ment.
equipment,
of
for movement
50.12.4 Insure DOD Certificates of Essentiality
have been obtained for equipment exceeding limitations
equipment
of criteria established in contract requirements.
has been annotated
on
50.12.5 Insure transportability approval
design documents and shali remain as long as no design changas are
made that modify significant transportability parameters.
I
Identify
equipne~t
50.12.6
transpor:abi:ity of material
to
be
i> Military
test
loaded
A:rcrafr.
for
air
50.13 Test
50.13.1 Review updating changes co all specifications
subsequent
to the PDR, to determine whether Section 4.0 of the specifications
adequately reflects these changes.
Cesr documentation
for currency,
50.13.2 Review all available
with
adequacy,
and compatibility
Section 4.0 of all
technical
Specificazi on requirements.
model,
prototype, etc.,
on which
50.13.3 For any development
have been performed, exanine test ?esu;ts
for design
testing may
Requirements,
compliance with hardware Development, Soft=are
and
Interfac= Requirements Specification requi~e-ents.
Rev iew
quality
assurance
?ro.3isions/quaiif ication
50.13.4
requirements
in HWCI ?roduct, Software Requirements, or Interface
Specifications
for
technical
Requirements
completeness
and
Section
4.0 of these specifications shall include the
adequacy.
minimum requirements that che item, materiel, or process must meet
to be acceptable.
50.13.5 Review
all
test
documentation
63
requireti
:0
suppo rt
test
I
MiL-STD-i521B
APPENDIX E
I
4.0 of HWCI Product Spec fications
requirements
of Section
compatibility, technical adequacy, and completeness.
I
1’
50.13.6 Inspect any breadboards, DOCkUDS,
available for test-program implications.
or
Drototne
..
o
for
hardware
they are
50.13.7 Review Software Test
Descriptions
to e?sure
the Software
Test
Plan &nd thev thoroughly
vith
prerequisites-” to
identify necessary
and
enable
parameters
execution
of each planned
software test antimonitoring of test
results. As a minimum,
identify
test descriptions
shall
the
following for each test:
consis:enc
a.
and
Required preset hard.~aee and Soft.-are conditions
necessary input data, including the source for all data.
b.
Criteria
c.
Prerequisite conditions
test execution.
d.
Expected
for evaluating
the
-
test results.
to be established or
set
prior
to
or predicted test results.
50.14 Maintenance
and Maintenance
—
50.14.1 Review adequacy
Data
of maintenance
50.14.2 Review ststus of unresolved
data problems since the PDR.
plans.
maintenance
and
maintenance
with Data
50.14.3 aeview
Item
titled
status of compliance
“Reliability,
Maintainability Data Reporting and Feedback Failure
Summary Reports.”
50.15 Spare —.
Parts and Government
Furnished Prooerty -(GFP)
through normal
logistics
planning
50.15.1 Review
provisioning
(ACO )
channels
Administrative
Officer
and
Contracting
representative (Industrial Specialist) to insure its compatibility
(contant and time phasing) with contractual requirements (data and
SOW items). The end objective is to provision by a met hod which
shall
iasure system supportability
at operational date of the
first site. Also accomplish the folio-~ing:
a.
Insure contractor understanding of contractual requirements,
inciuding
time phasing, instructions from logistics support
interim release authority
and procedure,
agancies,
end
responsibility to daliver spare/repair parts by need date.
b.
actions,
that
scheduled
provisioning
such
tLS,
guidance
meetinss, interim release and screening, are being
accomplished .sdequazely and on time.
Determine
. 64
●
I
●
.,.
MI L-STD-1521B
APPENDIx E
~o
c.
I
1!
Identify existing or potentia:
problems.
5i).15.2 Jete:aine
quantitative
hrd qsa:itative
at!equacy
of
Verify that Loaistics Critical
Provisioning
dra-~i~gs and data.
procedwes
~tems are listed for-consideration anti t.ha
- t ade~ate
change information in provisioning
design
exist
for reflecting
documentation ant! Technical Orders.
I
have been determined
50.15.3 Insure support
:equirenen:s
for
checkout,
installation,
for approval
and tes:
by concracti>g
agency.
Insure screening has been accomplished and
resul :s
are
included into support requirements lists.
I
I
J
>ave
space requirements
50.15.4 t)etemine tha: adequa:e s:ora~e
been programmed for on-site handling of Installation and Checkout
for
(s&c), test support material, and a scheme has been developed
and joint
use of
insurance
(high cos=) or
‘down straamingm
catastrophic failure support items.
I
50.15.5 Assure that Acquisition
I
Method Coding
(AMC) is considered.
50.16 PackatJinq/SDPE
i,
!
1
I
50.16.1 Review proposed pac’kage design
to insure that adequate
protection
to the HUCI , and che media on wh ch che CSCI is
recorded,
is
provided
natural
against
and
induced
:0 which
the equipment
vi 1 be subjected
environments/hazards
with
and to
insure
throughout
its life cycle.
compliance
contractual requirements.
Such analysis shall include, but not be
limited to, the folloving:
a.
Methods of preservation
b.
Physical/inechanical/snock
protection
including
cushioning
shock mounting and isolation features, ioad factorsj
media,
support pa~s, cushioning &evices, blocking and bracing, etc.
c.
Mounting
d.
Interior and exterior
e.
Handling
materials
f.
Container marking.
9.
Consideration
commodities
1
l\
provisioning
e
-.
facilities
and sezuring/hold-down
container designs.
and
provisions
compat
handling system ($63L)
and
provisions
identification
~Y
with
.airz?aft
dangerous/hazardous
50.16 2 Review ~esign of SDPE HWCI co determine if a cacegor:~ I
The analysis of the proposed COnt8iner or
ner
is required.
handl ngt sh.l?plng C<.Jlvalent shall encam?ass as a ~iniinum:
conta
65
9
MI L-STD-1521B
APPENDIX S
I
I
a.
Location
and
provisions
b.
Vibration
- shock
isc:ation
features ,
the
based
on
pre-determined
fragility rating (or other constraint of the
item %0 be sni?peti.)
c.
Ser-~i:e items (indicators, relief .~a!ves, e:c.)
d.
Environmental
e.
External handling, stacking -and
stress ratings
f.
Dimensional
9.
Bill-of-material
h.
Marking ?rovisions
i,
(self-powered
or tractOr/trailer)
For wheeled
the
SDPE
OVera~~ length, width, and height with mounted item, turning
radius, mobility, number of axles, unit contact load, number
of tires, etc.
j.
Position and travel of adjustable wheels, titling, or
adjustments to facilitate loading.
type
of
inte:nal
●
amunting
or
actactitng
control features
and weitjht data
tie-dobm
provisions
with
(gross and net)
including the center-of-gravity
50.16.3 Review the results of trade studies, engineering
to substantiate
selected package/SDPE
design
choice of mace:ials, handling pr~visions, environmental
etc.
● tc.,
Insure
Chat packaqe/S3PE
design provides
50.16.4
‘balance between cost and desirec? performance.
location
othe m
analyses,
approach,
features,
reasonable
50.16.5 Review all preproduccion test results of the prototype
package
design
:0 insure that Lhe HWCI is afforded the proper
degree of protection.
!iWCI
of the
Product
50.16.6
Reviev
Section
5, Packaging,
accuracy
an~
technical
adequacy.
Specification for correct format,
procedures
to
assure
that
the
50.16.7 ?.ev~ew contractor
for Delivery
of the
5! ?repara:ion
requirements
of Section
approved .HWCI ?roauc: Speciflca:ion, ‘Jill be incorporated into the
package tiesign data for provisioned spares.
50.17.1 Review maintenance
PDR .
of the System Allocation
66
I*
I
Document since
●
1
MIL-STD-i521B
APPENDIX ‘E
p
item
Cocf guration
initiated
for
50.17.2
Insure ?lans are
that aa:y be necessary
due :0 at :ions occ”~r:~ng
re-allacacions
prior co, or duri?g, CDR.
I
and .Yaaufac:uring
50.16 Desiun ?rsducibiiizy —
I
s~o~a.l ~ev~e., :he scacus of ali producibility
effar:s for :0s: and s=hed’uie considerations.
(and
productivity)
manufac:urixj
to resolve
50.15.2 Review the status of efforts
concerns
identified
in-previous technical reviews /and their cost
and schedule impact to the production-program.
53.18.3 9evie.~ :he status of *enufacturi>q ?echnoloSy >rograms an~
other previously recommended actions to reduce cost. manufacturing
risk and industrial base concerns.
concerns
:hitt require
5D.lB.4
Identify open manufacturing
ninimize
risk
:0
the
production
direction/ef?or:
co
additional
program.
●✌✌
efforts.
50.1S.5 Review the status of manufact.~ring engineering
proofing
of new
and test equipment
demonstrations,
tool ing
processes,
methods,
az~
special
tooliz5/:est
equipmen:.
materials,
system
ctnci
50.la.6 Review the intended manufacturing
management
organization for the production program in order to show how their
efforts will ● ffect a smooth transition into production.
50.19 Post
Review —
Action
——
50.19.1 After completing the CDFI, :ne contractor shall publish and
The
contracting agency
Distribute
copies of Review mir.utes.
in
as indicated
completion
of a CDR
paragraph 4.2.4.
recorded
50.19.2 The accomplishment of the CDR shall be
configuration item 3evelopmer.t Record by :he contractor.
I
..
.67/’68
on
the
MIL-STD-1521B
APPENDIX F
0.
\
1
60.
Test Readiness
——
Reviev
—
shall “m a formal review
of
the
testing.
It
is
begin
formal
CSC:
contractor’s
available
and
CSC
are
conducted after software test procedures
is complete.
testing
The purpose of TRR is for the
integration
contracting agency to determine .vhether the contractor is in fact
ready
to begin CSCI testing. A technical understanding shail be
the
reached on the informal test resuits, and on the validity and
of the Computer System Operator’s Manual
degree
of completeness
Compueer
System
(SUM), and
User’s
Manual
Software
(Cstx.f),
Diagnostic Manual (CSDM).
60.1
The
(TRR) .
General.
TRR
readiness
to
The
60.2 Items to be reviewed.
follo~f=
Zview:
60.2.1
Requirements
Specl* Lcatlon
Requirements
Specification(s) .that have been
impact CSCI testing.
I
o
contractor
present
shall
the
‘n~rcha!~~=r~~es~h~e~~~~~~
approveti since
,
Top-Leve 1
to
the
Softvare
Any changes
Desiqn chanaes.
60.2.2
Detailed
Design
Document, Data Base
Software
Design
Document,
Design Document(s), or Interface Design Document(s) that have been
made since PDR and CDR, and which impact CSCI testing.
)
Test procedures ta be
60.2.4 Software test procedures.
lnc~uding
retest procedures
conducting
cscI— cesclng,
anomalies and corrections.
used
in
for test
Csc
60.2.5 ~
integration test cases. procedures, and results.
~nformal
integration
test cases=
~ures
used In ~ductLng
CSC integration tests and the test results.
development
test resources.
60.2.6 Software
Status of the
IGFS), test
hardware,=overnment
Furnished
Software
facility
including
personnel, and supporting test softvare and ‘materiaLs,
test tool qualification and review of the zracea~ility
software
between requirements and their associated tests.
60.2.7 Test limitations.
limitat=.
Identification
Summary
60.2.8 Software problems.
including
all known &lscrepancies
software.
‘ ●.
1.
I
60.2.? Schedules.
Schedules
of
all
soft-aare
status
of software problem
of the CSCi and test support
for remeinincj milestones.
69
test
MIL-STD-1521B
APPENDIX F
●
Updates
to all evolving
60.2.10
Documentation
Updates.
previously del].~ered CDRL Items (e.g,, CS31, SUM, CSDM).
an~
60.3 ——
Post Review Action
60.3.1 After completing
distribute
copies of
paragraph 4.2.4.
and
the TRR, the contractor shall publish
The
contracting
agency
Review Minutes.
in
completion
of a TRR as indicated
shall be recorded
60.3.2 The accomplishment of the TRR
configuration item De~elopment Record by ~he contractor.
on
the
.
●
70
MI L-STD-1521B
APPSNDIX G
70.
Furic:ioaai Con fi;urac ion katii~.
The o>:ecti”+e of :he Functional Configuration Audit
70.1 General.
be to verify
that the zcnfiguratioz item’s actual
(FCA ) sha~i
performance complies .Jith its !-.arclwareOevelopmens
or Software
ilequiremen:s and Interface Requirements .spezifications. ?est data
snail be :evie.ae~ :0 verify Lhaz the hardware or c~mpu:er software
by
ics functional/allocated configuration
performs
as required
Far ccmfi,g.urationitems ‘developed at Government
identification.
expense,
an FCA shall
be a prerequisite to acceptance of the
configuration item. For software, a technical understanding shall
on the validity and :!vide5ree S: completeness of the
be reached
Test
Conputer
aeporcs,
arxi as appropriate,
SoftvaFe
Systea
operator’s
Manual
(CSOH),
~oft”~are User’s Manual
(SUM). and
Compu:er Sys:em Diacjnostic Manual (CSDM).
70.1.1 The FCA for a compiex configuration item may be conducted
when so specified by zhe contracting
on a progressive
basis,
throughout
:he configuratim
item’s development
and
agency,
culminates
at the completion of the q)~alification testing o’f the
configuration item vi:h a review of all discrepancies at the final
FCA shall be conduczed on that configuration of the
The
FCA .
is
representative
configurateion isam which
(prototype
or
of the configuration to be released for production
preprbduction)
When
a prototype
or
of the operational inventory
quantities.
preproduction
article is not produced, the FCA shall be conducted
For cases where configuration item
on a first production article.
qualification
can only
be determined through integrated system
FCA’S for such confiauracion items wili no: be considered
testinac
.
complete until completion of ~uch integrated testing.
70.1.2 ~e:ar.?.ezda:i~ns of cor.figuration a:ev. acceptance
or
non-acceptance
:0 the local contract management agency are based
and
outlined
in
upon and governed
by procedures
requirements
subsequent paragraphs.
70.2 Contract Requirements
on the
70.2.1 The ‘schedules for the FCA shall be recorded
i:e~ &eveLoFmez:
configura:io~
re:ord
by :he contractor.
A
be
:onfigura:ion item canno:
axditet! .W:::qc$ut:he contracting
agency authentication
of the functional and allocaced baseiine.
prod~~t
In addition, tune contractor stiaii submit :F)e final Craft
for the confiqura:ion
i:ert :0 be audited to the
Specifica:ioc
contracting a~ency for revie,~ pri3r to KA.
●
✼
70.3 Contractor
!lesoor,sibi
ii:-:
70.3.1 Prior to
audited) , the
to the contract
addition ta the
(for configuration
items to be
the FCA date
contractor shall provide the following information
ng ●gency (this information shall be provided
in
general requirements of Section 4.):
71
●
MIL-STD-1521B
APPENDIX G
a.
Contractor rcpresentation
attendance) .
b.
Identification
●
(the Sest
manager
should
be
in
of items to be a,udite~:
(1)
Nomenclature
(2)
Specification
identification
(3)
Configuration
Item number
(4)
Current listing of all deviations/waivers
against
the
configuration item, ●ither requested of, or approved by
the contracting agency.
(5)
Status of Test Programs to-test configured
automatic test equipment (when applicable).
70.4 Procedures
number
izeros with
and Requirements
—
70.4.1 The contractor’s
test procedures
reviewed for compliance with specification
70.4.2 The following testing
the FCA team.
and results
requirements.
information shall
be
shall
be
available
for
procedures,
and
a.
descriptions,
Test plans, specifications,
reporzs for the configuration item.
b.
list of successfully
accomplished? functional
A COmplet!?
tests during which pre-acceptance data was recorded.
c.
A complete list of successful
test data aye not recorded.
d.
tests
A complete
list of
functional
required
by
specification
but not yet performed.
(To be performed
system or subsystem test).
e.
Preproduction
and production
functional
tests
●
if’ detailed
the
as a
tast results
70.4.3 Testing accomplished with the approved test procedures
and
validated
data
shall
be sufficient
to insure
(witnessed)
configuration item performance as set forth in the specification
Section
3 and meet the quality assurance provisionsiquali:icat ion
requirements containeti i? the specification Section 4.
70.4.4 For those performance parameters which caznot completely be
verified
during
testing,
adequate analysis or simulations shall
results
of
the
have been
accomplished.
analysis
or
The
will be sufficient
irem
simulations
insure configuration
to
performance as outiinad in the specification.
~~
●
uIL-STD-1521B
APPENDIX G
70.4.5 Test reports, procedures, and data used
by the
shall be made a matter of record in the FCA minutes.
FCA
:em
70.4.6
A list of the
contractor’s
docaunenca:ion
internal
idrawings)
of the configuration item shall be revieved to insure
of
configuration
that the contractor has documented the physical
the configuration item for wnich the test da:a are verified.
70.4,7 Drawings of HUCI parts which are to.be provisioned
should
be selectively
sampled
to assure
that test data ●ssential to
manufacturing are inclsaed on, or furnished with, the drawings.
70.4.0 Configuration items which fail to pass
quality
assurance
test provisions/qualificat ion requirements are to be analyzed as
Appropriate corrections shall be
to the cause of failure to pass.
configuration
is
made
before
a
item
subjected
to
a
requalification.
●
which
70.4.9
A checklist
identifies
shall
be
developed
documentation
and hardware and computer software to be available
and tasks to be accomplished at the FCA for the configuration
item. See Pre-FCA checksheet.
,,
1
7IJ.4.1O Retesr.s or additional tests shall be ?erformed
compliance with paragraph 70.4.3.
tO
assure
70.4.11 Acknowledge accomplishment of partial
completion
of
qualification
FCA
for
configuration
those
items whose
continqe3t upon completion of integrated systems testing.
70.4.12
apply:
1
?or CSCIS
the
following
additional
requirements
the
is
shall
a.
The contractor shall provide the FCA team with a briefing
audited and shall delineate the test
for
each CSCI being
the
results and findings for each CSCX.
As a minimum,
discussion
requirements that were not
shall
include CSCI
met, including a proposed solution to each item, an account
of the ECPS incorporated and tested as .~ell as proposed. and
a general
test effort
presentation
of the .entire C5CI
delineating problem areas us well as accomplishments.
I
b.
An audit of the formal
test plans/descriptions/procedures
shall be
made and compared against the official test data.
The results shall be checked for completeness and acsuracy.
Deficiencies
shali be documented and made a part of the FCA
minutes.
shall
be
Completion dates for all discrepancies
clearly established ant! documented.
‘o
c.
An audit of the Soft”aare Yes? Repor:s Shail be performed
tO
and completely
validate
:hat the reports
are accurate
describe .the CSCI tests.
.
73
..
MIL-STD-1521B
APPENDIX G
d.
Al: SC?S :tiaz have been approved shall be reviewed to ● nsure
:kst
they have beefi technizaiiy incorporated and verified.
●.
A1l ~gda:es
to previously
aelivered
documents
shall
be
revie.~ed to ensure accuracy and consistency throughout the
documentation set.
f.
aev iew rairiures shall be
Desiqn
Preliminary and Critical
to ensure that all findings F.ave been incorporated
examined
ancl completed.
9.
a~d
the :es:i?g
The
:equiTemen:s
interface
requirements shall be re~-ieved [or CSCIS.
h.
70.5
of
these
Review data ba6e characteristics,
storage allocation
data
and timing,
and sequencing characteristics for compliance
vith specifisd req-uiremen:s.
Post Audit —.
Actions
——
70.5.1 After completion of the !YA, the contractor
shall
publish
F~
minutes.
and disr.ribute copies of
The contracting agency
in
officially acknowledges completion of the FCA as in~ica:ed
paragraph 4.2.4.
70.5.2 The accomplishment of the FCA shall
be recorded
configuration item Development Record by thr contractor.
on
—
I
●
the
●
MIL-STD-1521B
APPCHDIX H
●
60.
Configuration
Audit (PCA)
.—
The Physical Configuration Audit (pCA) shall be the
80.1 General.
formal examlna:ion of the as-bui:t version of a configuration item
against its design documentation in order to establish the product
completion
of the audit,
successful
After
all
baseline.
action.
chan5e
subsequent changes are processed
by engineering
The PCA also determines that the acceptance testing requirements
for acceptance
of
prescribed by the “documentation is adequate
production
units of a configuration
item by quality assurance
The PCA includes a detailed
audit
of engineering
activities.
technical
da:a and ;ests utilized in
specifications,
drawings,
production of HW~~u~;~
a detailed audit of design
documentation,
listinas.
and
for CSCIS.
The review shall include an
enaineerina
documentation
and quality
audit ““o~ the released
suremake
records
to
the
as-built
or as-coded
control
For software,
configuration is reflected by this documentation.
the
software
Specification
and Version
Description
Product
Document shaAl be a part of the Pa review.
1.
I
I
I
Physical
\
sO.~_l The
PCA shall be conducted
of
an the first article
items and those that are a reprocurement of a
configurateion
identified
shall
be
configuration item already in the inventory
and selected jointly by the contracting agency and the contractor.
item to be
A PCA shall be conducted on the first configuration
● ven
delivered
bv a new contractor
thouah PCA was Previously
delive;ed
by a ‘differen~
accom~lished- on the first article
contractor.
i
approval
by the contracting
the
agency
of
80.1.2 Formal
spec, fication,
tem Product
and the satisfactory
configuration
in establishment
of the product
cora:letion of a PCA results
baseline.
ao.1.3 Recommendations
of confiaurat ion item
acceDcance
or
nonacceptance
to the responsible= contract adminstra;ion office
(CAO) are based upon and governed by procedures
and requirements
outlined in subsequent paragraphs.
80.1.4 A final review shall be mat2e of all operation
and support
(i.e.,
Compurer
System
Operatar’s
Manual
(CSOM),
documenzs
MaRual
System Diagnostic
Software User’s Manual (SUM), Computer
Manual
Sugport
Programmer’s
(SPM), Firmware
(C.sml) , Software
with
Manual (FSM) ) to check format: completeness, and conformance
applicable data item descriptions.
80.2 Contract Reouiremenzs
shall be recorded
on the
80.2.1 The schedules
for the PCA
item Development
Record
by the contractor.
configuration
A
current set of listings
shaLl
be
provided
being
for each CSCI
The contractor
shall s-it
audited.
the final draft of the
75
MIL-ST’D-1521B
APPSNDIX Ii
product spec fication for the configuration
the contract ng agency for review prior
80.3 Contractor
item to be aud ted
to PCA.
to
Responsibility
inforsiation to
S0.3.1 The contractor
shall Drovide the follovina
sh~ll be provided in
agency
(this information
the contracting
of Section
4 and the
accordance with the general inst:uctions
contractual requirements):
a.
Contractor representation
attendance). -
b.
Identification
the test
manager
should
be
in
of items to be accepted by:
(1) Nomenclature
(2) Specification
Identification
(3) Configuration
item Identifiers
Number
[4) Serial Numbers
(5) Drawing
and Part Numbers
(6) Identification
Numbers
(7) Code Identification
(0) Software
c.A
Numbers
inventory numbering
list delineating
item
configuration
approved.
system
a~ainst
the
all deviations/vaivers
either reques:ed or cor.tracting agency
to the
80.3:2 The PCA cannot be performed unless data pertinent
audited is provided to the ?CA team at
item being
conflquration
compi le and make
this
time of the audit. The contractor shall
available
Required information
for ready reference.
information
shall include:
●
.
Configuration
item product specification.
5.
A list delineating both approved
the configuration item.
against
c.
Complete shortage list.
d.
Acceptance
e.
Engineering
test procedures
ckawing
and
outstanding
and associated
test data.
index including r,evis.ionletters.
76
changes
o
1
lo
MIL-STD-15213
APPZf4DIX H
‘1 t
II
!
●
maintenance,
I??2 illustrated
parts
breakdow.
and
aeceiving
f.
Qperating,
manuals.
9.
Proposed
Report”.
DD F~rn 253,
h.
Approved
nomenclature
i,
User’s
Software
%anuals
(SF?4S),
Software
Programmer’s
Operators
Manual (CSOM),
Manuals
(SUMS) Computer System
(CSiiM), and F irmnarc
Computer
System
Diagnostic
Mazual
SuppOrt Manual (FSX).
].
Software
k.
FCa minutes
1.
Findingz/Status
*Material
Inspection
and nameplates.
Version Description
Document.
for each configuration
item.
of Quality Assurance
Programs.
80.3 3 The contractor shall assemble and make available to the PCA
team at time of audit all data describing the item configuration.
Item configuration data shall include:
a.
of
issue
hardware
development
Current
approved
Requirements
and
Specification,
speci’ficatlon,
Software
Interface Requirements Specification(s) to include approved
not ices
change
and
approved
specification
deviations/vaivers.
b.
Identification
of all changes actually made during test.
c.
Identification
of ali required changes not completed.
d.
and documents
by the top drawing
All approved drawings
item product
in i he configuration
number
as identified
specification.
A1l drawings shall be of the category
and
form specified in the contract.
● ✎
Manufacturing ins?r-~ction sheets
contracting agency.
for lWCi$ identified by the
between
zhe
80.3. 4 The contractor shall identify any difference
configurations
of the selected production unit and the
physical
Development
for the FCA and shall
Unit(s) used
certify
or
these differences
do not
that
demonstrate
to the Government
degrade the functional
characteristics of the selected units.
80.4 —
PCA Procedures —
and Reouireme>ts
●
.
..
80.4.1 DravinS
Instructions:
and
S4acc$acturinq
77
?nst:ucz ion
Sheet
Review
MIL-STD-1521B
APPENDIX H
a.
number
of
and
associated
A representative
drawings
instruction sheets for each item of hardware,
manufacturing
shall
Co-Chairperson,
identified by the contracting agency
be reviewed to de:enaine their accuracy and insure that they
include the authorized changes reflected in the engineering
Unless otherwise directed by the
dravings and tk.e hardware.
inspection
of drawings
contracting agency Co-Chairperson,
sheets may be
associateri
manufacturing
and
instruction
The purpose of this
accomplished on a valid sampling basis.
the
manufacturing
instruction sheets
insure
review
is to
all design
details
contained
in the
accurately reflect
Since
the hardvare is built in accordance vith
drawings.
any discrepancies
instruction
sheets,
manufacturing
the
sheets and the design details and
the instruction
between
in the
will also be reflected
changes in the drawings
hardware.
b.
The followiog minimum
&raving reviewed:
information shall be recorded for each
(1) Drawing number/title
(2) Date of drawing
(include revision
letter)
approval
(3) List Of manufacturing
change
Letterltitles
with-this draving.
●
(numbers with
instruction sheets
and date of approval) associated
(4) Discrepancies/comments
reflected
on the
numbers
a sample
of part
(5) Select
dravinq.
Check to irsure compatibility “~ith the Proqram
to insure
Parts Selection List, and examine
the HUCI
that the proper parts .are actually installed.
c.
shall
be
following
inspect ions
the
AS a minimum,
for each drawing and associated manufacturing
accomplished
instruction sheets:
(1) Drawing number identified on manufacturing
sheet should mazch latest released drawing.
instruction
instruction
(2) List of materials on manufacturing
should match materials identified on the drawing.
sheets
(3) All special instructions called on the draving should be
on the manufacturing instruction sheets.
out
(4) All dimensions, tolerances, finishes, ● tc., called
on the drawin~ should be identified on the mancfaczurin
instruction sheets.
a
78
KIL-STD-1521B
APPENDIX H
should
out
en
the drawing
(5) All special processes called
be dentified on the manufacturing instruction sheets.
numbers
(6) Nomenclature
descriptions.
Dart
and
serial
markings
number
caiied out
“ - on the drawing should be
sheets.
identified on the manufacturing instruction
(7) Review drawings and associated manufacturing instruction
to ascertain that all approved changes have been
sheets
incorporated into the configuration item.
I
(8” Check ‘release
identified.
(9
record
to insure all drawings
Record the number of any drawings
five outstanding changes attached
reviewed are
more
containing
to the drawing.
than
the
10) Check the drawings of a major assembly/black box of
from top
for continuity
hardware
item
configuration
drawing down to piece-part drawing.
●
\
)
for the
configuration
80. 4.: Review of all records of baseline
comparison with contractor’s enqineerinq release
bv direct
RUCI
;hat the
~ontrol
procedures
to ●stablish
change
system -and
reflect released
configuration
being produced
does accurately
interim
releases of spares
This
includes
engine-ring
data.
delivery
of currently
provisioned
prior
to PCA to ensure
configured spares.
release
and change
engineering
8G.4.3
Audit of contractor’s
that they are adequate to properly
system
to ascertain
control
changes.
control the processing and formal release of engineering
The minimum needs and capabilities set forth belov are reauired of
The contractor’s ”fonnats,
his engine~:~ng release ~ecords system.
Information in addition
procedures are to be used.
systems,
part of the
requirements
is to be considered
to the basic
contractor’s internal system.*
shall
be
80.4-3.1 As u alnirnmm, ‘the “Zollowfng information
by the contractor,
release
contained
on one
record supplied
subcontractor, or vendor for.each drawing .nuuber, if applicable:
●
a.
Serial numbers, top drawing number,
b.
Drawing
number”,
title,
specification
code number, number of
number;
she?ts,
date
Assurance
(CAo)
Contract
Administration
Office
Quality
Representat ive [QAR) records can be reviewed for purpose of
recent
and most
past
determining the contractor’s
present
performance.
79
,
MIL-STD-1521B
APPENDIX H
I
I
I
1
I
●
of release,
change
letter, date of change letter release,
●ngineering change order (ECO) number.
00.4.3.2 The contractor’s release function and documental
be capable of determining:
will
on
a.
in
erms
The composition of anv Dart
at any level
subordi~ate part nu.nbe~s ~disregard s~andard parts);
b.
The next higher assembly using the part number,
assembly into standard parts;
c.
number
item or part
The composition of the configuration
with respec: to other configuration items or part numbers.:
d.
The configuration itez and associated serial number on which
subordinate
parts are used.
to
(This does not apply
not
level who are
producing
contractors
below prime
configuration items); .
e.
The accountability of changes which ha-de been partially
completely released against th? configuration item;
f.
The configuration
item
.snd
serial number
except
of
effectively
of
for
or
any
change.
part
numbers
9.
The standard specification number or standard
used within any nonstandard part number;
h.
document
and specification
The contractor
specification
~ssociared with any subcontrac~or, vendor,
numbers
control
or supplier part number.
release
enginecrinq
BO.4.3.3
The
documentation shall be capable of:
system
and
●
associated
a.
records of superseded
Identifying changes
and retaining
configurations formally accepted by :he contracting agency;
b.
Identifying all engineering changes released for production
incorporation.
These changes shall be completely released
and
acceptance
of
incorporated
prior to formal
the
configuration item:
c.
Determining
configuration
released
the
configuration
for
item a: the time of formal acceptance.
each
80.4.3.4 Engineering da:a shall be released or processed through a
central
:0 ensure coordinated
acrion and preclude
authority
unilateral release of data.
80.4.3.5 Engineering
change control numbers shall be unique,
ao
●
MIL-STD-1521B
APPENDIX H
●
configuration
80.4.4 Difference betveen
the configuration of the
item qualified and the configuration item being audited shall be a
matter cf record iz the minutes of the ?CA.
80.4.5 For Xl
acceptance tests date and procedures shall
comply
The
PCA
team
shall
determine
any
with
its product specification.
reserves
and
the
tests
to
be
reaccomplished,
acceptance
representatives
of the contracting agency
prerogative
t~ have
witness all or any portion of the required audits, inspections, or
tests.
BO.4.6 WCIS which fail to Pass acceptance test
requirements,shall
be repaired if necessary and be retested by the contractor an the
with
the
manner specified by the PCA team leader
in accordance
product specification.
SC.4.7 The ~$tractor
shall present data confirming the inspection
iterns at point
of
end
subcontractor
equipment
nd test
witnessed
by Government
Such data shall have been
manufacture.
representative.
●
●
‘)
80.4.8 The PCA ~eam reviews the prepared back-up data (all initial
item) for
which
the configuration
accompanies
documentation
coverage
at the
correct types and quantities to ensure adequate
tiaw
of shipment to the user.
80.4.9 Configuration items which have demonstrated compliance with
che product specification are approved for acceptance as follows:
a.
The Pa
team shal 1 certify
item has been
configuration
drawings and specifications.
signature
that
the
by
buil: in accordance with the
.90.4.10 AS a minimum, the following actions shall be performed
the PCA team on each CSCI bein9 audited:
a.
.#hich will comprise
the
~ev iew all documents
Product Specification for format and completeness
b.
Review
FCA minutes
for
racorded
discrepancies
and
by
Software
actions
taken
e
symbols,
c.
!?eview the design descriptions for proper entries,
labels, taqs, references, and data ~escriPtlons.
d.
Csrapare Tap-Leve: Computer Soft”~are Cocponen: (7LCSC) design
.~itb, LoMe:-Level Compu:er Software Components
descriptions
(LLCSC) descriptions for consistency
e.
Com?a re
software
lover-level
design
c!escriptions
all
iistings for accuracy and completeness
with
a~~
I
MIL-STD-1521B
APPENDIX H
f.
I
I
,1
9.
h.
Programmer’s a
Check
Software
User’s Manual(s),
Software
Manual , Computer System Operator’s Manual, Firmware Support
Manual , and Computer System
for format
Diagnostic
Manual
with applicable
data
item
completeness
and conformance
verification/acceptance
of
descriptions.
IFormal
these
be withheld until system testing to ensure
manua 1s should
are correct)
that the procedural contents
tapes.
actual
(card decks,
CSCI delivery
media
to insure conformance with Section 5 of the Software
Requirements Specification.
Examine
●tc.,)
with approved
Review the annotated listings for compliance
coding standards (e.g. Appendix C of DOD-STD-2A67).
Post
80.5 ——
Audit —
Actions
rejection
of
the
agency acceptance
or
80.5.1 Contracting
product
configurateion
itern
and
the
item
configuration
the
for-PCA
must be furnished
to
specification
presented
by the responsible
contract management
contractor
in vritinq
agency or other designated agency after completion of PCA.
shall publish
80.5.2 After completion of the PCA, the contractor
The contracting agency
and distribute
copies of PCA minutes.
in
PCA as indicated
officially acknowledges completion of the
paragraph 4.2.4.
●
shall be recorded
80.5.3 The accomplishment o; the PCA
configuration item Development Record by the contractor.
on
the
.
—.
)(IL-STD-1521B
APPENDIX I
I
‘
a.
90,
PO ma 1 Qualification
Review.
\
I
I
I
I
1“
The
objective of the
FQR shal &
90.l”General.
to verify
that
the actual performance of :::D;:nfiguration
terns of the system as
with
Developraen:
determined throuah
test
the hardware
and
Reqbi~ements
nterface Requirements
Specification,
~oft~are
the test
identify
report(s)/data
Specifications, and to
which
results
of qualification
tests of the configuration
document
items. The point of Government certification will be determined
by the contracting agency and vill depend upon the neture of the
program, risk.aspects ’of the particular hard~are and software, and
contractor
progress in successfully verifying the requirements of
the configuration items. When feasible: the FQR shall be combined
with the FCA ● t the end of configuration item/subsystam testing,
If sufficient test
results are not available et the
prior to PCA.
Fm to insure the configuration
items will perform in their system
environment, the FOR shall be conducted (post PCA) during
system
whenever
the necessary
have
been successfully
testing
tests
comnleted
enable certification
of configuration
items.
—_. ~—.— to
-.
For
non-combined Fti/FQRs, traceability, correla~ion, and completeness
of the FQR shall be maintain-d with the F~
and duplication
of
● ffort avoided.
90.2 Requirements.
90.2.1 In cases where the FQR and the FCA can be accomplished in a
single
Audit/Review,
combined
contractor
and
Government
‘certificacion- of the configuration items shall
be accomplished
after completion
of the FCA and such certification shall be
considered as accomplishment of the FQR.
responsible
for qualification
of the
90.2.2 When
the agency
configuration
items at the contracting
agency judges that the
system is not ready for FQR.az the time of ?C.A, the FQR will
be
delayed
until it is determined that sufficient information on the
The FQR may be delayed up co
qualification is available.
system’s
tha end of Systam testing if deemed necessary.
90.2.3 When a senarate FOR
is necessarv,
the contractor
shal 1
notify
c~ntracti;g
the
aqency of ‘the sufficiency
of ,the
to substantiate
confi~uration
items test results
a FQR and
with the Deputy
Director
for Test and
coordinaca
the agenda
Deployment.
The FQR team will be assembled in the same manner
as
that required
for the FCA team. NO duplication of FCA effor:
shall occur at the FOR; however. the fallowing additior,al ef forts
must be accomplished:
90.2.3.1 A review af the FCA minutes must be per:ormed and the FQR
shall be considered
as an extension
of FCA . New/additional
reviewed
to insure
shall
be
audited
and
qualification
c!ata
the
configuration
items
against
the
qualification
of
System/Seqment, Soft=are Requirements, and Interface Requirements
a
,.
.83
)41L-STD-1521B
APPSNDIX
I
Specifics””
.
90.2.3.2
Any testing
accomplished
cgai~st
canfigu:a:ion
qualification during System testing
shall
be considered.
item
90.2.3.3 The contractor shall, after notification of certification
by the contracting agency enter :he date of system certification
0:
the
test
of
qualification
the
identity
and
reports/documentation
which
sets
forth
the results of the
associated test(s) in the confi~uration
i:em Development ?iecord.
90.2.4 All other factors
such as:
agenda,
team organization,
review
etc., shall
be
procedures,
data
to
be reviewed,
accomplished as delineated in the F= and General Requirements
and
Procedures
sections
of
this standard to the ● xtent necessary to
●ccomplish the FQR.
90.3 ——
Post Review Action
—
90.3.1 After th? conduct of the FOR.
the contractor shall c.ublish
-.
The contracting agen;y will
and distribute copies of FQR minutes.
in
officially acknowledge the conduct of the Review as indica?ed
paragraph 4.2.4.
1-
84
i
HIL-sTD-1521B
I
‘“
APPENDIX I
PRE-F13 CHECK SF?E.V
I
Nor.lENcQTLas
(
CONFIGURATION .lTES1NO.
i
I
a
1.
I
DATE
1.
Waiver/Deviation
List Pzepared
2.
Qualification
Test Procedures Submitted
—.
3.
Qualification
TestinS
—.
4.
Qualification
Available
Test Resuits Compiled 6
Completed
—.
——
5.
Facilities
6.
~ualification
and Approved
Test Procedures Reviewed
7.
Qualification
Tes:ing Witnessed
8.
Qualification Test Data end Resalts
Re.:iewed azc?A?pro-:eti
for Conducting
FCA AvallaBle
—.
——
I
I
——
——
COFWWTS
Figure 3
Page 1 of li
es
MIL-S’I’D-1S21B
MPENDIX
I
SAMPLE CERTIFICATION
FUN~IONAL
ATTAC!IF!ENT
CONFIGURATIONN A~
FOR
CONFIGUTUTION
ITEM NO. (sJ
CONTRACT
EQUIPUENT MANUFACTURERS:
PRIME CONTRACTOR:
●
APPROVED BY
APPROVED
[CONTRACTOR 1
DATE
DATE
Figure
Page
86
3
2 of
11
MIL-STD-1521B
APPENDIX I
●✍✎
DEF-INITIONS:
A note explaining, illustrating, ox Criticizing the
meaning of a writing.
I:ems of tnis nature snould be expLored
by the contractor and/or the Contracting Aqency, Dut corrective
action is NOT necessary to successfully accomplish a FCA.
COMUENT :
Deficiencies conszst of two types:
DEFICIEWY:
(1) conditions
of characteristics >n any nardware/software wnxcn are not in
compl~ance with specified configuration, or (’”2) inadeqtwste tor
erroneous] configuration, .i&entificatlon wnicn ‘has resulted. or
may result in configuration ~tema tnat do not fulfill xtpproved
operational requirements.
I
..
Figure 3
Page 3 of 11
87
~
HIL-STD-1521E
APPENDIX I
SCOPE/PUR~
Scone:
configuration it&m:
Configuration
Item No.
Audit
(FCA) was conducted on the follwing
Nomenclature
Part No.
Serial Mo.
PURPOSE : The purpose of this FCA was to vezify that the confiquraticn
Ltem’s performance complies with the Type B Development Specification.
I
I
Figure 3
Page 4 of 11
BE
*
141L-sTD-15219
APPENDIX I
●
)
~
(per
Sofware)
zquipmenc/CoEpu:er
Contract:
Date
Contractor:
Configuration
IEem No.:
—
I
CLialification Test Procedures and Resuizs. The quali~ication test/
analysls xesults have been re-:~cwed :0 ensure that tasting is adequat,
[All test procedures and interface
properly done. and certified.
documents shall be reviewed to assure that the documents have been
All test aaca sheets shall be
zPProved by the Contracting Agency.
reviewed to assure that the teat was witnes.qed by a representative of
the Contracting Ageacy. )
*
Attached
is a list of the documents
reviewed.
Check One
Pracec?tiresand results zeviewed satisfy the requirements and
zre accepted.
for comments.
See Attachment _
Attached
Signature(s)
is a list of deficiencies.
of FCA Team $lember(s)
●
Figure 3
Page 5 of 11
MIL-STD-1521B
APPENDIX I
SPECIFICATION/TESTIXG REVIEW
Configuration Item No.
Nomenclature
Specification No.
Test p=Oceti~res
Spec Ref
TP Ref
Description
Teat Result
●
!
Figure 3
Page 6 of 11
●
.. . .. . .. . . . . .
.. .
.. .
.
..
-.’.
.
.
.
.
......
. .
MIL-STD-1521B
APPENDIX I
●
—
.
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
.
I
—
—
—
—
—
Figure 3
Page 7 at U
.
.
.
I
.
.. . . ..
.
.
.
.
. .. . .. .. .
. ..
. ..
-.’-
I
MIZ-STD-1521B
APPENDIX I
I
1’
7LNCTIONAL
CCNFIG”MTION
.qLq&
~FRTIFICaTION SHEET No. 2
(For Equipment/Computer Software)
Contract: —
Date
Contractor:
Configuration
Item So.:
Review Deviations/iiaivers. A review of all deviations/waivers to
mli>tary Spec:f lcat~ons and standarZs that have been approved. The
purpose is to determine the extent to which the equipmentl sl[computer
Software undergoing FCA vary from applicable specifications and
standards and to form a basis for satisfactory compliance
with these
specifications and standards.
In accordance with this paragraph, all applicable
have been reviewed with the following resulta:
deviations/w.sivers
Check One
O
;~~e~q;~pment(s) /computer software listed. on Certification
1 of this report complies with all applicable
See Attachment — for comments.
specifications and standards.
O
Attached
SlgtICtture(S)
is a list of discrepancies
of
and/cr conunents.
FCA Teem Member(s)
●
●Sub-Team
Ch alrpe=son
Fiuure 3
Pa~e 8 of 11
I
92
●
l“””
●
1!
.
.
.
.
.MIL-STD-1521B
AFPENDIX I
A. Deviation/Waiver Review Team Inszrzctions.
All approved
waivezs and deviations to mllltary specitlcations and standaraa
shall be reviewed and recorded.
Also, record any Part of the
FCA which fails to meet specifications or standards but
is not
—
an a?proved waiver/deviation.
I
B. Results of Team Review.
List the deviations~daivers against
the equipmen.t/computer:sofVaare being FCA’d that were
reviewed.
I
I
●
Fiqure 3
Page 9 of 11
,0 .,
93
MIL-sTD-1521B
APPENDIX I
—
—
—
—
—
—
—
—
●
—
—
—
.
—
—
●
—
o
z
—
—
—
—
—
.
i...
.
.
...-.................’..
.
.
.
.
.
I
HU-STD-1521B
.Xppzlmrx I
●✎
1
I
ATTACHMENT
FORM=
OUALZFXCATION
(For riqu.pme9VCompucer
II
REVIECf
Software)
Date
Contract:
Contractor:
Configuration
Item No.:
I
Qualification Test/Analysis results
Formal Qualification Review.
have been rev~ewed to verify that the actual performance of the
configuration item complies with its development or requirements
specification(s) and that sufficient test results are available to
ensure the configuration item will perform in its system environment.
~.
I
Attached
●
is a
list
of
the documents
reviewed.
Check One
I
o
Results reviswed=atisfy
FQR requirements and the eonfiquration
item is qualified for ent~~ into the Government Invento&.
El
Results zeviewed are unsatisfactory/insufficient
fOr FQR- FQR
will be delaved until it is determined that sufficient
information & the configuration items Qualification is available.
Signature(s)
of
FCA
Team Member(s)
Figure 3
Page 11 of 11
95
nxL-sTD-1521B
APPENDIX I
SAWLE
AITACHFIENT
PCA CHECKLIST
The following haz~ware, computer software, ~ocumentation
available, and the following tasks shall be accomplished
PCA .
shall be
at the
Elardware:
Computer Software:
Documentation:
(1)
(~)
[3)
(4)
(5)
[6)
[7)
(8)
(9)
(10)
(11)
(12)
(13)
Approved final dzaft of ‘the configuration item
product specification.
A list delineating both aPProved and OdtStanding
changes against the configuration item.
Complete shortage list.
Acceptance test procedures arid associated test
data.
Engineering Drawing Index.
Opezating, maintenance, an< illustrated parts
breakdown manuals.
List of approvefl natezial :eview board actions
on waivers.
Proposed DD Form 2S0, “Material Inspection and
Receiving Report-.
Approved nomenclature and nameplate.
Manuscript copy of all
CSCI manuals.
Computer Software Version Description Document.
Curzent set of listings and updated desiqn
descriptions or other means of design portrayal
for each
CSC1.
FCA minutes for each configuration
item.
——
——
——
——
——
——
——
——
——
——
——
—.
——
Figure 4
Page 1 of 20
96
●
141L-5T9-151LB
APP2NDIx 1
I
i.
,.
SAMPLE C==T?FICATION
SAMPLE PCA CHECKLIST
ATTACHMENT
[CONTINUED)
Yes
—.
No
Tasks :
[1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
●
(9)
[10)
(11)
(12]
(131
Define Product B~seline.
Specification Review and validation.
Drawing review.
3eview acceptance test proced’ares and :esults.
Review shortages and unincorporated ~csign
changes.
Review deviations/waivers.
Examine proposed DD 250.
Review contractor’s Engineering Release and
Change Control System.
Review system allocation d~cument.
Review Software User’s klanuals, Software
programmer’s -nuals,
Computer sYstem Diagnostic
Manual, Computer System Operator’s Manual, and
Firmware
Support Manual.
Review CSCIS for the following:
(a) Top-1evel and lower-level Computer Software
Component design descriptions or alternative
design portrayals.
(b) Top-level and lower-level Compute= Software
Component interface requirements.
(c) Data base characteristics, storage allocation charts and timing acd sequencing
characteristics.
Review packaging plan and requirements.
Review status of Rights in Data.
——
.—
.—
——
.—
.—
——
-
——
——
——
——
_—
——
—.
——
Figure 4
Page la of 20
97
.
I
141L-sTD-1521B
APPENDIX I
WY sIcu
ATTACIME31T
c0NFIf3JRAT10t4AUDm
rwi)
mR
CONFIGU’NiTION ITEM NO.(S)
CONTRA~
NO.
PRIMS CONTRACTOR:
EQUIPMENT MANUFACTURERS:
APPROVED
APPROVED
BY
(DESIGNEE)
CONTRA~OR
BY
(DESIGNEE)
CONTIUiCTING AGENCf
DATE
DATE
Figure 4
Page Z of 20
98
●
●✎
HIL-STD-152A8
APPENDIX x
DEFINITION
OF TERMS
com’fENT - A note-explaining, illustrating, or criticizing the
~
of a writing.
Items of this nature should be ●xplored
by the contractor and/or the Contracting Agency, but corrective
action is “NOT necessary .to.successfully .accomplish a PCA.
DISCREPAN&Y - i+not explaining, .illustrliting, or criticizing ths
difference between writings.
A note shmfing the variance between
what exists and what is acceptable.
Items of this nature shall
be rectified by the contractor prior to successful accomplishments
of a PCA.
1.
0 ,,
I
Figure 4
Page 3 of 20
! 0,
99
MIL-STD-152LR
APPENDIX I
SCOPE/PCRPCX5E
A Physical Confis=xa:icn Audit (PCA) was condac:ed on the
following ead itens of eq.~ipme>tlcomputer softvare:
CONFIGURATION
ITEN
NO>~;JCLATr~E
PART NUMBER
SERIAL ?40.
NSN
—
The pu-rpose of the PCA vaa to ensure accaracy of zhe identifying
documentation and to establish a product baseline.
The establishment of a pro~uct baseline for equipnent/computer
software
is not to be construed as meeting Contracting Agency
requirements for delivery 5y the contractor of an operational
system meeting approved acceptance criteria.
Figure 4
Page 4 of 20
100
●
1“
I
t41L-STO-1521~
APPENDIX I
o
I
1
“’
CERTIFICATION SHEET NO. 1
(For Equipment/Computer Software)
Contract:
Date
Contractor:
Product baseline.
The following documents of the issue and date
shown comprLse the product baseline for the listed equipment(s)/
computer software:
●
ASSR4SLY TOP
DRAWING NO.
SPEC NO.
ISSUE
EQPT./COMP.
SOFTWARE
NOMENC’f.ATURE
CONFIGURATION
ITEM NC.
)
\
signature(s)
of PCA Team Member(s)
●☛
*
● “Team c halrperson
“Sub-Team Chairperson
.
Figure 4
Page S of 20
.. .
,,.,
.“.
IYIL-S?I’D-1521B
APPENDIX I
~EYSICU
CONFIGURATION
~ERTIFICATTON SHE9
(For Equipment/Computer
Conzzact:
AUDIT
NO. 2
Software)
Date
Contractor:
—
Specifications have been
spa cification Review and Validation.
rev~ewed and validated to assure that they adequately define the
configuration item and the necessary testing, mobility/transportability, and packaging requirements.
Check One
❑
I
m
u
The Type C Specifications are complete and adequately
define the configuration item. They shall, taerefore,
constitute the product baseline.
See Attachment _ for
couments.
The Type C Specifications
list of Cisczepancies.
.signatu:e(s) of
are unacceptable.
Attached
●
is a
PCA Team Ne.mber(s)
●
*Sub-Team Chairperson
Figure 4
Page 6 of 20
iwi
●
I1
I
UIL-STD-1521B
-APPENDIX
●
I
The
Soecificacion Review and Validation Instructions.
in
paragraph
B.
Below shall
speclflcatlons llsted
detailed
be reviewed for compliance with the applicable requirements.
Each Specification shall serve as the basic document for
‘anfig~ration control of the subject configuration items.
The information contained within the specifications shall be
audited at the PCA.
A.
B.
Revlewana
1.
. . ..Results:
.
Specifications
SPEC NO.
2.
. ..
Vallaatlon
PART No.
Reviewed ’and Validated
.
DATE
EQP7 ./COHP .
somwAR.E
NOMENC2ATL~
CONFIGURATION
ITEM NO.
Specifications Reviewed and Disapproved:
(Provide attachment foz causes. )
Figure 4
Page 7 of
103
20
HXL-STD-1521B
APPENDIX I
P!I’YSICRLCOSFIGL?W?ION
xDD~
CERTIFICATION 5HSET F?O. 2
(Equip.ZencJ
Contzacr:
Date
Contractor:
Drawings have been compared with the equipment to
Drawi nq Review.
ensure that the latest drawing change letter has been incorporated
into the eqaipment, that part numbers sgree with the drawings. and
that the drawings a:e complete and accurately describe the equipment.
Attachment
_
is x list of the drawings reviewed.
●
Check One
•1
The drawings axe complete and accurately describe
See attachment — for comments.
D
Attachment
Signature(s)
Of
_
s a list of discrepancies.
pm
Team Members(s)
the equipment.
●
“Su5-Tea.m
Chairperson
Figure 4
Pa9e 8 of 20
104
●
MIL-STD-1521 B
APPENDIX I
A.
The following drawings were reviewed
Drawinq Review Results.
by tlie PCA drawing reviewing sub-teams:
DOCUMENT
DOCUMENT TITLE
NUMBER
.
Figure 4
Page 9 of 20
105
MIL-sTD-1521B
APPENDIX I
/WYSICAI. CONFIGURATION
AUDIT
CERTIFICATION SHEE?T NO. 4
iEquipmenc )
Contract:
Date
Contractor:
Acceptance Test Procedures and Results. The acceptance test results
have been reviewed to ensure that testing is adequate, properly done,
and certifie3.
Attachment
_
is a list of the documents reviewed.
Check One
O
Prcacefiuresand results reviewed satisfy the requirekts
and are accepted.
See Attachment _ for comments.
O
Attachment _
Signature(s)
of
PCA
is a list of discrepancies.
Team
Member(s)
*
I
●Sub
-Team Chairperson
Figure 4
Page 10 of 20
106
NIL-SID-1521B
APPENDIX I
Wfsx
CA>
CONFIGURATION
A~m
CERTIFICATION SNEET NO. 5
(For ~uipmcnt/Computer
Software)
Contract
Date
Contractor:
.
Review of Shortawes and Unincorporated Desiqn Changes.
The shortages
and unincorporated design changes listed on the proposed DD Form 250,
‘Wterial
Inspection and Receiving Report-. and ocher records have
been reviewed.
Check One
●
There are no shortages or unincorporated
design changes.
Attachment
is a list of shortages and/or unincorporated
design chan~s.
and the recommended corrective action required.
)
Signature(s)
●
*5ua-Tearn Cha~zperson
I
Figure 4
Page 11 of
1.
●
107
20
I
I
HIL-STD-1521B
APPENDIX
II
1
,i
A.
Review of Shortages and Unincorporated Design Changes.
shortsges and unzcorporated design changes llsted
on the proposed DD Form 250, ‘Material Inspection and
Receiving Report”,
shall be zeviewed by the Contracting
Agency or their designated representatives for a determination of what changes should be accomplished in the
field and what changes should be accomplished at the
contractor’s facility.
The Contracting Agency shall
also determine if the reported shortages and unincorporated changes are complete.
All
B.
Results.
List the shortages and unincorporated design
changes that were reviewed in compliance with requirements.
.
?iqure 4
Page 12 of m
I
108
I
I
m.
L-sTD-1521B
APPENDIX I
I
~@’I
!
1
1
I
fJ?#T1~Ic&+rfjNs=
NO. 6
(Fox Equ~pment/Computer Software)
I
Contract:
II
Date
Contractor:
1.
Review Deviations/Waivers.
A review o: all deviations/waivers to
mal~tary speclflcataons and standards that have been approved.
The purpose is to detarmine the extent to which the equipment(s)/
computer software undergoing PCA vaq from applicable specifications and standards and to form a basis for satisfactory compliance
with these specifications and standards.
In accordance with this paragraph, all applicable aviations/
waivers have been reviewed with the follwing results:
●
Check One.
,
The ●quipment(s)/computer Software listed on Certification
Sheet No. 1 of this report complies with all applicable
specifications and stan~ards. See Attachment _ for comments.
Attachment _
is a list of discrepancies and/or comments.
Signa:ure(s ).of PCA Team Member(a)
.
*Sub-Team Chairperson
!0
Flqure 4
Page 13 of 20
109
lJIIL-sTD-ls21B
APPENDIX I
A.
Deviation/Waiver Review Team Lnstractlon.
All approved waivers
and deviations to m~litary Speelxxcatxons and srandards snail be
reviewed and recorrled. Also, recora any pare of the PCA which
fails to meet specification
or standards —
but is not an approved
waiver/deviation.
B.
Results of Team Review.
List the deviations/waivers against the
equ2pment/computer software being PCA’S that were reviewed. ,
~
I
Figure 4
Page 14 of 20
110
nIL-sTD-ls22a
APPENDIX S
PXYSICAi. CONFIGLPLATION .aL~T?
I
(For Equipment/Compuuer
Contract:
:oftware)
Date
Contzaetor;
Examination of the Proposed DD 250. The DD Form 250 has been examined
?eflnes the eqaipment/computer software
to ensure tha: ~t adequately
and that unaccomplished casks are inlc~ded as deficiencies.
Check One
D
‘m
n
The .DD Form 250 adequately defines the equipment/computer
software and all unaccomplishe~ tasks are
included as
deficiencies.
_
Artac.hment
Signature(s)
of
is a list of discrepancies
andlor comments.
PCA Team Member(a)
I*
I
‘Sub-Team chairperson
Figure 4
Page 15 of 20
111
.
MIL-STO-15219
APPENDIX 1
A.
of the Proposed DD Form 250. The FroPosed DD Form
and an accurate definition
of the equipment/computer software.
Unaccomplished tasks,
shortages, and certain specified discrepancies uncovered at the
PCA shall be inclzded in the DD Form 250. If the equipment/
computer software is to 5e shipped from the plant, the Program
Office representative will recommend to the CAO that the DD Form
250 be executed i-naccordance with the terms of the contract.
a.
Results.
Include a statement that the proposed.DD Form 250 was
‘d
and was recommended.
Examination
I
I
I
Figure 4
Page 16 of 20
112
●
I
rA*A.-a 4”-..&*APPENDIX I
~=SICAL
CO??FIGURAT ZON AL~IT
c~RTIFI~~TON
~
(For Equipment/Computer
Contract:
Software)
Date
Cantractoz:
Review of Contractor’s Enaineerinc Release acd C5anae Control system.
The cantraczor’s engineering release system and change control procedures have been reviewed to ensure that they are adequate to properly
control the ?rocessing and formal release cf engineering changes.
I
Check One
The contractor’s engineering release system and change control
procedures are adequate for
the processing and formal release
of engineering changes.
See Attachment — for comments.
I
o
)
Attachment
SigmSture(S)
I
of
_
PCA
Team Metier(s)
●
I
I
I
“.Sub-Team Chairperson
Figure 4
Page 17 of 20
113
MIL-STD-1521 B
APPENDIX I
r~ 3
(For EqulpmentjCompu~er
Contract:
NO. 9
Software)
Date:
Contractor:
The following System Allocation
System Allocation Document Review.
form drawings have been reviewed and validated to ensure
that
they adequately identify, and are compatible with, the shipping
instructions .
book
I
Check One
O
The System Allocation Documenc
defines the equipment/computer
location.
is complete and adequately
software scheduled for each
O
The System A:locacion Document
a list of discrepancies.
is unacceptable.
O
This task is notrcqaired
Signature
o
Attached is
by contract.
●
I
I
.
●Sub-Team
Chaizperscn
Figure 4
Page 18 of 20
114
.
●
MIL-S7’D-1S21B
APPENDIX I
I
The System Allocation Documents, both Part I and Part Ii,
1.
applicable to ,the contract shall be reviewed to determine their
accuxa~
and insure chat they adequately desczibe the equipment/
compute; software.
2.
The followiag
information
shall be recorded:
Part I.
a.
b.
e.
d.
e.
●
1
System employment and configuration.
Specification reference.
Location.
Mission Eauiument.
Confiquraiio; Item Number
Short Title
Part Number
Serial Number
Installed equipment/computer software.
Configuration Item Number
Short Title
Part Number
Serial Number
Drawing Title and Number
Number of sheets
Issue Number.
Part II.
I
Location.
Specification Numbe:
Equipment/computer software nomenclature.
Configuration Item Quantity.
Fkssembly Drawing Number
3. Insure that the System Allocation
tke priorities and shipping inatrucciona.
I
a
~.)~$--~
-.
AIL=cazion
Documents are compatible .wi
Alloc=ti~n Do-e~t
Revi5w Res.:!ts. The fallowing Sjrs:ez!
Dorxtimen:swere reviewed by the PCA ~eviewing Sub-Team:
DOCUMENT TITLE
DOCUMENT NUMBER
I
!0
Figure 4
Page 19 of 20
.
11s
M:L-STD-1S21B
APPENDIX I
mYSICU
CCNTIG-URATION AUDIT
~TI?ICATION
SliZET NO. 10
(Equipment)
Contract:
Date
C3nzraetor:
1. Review of Louis’tics Suormrt Plan for Pre-operational Suooort.
The Logistics Support Plan ZOr Pre-operational Support has been
reviewed to ensure that it is adequate to support the acquisition
phase and is compatible with the operational phase maintenance concept
and support requirements.
Check One
c1
The contractor’s Logistic Plan for pre-operational SLIppCrt
will fulfill the acquisition phase req~izements and is compatible with operational phase needs.
c1
Attachment
—
Review of Lonq Lead Time Items and Provisione~ Items Processed to
P&.
Long Lead Time items released anti items provisioned, prior to
=
have been reviewed to ensure that obsolete items resulting from
pre-PCA design chanyes are ?urged from the system. “Where basic items
may be upgraded “by rework or modification these actions have been
verifie~ as accomplished or in process based upon design change notice.
●
Check One
❑ Long lead time itezs aad provisioned items processed, grior
aze all of current configuration
a
Attachment
Signature(s)
of
_
to
PCA,
at time of PCA or are in work.
PCA
Team
?Iember(s)
●
●Sub-Team
Chairperson
Figure 4
Page 20 of 20
.
116
●
UIL-STD-1521B
APPENDIX J
100.
ApplicationtGuide
for Tailoring
14:L-STD-1521
100.1 Scooe
for the
This
appendix
sets forih guidance
cost
effective
when
of this standard
application
of the requirements
this
standard is contractually invoked during the acquisition
process.
This
appendix serves as guidance for the activity responsible for
the preparation of contract requirements and does not form ‘a parr
of the contract.
100.2 Purpose
The guidelines
contained
herein
the Department
implement
af
Defense
Directive
4120.21,
Specification
and
Standards
which
requires
Application.
all
DOD
components
to
●pply
selectively and tailor military specifications and standards prior
to their contractual imposition and:
a.
Eliminate
b.
Provide for adding/modifying necessary technical
audit factors not included in MIL-STD-1521.
c.
Eliminate redundancy and inconsistency
specifications end standards.
inapplicable and unnecessary
requirements.
with
review
other
and
con:ract
100.3 Objective
The objective of this guide is to establish the applications
and
MIL-STD-1521
limitations
of tailoring
MIL-STD-1521.
is nut .s
stand-alone document.
It
is dependent
upon the work
effort
specified
in the contractual requirements (e.g., SOW, etc.) The
taiioring of specifications shoulcf cake place
in
all
phases
of
military
procurement, but is ●specially applicable to the initial
preparation
solicitation
and
contract
stages of
package
negotiation.
DeDendino
tme
‘of ● rid-item(s) under
uno n the
may
procurement, the r;vievs-and” audits outl-ined by MIL-sT9-1521
or may not be required for all programs.
100.4 Considerations
—for ?ailorinq
100.4.1 .Relationshiu .—
to the Statement
of .—
work
—
The Program Manager must
keep
in mind
that technics: ,reviews
visibility
proviUe
into the contractor’s implementation of the
work effort required under the terms of the SOW and the contract
timely
and effective
a:tention
to the technical
to
assure
inrer?retation of contract ,requiremen:s.
The
key
to
tailoring
MIL-STD-1521 is to match the MIL-STD-1521 requirements against the
It
details of the applicable SOW/Contractual task requirements.
-*i11 become
obvious
immediately
that MIL-STD-1522 may contain
117
.
MIL-STD-1521B
APPENDIX J
technical review factors that are not applicable to the contract
under consideration.
(For example, if a contract t!oes not include
all references
%0 the review of Computer
computer softvaze,
.,i~l not apply. ) When
materials
in
MIL-sTD-1521
Software
MIL-STD-1521 is used,
then a cask containing
the applicable
requirements will be specified in the SOW. Review factors not set
forth in MIL-STD-1521 but considered
necessary
because
of the
nature
of the particular Drogram shoulr! be adced in the S3W. By
th-is evaluative
carefully going through
process
the technical
review
and audit requirements will become p?agram specific rather
than an all purpose document to be continually
negotiated
during
contract performance.
100.4.2 Elimination ~
—
iledflndanc~and
Ambiquitv
—
While MIL-STD-1521 is the broad program
document
for :echnical
rev iews and audits,
in existence also require
other standards
reviews or audits.
MIL-STDS
for
For
example ,
technical
maintainability,
system engineering and others can
reliability,
Review of these aspects
of the
require reviews and/or audits.
would also be required under MIL-STD-1521; therefore,
design
if
are contractually
together
stipulated
with
such standards
MIL-STD-1521,
the SOw should include a provision to show how and
whether the technical review requirements of these other standards
with
technical reviews/audits in MIL-STD-1521.
can be combined
Combining revievs
does not nullify other MIL-STD(S), ‘Plans”, etc,
which
requirements
for reviews/audits.
The contract
contain
comprehensive
integrated,
should require the minimal
technical
review effort that will provide the desired visibility and
design
assurance of contract compliance.
100.4.3 Contractor Participation
●
in Tailorina
‘When requiring a particular revie.~ or a“udic, it is important
that
the topics to be reviewed are aligned zo the program requirements.
Therefore, the offeror should be given an opportunity to recommend
The
and identify topics/items he considers appropriate.
changes
for proposal
program office should request, in the instructions
the MIL-sTD-1521
preparation,
that
offeror
recommend
the
topics/items and their related details
to be covered
at the
reviews or audits required by tha SOW. This will allow
various
~y a~ditions
the offeror to tailor the topicsiite!ns ard details
for the particular review/audit.
and deletions
In addition,. ii
must be recognized that effecive tailoring requires several poir,ts
of review. The requir~ment, hoveve:, for the rev!ev/audit must be
.
‘inalized
prior :0 concract award.
100..4.4 Comalexitv
a.
and
System/Segment/subsys:em/conf iguracion item complexity
type of program is central in determining both the Reed for
a small
When developin~
and the number of such reviews.
218
●
i
-MIL-STD-15219
APPENDIX J
non-complex
sysrem some reviews nay not be required, or, if
Tke tailoring procedures
..
re~zizet, =ay be -.”.
‘:-:”e~
i> S=z?e.
should
result
either
in the ●xclusion of
disczssed
earlier
a
M;L-STD-1521 0: in a tailored MIL-STD-1521 that reflects
review effort.
Conversely, in a
limited
scope
technical
very complex development the review process will increase in
levels and numbers of reviews.
!’
b.
I
●
application
of
is
In :addi:ion to the -above, the tiegree
the configuration item state of development
upon
dependent
commercially
available)
or the
(example, ne,d &esign vs.
For example:
a
deqree
of any tnodifica:ions. if involved.
newly developed item may require the majority of the rev iew
a commercially available
and audits, while
topics/items
configuration item with the appropriate documentation. i.e..,
verified
may
test results, specifications, drawings, ●tc.
require reviews
or audits limited to its application to the
In the case of modified designs
program and its interfaces.
the
and
effect
of
degree
the
one must
consider
Reviews
and ● udits may be limited to the
modifications.
modifications and their interfaces.
100.5 Scheduling
~
Technical
Reviews and —
Audit”s
——
Reviews
and Audits
for Tech~ical
is ●xtremely .
The schedule
important.
[f
they
● re
conducced
too
●erly,
the
item for review
will not be adequately defined.
Conversely, if the review is too
commitments could have been made erroneously,
late, the program
For planning
and correction vill be both difficult and costly.
a
good
method
for
scheduling
technical
reviews is to
purposes,
For example,
relate them
to the documental ion requirements.
the hard,~are Development Specification or
schedule
a PDR after
and Software
Test Plan are
Software Top Level Design Document
available,
since
of the PDR
is to assess
the
the
essence
these requirements
of these
contractor’s approach
to meeting
not only on
are dependent
documents.
Scheduling
of audits
hard-~are/software
availability
but
documentation
also
on
availability,
and
the completion of the acceptance qualification
documentation
Table 1 contains
a list of the primary
tests.
wi:h ● ach review 07 aud it aed
the ●stimated time
associated
phaskng:
SCHEDULING
Review
SRE
TABLE 1
TZCXSICAL ?.E’JIZUSAND A2!31TS
Time
Phase
.—
Usually accomplished in :he
“COnCept Exploration phase.
However, may be used in
other phases when the
Primary Documentation
various anal:~sis and
trade study reports
used to develop zhe
system/segmen:
I
I
..
MI L-STD-1521B
APPENDIX J
Time Phase
?rimarv Documen:atio7.
Concept Exploration phase
is not accomplished.
re~irements
for
specification.
SDR
usually in the De!nonstration
and Validation phase.
SystemlSetjme9t
Specification,
preliminary Gpezational
Concept Document,
preliminary Software
Requirements and
Interface Rcquiremencs
s.pccificat~ons, analyses,
trade studies, Drawings
Level I DOD-D-1OOO.
SSR
USUAllY early in Full
Scale Development
Specification. Interface
Requiremen~s
Specifications,
Operational Concept
Document.
PDR
usually accomplished in ~he
Demonstration and Validation
and/or Full Scale
Development Phase
Development, Type B
Performance
Specification,
Drawings Level I
DOD-D-1OOO, Software
Top Level !2esign
Document, Software Test
Plan, preliminary
Computer Resources
Integrated Support
Document, preliminary
Computer System
Operator’s Manual,
preliminary Software
User’s manual,
preliminary Computer
System Diagnostic Manual.
CDR
Usually accomplished in the
Full Scale Development
phase
Draft Product, Type C
Specification, and
referenced documentation,
Drawings Level 1 or 11
DOD-D-1OOO, Software
Detailed Design Document,
Interface Design
Document(s), Data Base
Design Doc’.Jment(s),
Review
:20
the
●
●
m,1
U2L-STD-1521B
APPs.VDIX J
Review
Time
.—
Primarv
Phase
Documentation
Test
Description, Computer
Resources Integrated
Support Doc”ument,
Seft.iare Programmer’s
Manual, Firmware
;Support Manual, Informal
Test Descriptions/Test
Procedures, Software
Development File(s).
Software
I
TRR
Usually accomplished in
the Full Scale Development
phdSe
Software Test Procedure,
Informal software
test
results (of development
tests) .
FCA
.Usually accomplished
end of Full Scale
Development
Test plans, test
descriptions, test
procedures, Software
Test Reports, Computer
System Operator’s Manual,
Software User’s Manual,
Computer System
Diagnostic Manual.
PCA
Usually accomplished early
in the initial production
when the developing
contractor is preselected
as the production
However, may
contractor.
be accomplished at the end
of Full Scale Development
when the developing
contractor is not
preselected as the
?roduction contractor.
And the PCA is repeated
with each .saktseqaen:
contractor or break
in produc:ior..
I
●
at
Final Part II
Specification/Type C
Product Specifications
and referenced
documents
Drawings
and dravangs.
Level 11 or 111
DDD-D-1OOO.
Software
Product Specification,
Version Description
Document.
Al:hough the time frame for reviews and audits is suggested above,
they may
vary depending on the partic~lar proqram.
The schedule
for each review or aadit may be requesteti from the cfferor as part
of his proposal, or as part of the system engineering management
plan (which can be part of the proposal).
121/122
~
MIL-STD-:5219
APPENDIX K
●
“-
110.
Production
Readiness
110.1 For specific
Rev iew.
Review
—.
guidance,
(PRR)
see AFSCR 64-2, Production
●
)
123
Readiness
MIL-STD-1521B
Custodian:
Air Force - 13
Preparing Activity:
Air Force - 13
Review Activity:
Air Force - l13,~l,Bo,B5
(Project
124
9
CMAN-O-006)
I
I
●✍
1
111111
Orsslak
m.uv.
● swm
●om ●mws7m
use uca
BWdfNESS
REPLY
MAIL
mm”8?
no.
,,*”---- -C
MS?.oa
-al..c AIOV-99.***=*O===
●
●
●
●
COMMAMYJE?,
!I[ACOUAR7ER5
Electronic
●
✎
SYSTEMS 3; ’J IS ION [HO ES3)
ATN:
ALET, SYSTEt4S E!iGI !!EE?!
H.W.CCM
AFL3. ‘a
‘7711
!!-
!G
C! ’?! SICt!
-
..,-.
.~
...
~
. . .
I
=N4DARDIZATIOR
I
a-v
auaa
I* 00---7
DOCUMENT lMPR0VEMEN7
rk Im!nnuml - R&?w w)
Tltm
PROPOSAL
I
?**S O*
Onti”,
zArto”
,m-
-,
D
tma
o
m“an,J-Oa,:
●
❑
I
I
.
I
I
.
.
I
n—m.-.
● nlMAnr,a
I
c1
,.
——:

1521B - Product Lifecycle Management

Transcription

Similar documents

Pay Application #4 - the Public Building Commission of Chicago

Mark Rycraft

How to Open an Account with Great Lakes Credit Union (GLCU)

Qualification Form - Integrity Construction Service

AGREEMENT FOR TRANSPORTATION SERVICES Contractor

yaggy colby - Olmsted County

2008 - SEAOSD

the PDF

Patrick D. Harder, Esq. Nossaman Guthner Knox

Software development