Getting Formal with vManager

GETTING FORMAL WITH VMANAGER
Stuart Hoad
October 2015
Proprietary and Confidential
AGENDA
• How, where, and when PMC uses formal methods.
• Overview of the verification management process.
• Why we need to structure formal work.
• Metrics for formal verification management.
• Formal verification planning.
• Using Jasper with vManager for end-to-end proofs.
Proprietary and Confidential
2
WHERE IS FORMAL USED IN A TYPICAL
PMC DESIGN FLOW?
• We deploy formal methods throughout the design
and verification process to complement and enhance
other modelling, design and verification techniques.
• Architectural proofs/exploration
Silicon
• Design exploration and bring-up
• Interface proofs
Emulation/FPGA
• Embedded RTL proofs
Simulation
• End-to-end checks
• Connectivity/integration checks
Formal
• Configuration checks
• Power/clock gating proofs
• Formal performance analysis
Modelling
Requirements
Silicon
• Debug closure
• …
• Formal techniques are actively planned and
managed at PMC to maximise integration and ROI.
Proprietary and Confidential
3
VERIFICATION MANAGEMENT AT PMC
• PMC design and verification teams produce plans with Cadence
vPlanner and monitor progress and status with Cadence vManager.
• Test-cases, and functional coverage for simulation and emulation are
mapped to a verification plan.
• Allows a structured approach to be taken.
• Allows progress towards completion to be measured.
• Allows verification holes to be identified and filled.
• The same should apply to formal methods.
• Working with Cadence, PMC have now added the same planning and
tracking capability to formal methods with vPlanner and vManager.
Proprietary and Confidential
4
WHY FORMALISE FORMAL?
• Using formal methods in an unstructured, interactive manner is quick
to get results with minimal barriers and effort.
• Whilst getting started quickly can be a good thing, PMC has found
problems:
• Lack of rigour.
• Difficult to track progress or predict completion
• Hard to understand if formal work done was useful.
• Overlap and duplication with other verification techniques
• Difficult to align the results from different formal techniques.
• This contributes to poor actual and perceived ROI.
Proprietary and Confidential
5
INTEGRATION WITH SIMULATION AND
EMULATION
• Formal verification can fit in the same
structure as existing verification.
• We use vManager to
bring results into the
same view.
Requirements
VPlan
• Metrics and use model
for formal verification are
very different.
• Need to determine the
contribution (or lack of!)
of properties in a
different way.
Proprietary and Confidential
VManager
Emulation/FPGA
Simulation
Formal
Requirements
6
Silicon
FORMAL MANAGEMENT
FORMAL VERIFICATION PLANNING
• We use Cadence vPlanner to create vPlans to plan and capture the
metrics we wish to track.
• Everything belongs in the same vPlan.
• Verification planning must include all methodologies.
• The entire verification plan is visible in one place.
• Pick the right methodology for the problem.
• A combination of techniques give the best coverage and ROI.
• In many cases vPlan does not exist for sub-block simulation.
• For formal a significant part of the verification is at this level, so a vPlan is
created.
• We use the block-level vPlan for formal design bring-up too!
• Use a hierarchical vPlan approach to bring the sub-block plans into
the higher level sub-system or device plan.
Proprietary and Confidential
7
FORMAL MANAGEMENT
FORMAL VERIFICATION PLANNING
• We are creating and mapping assertions but how do we know they
are contributing anything useful?
• Review of the properties.
• Assertion metrics and COI analysis.
• Check the precondition and witness covers.
• Formal code coverage.
• Use of JG-COV app produces a code coverage report for some or all of the
assertions.
• We need to capture the quality metrics in the plan too.
• Proofs in formal are often incomplete (bounded); in our methodology
we can still extract value from these.
• We can use cover properties and design analysis to show the proof has
reached an acceptable depth (reasonable bound).
Proprietary and Confidential
8
USING JASPER WITH VMANAGER
FORMAL VERIFICATION PLANNING
• PMC block level vPlan is built from a standard template which
includes:
• Interfaces
• Embedded
properties
• Expected
behaviours
• Core features
• Quality metrics
Proprietary and Confidential
9
FORMAL MANAGEMENT
FORMAL VERIFICATION TRACKING
• vManager is used for tracking results
• Formal runs are made either in Jasper directly or under the control of
vManager.
• Property status can be analyzed across one or more runs.
• The vPlan can be imported
and properties mapped to
indicate the current status
of the verification plan.
• Jasper Visualize can be
launched from vManager
to view covers or failures.
Proprietary and Confidential
10
END-TO-END PROOF EXAMPLE
THE PROBLEM WITH END TO END PROOFS
i/f
• Formal end to end proofs on
complex designs are
sometimes intractable.
i/f
Interface properties
i/f
Interface properties
Interface properties
Interface properties
i/f
Interface properties
i/f
Sub-block 0
FBM
i/f
Interface properties
End to end properties
• Decomposition is used to
break the design into
manageable chunks.
Subsystem
i/f
i/f
i/f
i/f
i/f
i/f
i/f
i/f
i/f
Sub-block 0
• But now we have a set of
block level formal proofs, not
end to end proofs.
i/f
Embedded
properties
Sub-block 1A
i/f
i/f
i/f
• Solution: Use vManager to
compose block proof results
and interface guarantees.
i/f
CSR
(PCBI/XCBI)
i/f
i/f
i/f
i/f
CSR
FBM
i/f
i/f
11
Sub-block 2
i/f
i/f
i/f
i/f
Sub-block 1B
Proprietary and Confidential
i/f
i/f
Sub-block 3
END-TO-END PROOF EXAMPLE
OVERVIEW AND TIMELINE
• Blocks have common setup and
shared interfaces.
Emulation/FPGA
• Re-uses effort.
• Formal exploration, bring-up,
and verification for each block.
• Block results composed.
• Simulation starts with
confidence, reduced debug
effort, and is efficiently targeted.
• Formal for fast interactive debug
and breadth. Simulation and
emulation provide the deep
verification.
Proprietary and Confidential
Top Simulation
Setup and interface definition
• Common re-used interfaces allow
composition.
Formal Bringup Block FormalV
Formal Bringup Block FormalV
Formal Bringup Block FormalV
Requirements
12
Top Recompose
Silicon
END-TO-END PROOF EXAMPLE
FORMAL DESIGN BRING-UP
• A vPlan for each block is
created with a standard
block template.
• Minimal designer effort in
vPlan creation and
vManager usage.
• Jasper results are exported
to vManager.
• Goal is expected behaviours
covered and an absence of
assertion failures.
Proprietary and Confidential
13
END-TO-END PROOF EXAMPLE
BLOCK LEVEL VERIFICATION
• Block verification vPlan is
developed from design bring-up
vPlan.
•
Testpoints and covers are refined
to provide a functional testplan.
•
Assertions and covers are mapped
to refined testpoints and covers in
vPlanner.
•
The verifier adds covers to judge
useful property bounds.
• We can run multiple tests with
different constraints or
techniques and merge to view
combined coverage.
Proprietary and Confidential
14
END-TO-END PROOF EXAMPLE
TOP-LEVEL VERIFICATION
• A separate top-level vPlan is created and results mapped to this.
•
Top-level vPlan
references block level
vPlans.
•
Interface properties are
gathered into a single
testpoint for each
interface.
• All block test results are
merged in vManager and
mapped to the top level
plan.
• We now have a complete view of the status of all formal design and
verification work in one place.
Proprietary and Confidential
15
END-TO-END PROOF EXAMPLE
TOP-LEVEL VERIFICATION
• In many designs Jasper will not be invoked at all at the top-level.
• vManager is still used at this level to merge the sub-block results.
• Allows the composure of block level proofs to give top-level confidence.
• Allows interface guarantee coverage to be measured.
• Results from multiple separate regressions can be merged.
• Aligns with simulation and emulation plans.
• vManager links and tracks all results and tests in one place.
• With composition of results, end-to-end formal proofs can effectively
be made.
Proprietary and Confidential
16
SUMMARY
• Being able to track formal results and status enables rigourous
application of new formal techniques.
• Integrating formal metrics from Jasper into the existing vPlanner flow
provides a complete view of design and verification status.
• vManager allows Jasper to be a first class peer in a high-quality
design and verification process.
• Managing formal as a unified process with vManager maximises
Jasper efficiency and ROI.
• Jasper and vManager becoming increasing integrated. Current
version has some limitations, but essentially working and deployed.
Proprietary and Confidential
17
NASDAQ: PMCS
Proprietary and Confidential
18