talk - SPaCIoS

Model-Based Vulnerability Testing
for Web Applications
F. Lebeau, B. Legeard, F. Peureux, A. VERNOTTE
FEMTO-ST Institute / University of Franche-Comté
UMR CNRS 6174, 25030 Besancon, France.
Smartesting R&D Center, 25000 Besancon, France.
SecTest 2013, Luxembourg
March 22, 2013
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
1 / 22
Table of contents
Introduction
Current Dynamic Application Security Testing Techniques
Model-Based Vulnerability Testing
Detailed Process with Example
Discussion
Conclusion
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
2 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Web evolution and Security
Continued growth and complexity of the internet:
I
increasing ubiquity of uses (banking, e-commerce, social...)
I
increasing combination of technologies (server, client)
→ maintaining security is a real challenge
An urgent issue: Web Application vulnerabilities
The later a vulnerability is found, the more expensive it is to fix.
⇒ Vulnerability discovery part of the development process.
Two main techniques:
I
SAST: Static Application Security Testing
I
DAST: Dynamic Application Security Testing
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
3 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Web evolution and Security
Continued growth and complexity of the internet:
I
increasing ubiquity of uses (banking, e-commerce, social...)
I
increasing combination of technologies (server, client)
→ maintaining security is a real challenge
An urgent issue: Web Application vulnerabilities
The later a vulnerability is found, the more expensive it is to fix.
⇒ Vulnerability discovery part of the development process.
Two main techniques:
I
SAST: Static Application Security Testing
I
DAST: Dynamic Application Security Testing
Penetration Testing
Scanners
A. Vernotte et al.
Fuzzing
Model-Based
Security Testing
MBVT for Web Applications - SecTest’13
3 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Current DAST techniques
Manual/Tool-Based Penetration testing
Strengths
I Precise and Reliable to detect design flaws
I
Based on insights and experience
Weaknesses
I Fastidious (thorough testing for XSS)
I
Constant need of manpower and expertise
I
Based on insights and experience
D. Allan, Web application security: automated scanning versus
manual penetration testing, Somers, IBM White Paper, 2008
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
4 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Current DAST techniques (2)
Vulnerability Scanners
Strengths
I Point-and-shoot solutions
I
Efficient for a majority of technical vulnerabilities
Weaknesses
I Suffer from a fair amount of false positives
I
Struggle with complex vulnerabilities (Multi-step XSS)
I
Struggle with vulnerabilities related to business logic
A. Doupé et. al., Why Johnny can’t pentest: An analysis of
black-box web vulnerability scanners, Journal of Detection of
Intrusions and Malware, and Vulnerability Assessment, pp
111–131, Springer, 2010
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
5 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Current DAST techniques (3)
Black-box Fuzzing
Strengths
I Low cost solution
I
Efficient to spot unattended behaviors (or “black swans”)
Weaknesses
I Weak Oracle (crashes, freezes)
I
Improving the Oracle is challenging
R. McNally et. al., Fuzzing: The State of the Art:
DSTO-TN-1043, DSTO Formal Reports, 2012
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
6 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Existing DAST techniques (4)
Model-Based Security Testing
Strengths
I Efficient to address functional security properties
I
Automation capacity
I
Handle well application evolution
Weaknesses
I Needed effort to provide models
I
Needed effort to develop a concretization layer
I. Schieferdecker, Model-Based Fuzz Testing, Invited talk of the
5th IEEE Int. Conf. on Software Testing, Verification and
Validation, 2012
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
7 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Model-Based Vulnerability Testing (MBVT)
MBVT: Vulnerability testing based on models and test patterns.
Goal: Improve the accuracy and precision of vulnerability testing.
Precision
Accuracy
Capability to focus on the relevant part of the software
(e.g. from a risk assessment
point of view) depending on the
targeted vulnerability types.
A. Vernotte et al.
Capability to avoid both false
positive and false negative.
MBVT for Web Applications - SecTest’13
8 / 22
Introduction SoTA MBVT Process Discussion Conclusion
MBVT Overall Process
This MBVT approach is composed of 4 activities:
1. Test purpose definition
2. Model design
3. Test generation
4. Concretization, test execution and verdict
assignment
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
9 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Challenge: Multi-step XSS Discovery
A1 - Injection
A6 - Sensitive
Data Exposure
A2 - Broken
Authentication
and
Session
Management
A7 - Missing
Function Level
Access Control
OWASP TOP 10 2013
A3 - Cross-Site
A4 - Insecure
Scripting (XSS)
Direct Object
References
A8 - Cross-Site
Request Forgery
(CSRF)
A9
Using
Components
with
Known
Vulnerabilities
A5 - Security
Misconfiguration
A10 - Unvalidated Redirects
and Forwards
Multi-step XSS is a challenging vulnerability for automated
tools1 .
→ It requires knowledge from the targeted application.
This MBVT approach deals with this class of vulnerability, by
applying a “Def/Use” approach (All-def criterion).
Experiments have been conducted on WackoPicko.
1
J Bau et. al., State of the Art: Automated Black-Box Web Application
Vulnerability Testing, 2010.
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
10 / 22
Introduction SoTA MBVT Process Discussion Conclusion
1 - Test Purpose Definition
Vulnerability Test Patterns (vTP) are the entry-point of MBVT:
I
Based on a study from the ITEA2 DIAMONDS project
I
Express testing needs and procedure to highlight a breach
The goal is to translate vTP into a machine-readable language.
⇒ Reuse of the Smartesting Test Purpose Language 2 :
I
Designed for security means
I
Textual language based on regular expressions
I
Reasons in term of states to be reach and operations to be
called
2
J Botella et. al., MBT of Cryptographic Components, Lessons Learned
from Experience, ICST 2013.
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
11 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Test Purpose for Multi-step XSS
Name
Description
Objective(s)
Translation
from vTP
to test
purposes
Prerequisites
Procedure
Oracle
Variant(s)
Known Issue(s)
Affiliated vTP
Reference(s)
Multi-step XSS
...
Detect if an input can embed malicious datum
enabling a Multi-step XSS attack.
N/A
Identify a sensible user input, inject the malicious
datum <script>alert(rxss)</script>.
Find the page where the input is rendered, and
check if a message box ’rxss’ appears.
...
...
Reflected XSS
...
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
12 / 22
Introduction SoTA MBVT Process Discussion Conclusion
2 - Model Design
MBVT behavioral notation is based on UML4MBT3 .
A generic class diagram depicts the
structure of the SUT: pages, actions,
and in/out data (following the def/use
concept).
MBVT also requires:
I
Class Diagrams: defines the
static aspect of the SUT
I
Object Diagrams: defines the
initial state of the SUT
I
State Diagrams: defines the
dynamic of the SUT
3
F Bouquet et. al., A subset of precise UML for model-based testing, 2007.
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
13 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Modeling: WackoPicko Example
Class Diagram
Statemachine
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
14 / 22
Introduction SoTA MBVT Process Discussion Conclusion
3 - Test Generation
Test cases are generated using Smartesting CertifyIt.
Test generation process is driven by test purposes and models:
I
Test generator unfolds test purposes
I
Models give the path to follow in order to reach each stages
of a given test purpose
Result: A suite of abstract vulnerability test cases
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
15 / 22
Introduction SoTA MBVT Process Discussion Conclusion
4 - Execution and Verdict Assignment
Human Intervention during concretization:
I
List of malicious vectors (xml file)
I
Body of the SUT’s operations (HTTP level, Browser level)
Observation Technique for XSS: crawl the source page to see if the
injected vector has been sanitized.
Test terminology dedicated to Vulnerability Testing:
I
Test verdict is OK → Attack-pass: System is vulnerable
I
Test verdict is KO → Attack-fail: System is resistant
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
16 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Early Results on WackoPicko
Experiment on Multi-Step XSS testing:
I
I
1 test purpose
2 abstract test cases:
I
I
I
210 test executions:
I
I
login input
comment input
105 variants retrieved from OWASP’s XSS Cheat Sheet
Results:
I
I
All failed for login input
85 Successes / 20 fails for comment input
Concordant with our manual experiments.
⇒ 0 false positive, 0 false negative
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
17 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Early Results on a real application
Application Under Test presentation:
I Virtual Learning Environment
I Highly used by french learning academies (> 90%)
I More than 15000 users
Experiment on Multi-Step XSS testing:
I 1 test purpose
I 11 abstract test cases
I 1155 test executions:
I
I
I
105 variants retrieved from OWASP’s XSS Cheat Sheet
16 steps per test case
8 steps between injection and observation
Concordant with our manual experiments.
⇒ 0 false positive, 0 false negative
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
18 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Discussion
MBVT appears as an accurate and precise technique.
It also has its limitations, inherited from MBT:
I
Needed effort to provide Models
I
Needed effort to design Adaptation
Potential solutions:
I
Use of a behavioral crawler to infer most parts of models
I
Use of User traces to complement the results of the crawler
I
Identify the reusability capacity of each artifact
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
19 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Discussion
MBVT does not suit every vulnerability type.
⇒ MBVT Scope based on OWASP TOP 10 2013:
A1 - Injection
A2 - Broken Authentication and Session Management
Legend
A3 - Cross-Site Scripting (XSS)
Done
A4 - Insecure Direct Object References
Doable
A5 - Security Misconfiguration
Out of scope
A6 - Sensitive Data Exposure
Under study
A7 - Missing Function Level Access Control
A8 - Cross-Site Request Forgery (CSRF)
A9 - Using Components with Known Vulnerabilities
A10 - Unvalidated Redirects and Forwards
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
20 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Conclusion and Future Works
MBVT is a novel technique for Dynamic Application Security Testing.
Goal
To improve the precision
and accuracy of
vulnerability testing.
Limitations
Needed effort to provide
models and design
adaptation layer.
First Approach
I
vTP into Test purposes
I
Modeling of the SUT
I
Test cases fed with a vectors battery
Work in Progress
Model inference, User traces, generic
artefacts, test purposes extensions,
real-life applications experiments.
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
21 / 22
Introduction SoTA MBVT Process Discussion Conclusion
Thank you for your attention
Source - http://model-based-testing.info
A. Vernotte et al.
MBVT for Web Applications - SecTest’13
22 / 22