Interactive Property Checking at Runtime with a - Check-exec

Master of Science in Informatics at Grenoble
Master Mathématiques Informatique - spécialité Informatique
option PDES
Interactive Property Checking at
Runtime with a Debugger
Raphaël Jakse
23 June 2016
Research project performed at LIG / INRIA
Under the supervision of:
Yliès Falcone (LIG), Jean-François Méhaut (LIG), Kevin Pouget (LIG)
Defended before a jury composed of:
Noel de Palma
Martin Heusse
Thomas Ropars
Arnaud Legrand
Olivier Gruber
External expert: Albert Cohen
June
2016
Abstract
Monitoring is the study of a system at runtime, looking for input and
output events in order to discover, check or enforce behavioral properties.
Debugging is the study of a system at runtime in order to discover and
understand its bugs and fix them, inspecting interactively its internal state.
In this work, we combine monitoring and debugging. We define an efficient and convenient way to check behavioral properties automatically on a
program using a debugger. We aim at helping bug discovery while keeping
the classical debugging techniques and interactiveness, which allow understanding and fixing bugs.
Résumé
Le monitoring est l’étude d’un système pendant son exécution, en surveillant les évènements qui y entrent et qui en sortent, afin de découvrir,
vérifier ou pour faire respecter des propriétés sur son comportement. Le débogage est l’étude d’un système pendant son exécution afin de trouver et
comprendre ses dysfonctionnements dans le but de les corriger, en inspectant son état interne, de manière interactive.
Dans ce papier, nous combinons le monitoring et le débogage. Nous
définissons un moyen efficace et pratique de vérifier automatiquement des
propriétés sur le comportement d’un programme à l’aide d’un débogueur.
Notre but est d’aider à détecter des anomalies dans son code, en conservant
les techniques et le caractère interactif du débogage classique permettant de
comprendre et corriger les bugs.
Contents
Abstract
i
Résumé
i
1 Introduction
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Overview of our Approach . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Organization of this Report . . . . . . . . . . . . . . . . . . . . . . . .
1
1
2
4
2 Background
2.1 Existing Methods for Finding Bugs . . . . . . . . . . . . . . . . . . . .
2.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
5
7
3 Joint Execution of the Debugger, the Monitor and the Program
3.1 Configurations of the System . . . . . . . . . . . . . . . . .
3.2 Description of Events . . . . . . . . . . . . . . . . . . . . .
3.3 Property Model . . . . . . . . . . . . . . . . . . . . . . . .
3.4 Evolution of Configurations . . . . . . . . . . . . . . . . . .
3.5 Semantic rules . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
9
9
12
13
15
18
.
.
.
.
.
27
27
28
30
32
33
5 Related Work
5.1 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
37
4 Implementation and Evaluation
4.1 Check-exec: a GDB extension
4.2 Syntax of Properties . . . . . .
4.3 Using Check-exec . . . . . . .
4.4 Architecture of Check-exec . .
4.5 Evaluation . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5.2
Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
38
6 Conclusion and Future Work
6.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
39
40
A check-exec Command List
43
B Check-exec’s Monitor Interface
45
C Executing an Example of a Program
49
D Proof (sketch): the execution of a program is not affected
53
Bibliography
57
iv
1
Introduction
1.1
Motivation
When developing a software, fixing bugs as early as possible is important. This can
prove to be difficult. A way to do this consists in observing a bad behavior and start a
debugging session to find the cause. A debugging session generally consists in repeating the following steps: execute the program in a debugger, set breakpoints before the
expected cause of the bug, find the point in the execution where it starts being erratic
and inspect the internal state (callstack, values of variables) to determine the cause of
the problem. This process can be tedious and prone to a lot of trials and errors. Besides,
a bug does not systematically lead to a crash, it can remain undetected during a whole
development cycle. Therefore, we need methods and tools to ease bug detection.
A difference between the specification of the program and an actual execution is an
anomaly which has to be fixed. This report presents a method to ease failure detection
by automating some tedious aspects of a debugging session by making some checks systematic. More specifically, our idea is to check properties linked to the specification of
the program or the API (Application Programming Interface) provided by the libraries
it uses.
Our approach is inspired by monitoring. In this domain, the studied system is viewed
as a black box. What is observed is the sequence of events it produces and receives.
Considering the execution from this point of view can ease the expression of properties
on the expected runtime behavior of the program. When a property is violated, the
developer can be warned and is able to look at the internal state of the program using
the debugger to understand what happened.
Our approach is meant to be as little intrusive as possible in term of workflow for the
developer. It integrates well to the developer’s toolbox, which may ease its adoption. It
also takes advantage of the infrastructure already provided by the debugger.
Part of this work will presented at the national conference COMPAS 2016 [22]. A
new paper will be submitted to an international conference later.
1.2
Overview of our Approach
Here we summarize the main ideas and the general principles of our approach.
1.2.1
Execution Model of the Program
When debugging, the execution is seen as a sequence of program states that the developer inspects step by step using a debugger in order to understand the origin and the
cause of a misbehavior. The state of the program can be modified at runtime. This
lets the developer try hypotheses on the bug without having to recompile and rerun the
whole program, which would be time consuming. The bug is also not guaranteed to
show up in the new execution.
When monitoring, the execution is abstracted into a sequence of events of program
state updates. The main goal of monitoring is to detect misbehaviors of a system considered as a black box: the set of its internal behaviors is not accessible and its internal state
cannot be altered. Information on the internal state can be retrieved by instrumenting
the execution of the program. Analyzing the execution trace can be done postmortem.
This abstraction of the execution can ease the expression of properties on the program’s
behavior.
1.2.2
Methodology
Our approach combines ideas from debugging and monitoring as follows. The developer
describes properties to verify. These properties are checked during the execution of the
program following the trace in real time.
Decisions can be taken according to the current state of the evaluation of the property. Our approach can be used for instance as follows: as soon as a property is violated,
the execution is suspended to let the developer inspect and debug the program in the
usual way.
We implemented this approach by extending the debugger, making it capable of
running monitors to check properties during the execution and by making this evaluation
capable of altering the execution. A monitor is a device (a piece of hardware or software)
that evaluates a property against an execution trace, giving a verdict upon the reception
of each event of the trace. The verdict corresponding to the last event of the execution
trace is called the final verdict [15]. In Chapter 4, we give details on the implementation
of our proof-of-concept extension for GDB.
1.2.3
Model of Property and Instrumentation
Properties can be expressed on the whole set of events that can be retrieved from the
debugger. Events can be parameterized, that is, values can be linked to events. For
2
Slice on q
Init
N:=0, Max:=0
If N < Max : queue_push(q) / N := N+1
queue_new(q, size)
Max := size – 1
queue_ready
If N ≥ Max :
queue_push(q)
If N ≤ 0 :
queue_pop(q)
sink
If N > 0 : queue_pop(q) / N := N-1
Figure 1.1: Property on a queue overflow in a producer-consumer program described in
Subsection 4.5.1.
instance, a function call is an event parameterized with values of arguments passed
during this call, as well as values that are accessible at this time (global variables for
example). Events can be monitored by setting breakpoints. Thus, when the evaluation
of a property requires monitoring calls to a particular function, a breakpoint is set on
this function. When it is reached during the execution, the relevant monitor updates its
state and can let the execution resume or let the developer interact with the debugger
if the property has been violated. When an event does not affect the evaluation of any
property anymore, the breakpoint becomes useless, it is then unset: the instrumentation
is dynamic.
We describe properties in a model based on finite-state machines. It is composed of
states, transitions and a memory and it recognizes sequences of events. Transitions have
guards that are expressed in function of event parameters and the memory. The memory
can be updated when taking a transition.
The execution trace can be sliced, so that a property is checked against parts of the
trace concerning specific instances of objects (e.g. opened files) instead of the global
state of the program [6]. In this case, a monitor does not correspond to a single finite
state machine, but to a set of finite state machines, one for each particular instance of an
object.
An example of property is given in Figure 1.1. We define our model of property
more precisely in Chapter 3.
1.2.4
Scenarios
A scenario describes what should be done given the evaluation of the property. An example of a scenario is: when the verdict given by a monitor becomes false, that is, when
the evaluation of a property enters a non-accepting state, the execution is suspended.
Scenarios allow binding actions to events generated by the monitor. Examples of these
events are “the state X has been entered”, “the state X has been left”, “an accepting state
has been entered”, “a non-accepting state has been left”. These actions can affect the
state of the debugger or the program.
Scenarios are defined more precisely in Chapter 3.
1.3
Organization of this Report
In Chapter 2, we present the state of the art and the building blocks behind our approach.
In Chapter 3, we gather these building blocks and describe our approach more precisely.
In Chapter 4, we present our proof-of-concept implementation of this approach, checkexec, and we use it to evaluate our approach. In Chapter 6, we conclude by presenting
our future works.
4
2
Background
2.1
Existing Methods for Finding Bugs
There are several families of techniques for finding bugs. Three of them are static analysis, testing and runtime verification. In this chapter, we briefly present several techniques from these three families.
2.1.1
Testing
Manual testing. Beta testing is a widespread way of testing. Most obvious bugs can
easily be spotted this way during the development of the software. Modifications to the
code are manually tested, possibly by a team responsible for testing the software [21].
Bugs are also spotted by final users of the software, which, depending on the development model, the kind of software and the severity of the bug, is more or less undesirable.
Automatic testing. Automatic testing through unit tests can be used to ensure that
the already-fixed bugs do not show again, to limit regressions and to check whether the
code is correct for a restricted set of inputs [20]. Some research efforts have been carried
out on the automatic generation of unit tests. For instance, [7] defines a way to generate
test oracles from formal specifications of the expected behavior of a Java method or
class.
Interactive debugging. In interactive debugging, the program is seen as a white
box. At any time, its internal state can be inspected using the debugger, including the
call stack and values of variables. The execution can be affected: variables can be
edited, functions can be called, the execution can be restored from a previous state.
This lets the developer test hypotheses without having to modify the code, recompile
and execute the program. However, the execution is seen at a low level (assembly ode,
often mapped to the source code) and does not really address bug discovery: usually,
when debugging, one already knows there is a bug and uses tools to try to understand it.
The main debuggers are GDB, LLDB and the Visual Studio debugger. GDB is a crossplatform free debugger from the Free Software Foundation. LLDB the cross-platform
free debugger of the LLVM project, started by the University of Illinois, now backed
by various firms like Apple and Google. The Visual Studio debugger is Microsoft’s
debugger.
Reverse debugging [14, 17, 34] is another debugging technique. When the developer
is looking for a bug, instead of running the program in a debugger, (s)he runs it normally
and records its execution. If the bug is encountered, the execution can be replayed in
a debugger, and reversing the execution is also possible. It also allows replaying and
debugging the record deterministically. This guarantees that the behavior of the program
remains unchanged and the bug happens again. UndoDB and rr are GDB-based tools
allowing record and replay and reverse debugging with a small overhead.
2.1.2
Static analysis and abstract interpretation
In static analysis and abstract interpretation [11], the source code of the software is
analyzed without being run. The goal is to find programming errors and chunks of
code that can cause maintenance difficulties. Maintenance difficulties raise the risk of
introducing bugs during subsequent modifications. Properties can also be proven over
the software’s behavior. Unfortunately, theses approaches have limitations: they can
be really slow, limited to certain classes of bugs or properties, produce false positives
and false negatives. While static analysis and abstract interpretation can provide certain
guarantees by proving properties over the program, proving correction of a software by
static analysis is undecidable in general [26].
Model checking. Model checking is an automatic verification technique for finitestate reactive systems. Model checking consists in checking that a model of the system
verifies temporal properties [8].
2.1.3
Runtime verification and runtime analysis
Valgrind. Valgrind [33] is a tool to detect illegal memory accesses, cache-related
performance and memory consumption problems and can interface with GDB. Valgrind
instruments programs by dynamically adding instrumentation to the machine code of
the inspected program at runtime and by running it on a simulated CPU.
Monitoring. In monitoring [12, 18, 27, 35], the program is seen as a black box.
The execution is rather seen as a sequence of input and output events. This trace can
be analyzed offline (i.e. after the termination of the program) as well as online (i.e.
during the execution) and constitutes a convenient abstraction one which it is possible
6
to express runtime properties. JavaMOP [23] is a tool that allows runtime property
checking on Java programs. In monitoring, there are different ways of instrumenting,
like aspects [25, 24] and code injection.
In monitoring, the execution of the program can also be affected, but only from the
outside: this can be done by modifying the sequence of input events. The sequence of
output events can also be edited to make it comply with some properties [16]. This,
however, does not allow to understand an incorrect behavior and does not actually fix
the program.
2.2
Contribution
Theses approaches to finding bugs, as well as other approaches not covered in this introduction, have their own sets of drawbacks and benefits. This makes them suitable for
discovering different sets of bugs in different situations: some classes of bugs are more
easily avoided by analyzing the software statically, others are more easily and rapidly
spotted by testing the software directly. The relevance of each technique also depends
on the point at which is life cycle of the development of the application.
There are, however, bugs that are not discovered or understood easily or efficiently
by aforementioned methods: as said above, static analysis cannot find every bug in all
softwares and some bugs are not apparent in any obvious way. When a malfunction
shows during a test, its cause is not always easily understandable as the programming
error at its root could be caused earlier during the execution of the software (e.g. a
pointer that should contain a valid address instead of a null or a random address).
Our approach combines and takes its roots from debugging and monitoring. Both
domains aim at program correctness in two different ways. They are complementary:
monitoring helps bug detection while debugging helps understanding and fixing bugs
when one already knows that there is a bug.
Our idea is to get the best of both worlds to ease bug discovery and understanding
by leveraging the possibility to check runtime properties using a debugger. Our general
long-term objective is that during development of a software, systematically checking
properties while running the software helps finding, understanding and fixing bugs and
regressions.
3
Joint Execution of the Debugger, the
Monitor and the Program
Our approach relies on the joint execution of different components: the program, the debugger, the monitor and the scenario. The execution of these components is concurrent.
In this chapter, we describe the system composed of these components as a Labeled
Transition System (LTS). We first present its configurations and their components individually in (Section 3.1). Events play the role of symbols of the LTS. We define the
notion of events in Section 3.2. In Section 3.3, we describe the model used to describe
properties, based on an extension of finite-state machines. We then give semantics rules
describing the joint execution (Section 3.4).
Semantics rules provide a rigorous definition of the composition of the entities and
facilitate the reasoning on their properties.
3.1
Configurations of the System
The composition of the whole system is represented in Figure 3.1. Its set of configurations is Γ = (Mem p × Address) × (Qd × Breakpoints × Checkpoints) × (Qm × Memm ×
Mems ). Its initial configuration is γ0 = ((m p,0 , pc0 ), (I, ε, ε), (init, mm,0 , ms,0 ).
The Program
The Debugger
The Monitor
Memp
Address
Qd
Breakpoints
Checkpoints
Qm
Memp
Mems
mp
pc
qd
B
C
qm
mm
ms
Debugger's state
Debugger's memory
breakpoints
Debugger's memory
checkpoints
Monitor's state
Monitor's memory
environment
Scenario's memory
environment
Program's memory
Figure 3.1: The elements of the configuration of the whole system
Passive State:
Program
executes
without
interruption
CTRL+C
Program terminates
Program is suspended
Other commands
continue
run
Interactive State:
Waiting for users’
Input.
Program execution
Is suspended
Figure 3.2: The states of the debugger
In this section, we describe and explain each component of this configuration individually.
The Debugger. A configuration of the debugger is a tuple (qd , B, C ). qd is the
current state of the debugger, that is either I (interactive mode, in which the user can issue commands in the debugger) or P (passive mode, in which the program runs without
interruption). See Figure 3.2 for a representation of the two possible states of the debugger. B is the sequence of breakpoints. A breakpoint is a tuple (addr, instr, isUserBP)
where addr is the address of the breakpoint in the memory of the program, instr is the
instruction to restore when the breakpoint is removed and isUserBP a boolean equal to
true if the breakpoint was set by the user, false if it was set by the monitor.
C ∈ (Mem p × Address × Qm × Memm )∗ is the sequence of saved checkpoints. A
checkpoint is a snapshot of the whole internal state of the program during its execution
that can be restored (i.e, the checkpoint can be used to drop the current internal state
of the program and make the execution continue from the snapshot). A checkpoint
is a tuple (m p , pc, qm , mm ) where m p is the memory of the program at the moment of
the creation of the checkpoint, pc is the current program counter, qm is the state of the
monitor and mm the memory of the monitor at the time of the checkpoint. See Example 1
for an illustration of a checkpoint.
Example 1 Let P be the program given by the following source code:
a := 0
b := 1
a := a + b
A checkpoint when the third line is about to be executed is a tuple (m p , pc, qm , mm )
where m p = [a 7→ 0, b 7→ 1], describing the internal state of the program P , pc =
<the address of the third line>, the address of the instruction to be executed, qm and
mm are the state of the monitor and its memory when the checkpoint is set.
10
The Program. A configuration of the program is a tuple (m p , pc) ∈ Mem p ×Address.
m p is the memory environment, that maps addresses to machine words and pc is the current program counter (an address in the program’s memory), that is, the address of the
next instruction in the memory. The program is defined by an initial memory environment, a symbol table Sym that maps symbol names to addresses, a start address that
points to the first instruction to run in the memory and a function runInstr representing
the instruction set of the program. The function runInstr takes the program memory and
an address (the current address) and returns a new program memory and a new address.
The exact definition of function runInstr depends on the instruction set of the program.
It is here to abstract away the programming language we use.
The Monitor. A configuration of the monitor is a tuple (qm , mm ) where mm is the
current memory of the monitor and qm to the current state of the property.
The monitor is defined by an initial memory environment mm,0 , its set of states Qm ,
its initial state qm,0 , Σm its set of input events, ∆m : Qm × Memm × Σm → Qm × Memm its
transition function. We denote by enabled(q) the set of uninstantiated events to which a
state is “sensible”, that is, ∀q ∈ Q, e ∈ enabled(Q) ⇐⇒ (q, e) ∈ D(∆m )1 . See Example 2
for an illustration of enabled.
Example 2 For the property of Figure 1.1:
• enabled(Init) = {e f ,be f ore (queue_new)};
• enabled(queue_ready) = {e f ,be f ore (queue_push), e f ,be f ore (queue_pop)};
• enabled(sink) = 0.
/
where e f ,be f ore ( f ) is the formal event corresponding to a call to function f . It is a before
event.
The Scenario. The scenario is defined by an initial memory environment ms,0 and a
list S of actions triggered by monitor updates. The configuration of the scenario is ms ,
its memory. S is a set of tuples, each containing the information on when to trigger an
action and the action itself. More precisely, S ∈ (When × Point × P(Qm ) × A )∗ where
When = {before, after} and Point = {entering, leaving}
The scenario element (ba, lt, Qs , a):
• is triggered before the effective update if ba = before, and after if ba = after.
• if lt = leaving and if an element of Qs is the current state of one slice of the
monitor the current state of the slice changes, the action is run.
1 D( f )
is the domain of definition of f .
• if lt = entering and if the current state of a slice of the monitor changes and the
new current state of the slice belongs to Qs , the action is run.
a is a list of instructions that can be used to control the debugger and the monitor. An
instruction consists in setting a user’s breakpoint, watchpoint, checkpoint, restart from
a checkpoint, update and consult the memory of the scenario ms . It can also consist in a
condition or a loop, allowing the expression of complex behaviors. For instance, it lets
express conditions like: If this is the second time the state x is accessed, run action a.
Otherwise, run action b instead.
In this section, we presented the configurations of the system and we described each
components of these configuration. In the next section, we give the description of events,
that are the symbols making the system transition from one configuration to another.
3.2
Description of Events
An event is an object that has a name, a list of parameters (that can be empty) and possibly a list of instances of these parameters. An event is generated during the execution
of the program whenever something happens. A function call, for instance, generates an
event. The name of this event is the name of the function, its type is Call, its parameters
are the parameters of the function and their instances are the arguments passed during
the call.
Our approach is based on the ability to capture events from the program execution
with the debugger. We therefore need a precise description of events that is suitable in
this context.
Let e be an event. We denote params(e) its formal parameter list, name(e) its name,
type(e) its type, values(e) : Name → Value the function that maps parameters to their
values if e is an instantiated event. isBefore(e) equals true if e is a before event, that
is, the event happened before the instruction that triggered it. isBefore(e) equals false
otherwise (i.e. if e is an after event).
The following are valid event types:
• Call: an event generated by a function call. A before event is triggered before
the first instruction of the function and after the jump to the function body. An
after event fires after the last instruction of the function and before the jump to the
caller.
• ValueWrite: an event generated by an assignment. A before event fires before
the assignment instruction. A after event fires after the assignment instruction.
• ValueRead: an event generated by a variable read. A before event fires before
the read instruction. An after event is triggered after the instruction that reads the
variable.
12
• UpdateExpr: an event generated whenever an expression’s value is changed. A
before event is fired before the update (in which case Ret<e> denotes the value
of the expression before the modification). An after event is fired after (in which
case Ret<e> denotes the new value of the expression).
Let e be a Call event and N the number of parameters of the function.
• arg i, 0 < i < N is a valid parameter name. Its value is the value of the corresponding variable at the time the event fires. The value is not necessarily the value
passed during the call for an after event of type Call but the current value of this
variable.
• v is a valid parameter name, if v is the name of a variable (including a parameter)
defined when the event fires.
• *p, is a valid parameter name if p is a valid parameter name and p refers to a
pointer in the program, ∗p is the value pointed by p.
• &p, is a valid parameter name if p is a valid parameter name, &p is the address of
p in the program.
• Ret<e> is a valid parameter name if e is an after event. Its value is the value
returned by the call.
Formal events are used when writing properties. Arguments are not known at this
time. During the execution, instantiated events are then matched to formal events. An
instantiated event matches a formal event if all its attributes, except values are identical
to the attributes of the formal event
Definition 1 An instantiated event ei matches a formal event e f if name(ei ) = name(e f )
and type(ei ) = type(e f ) and isBefore(ei ) = isBefore(e f ) and params(ei ) = params(e f )
and
Example 3 “Before push(q, 5)” is an instantiated event matching the uninstantiated
event “Before a call to function (type) push (name) that takes a queue and an element
in parameters (list of parameters)”.
3.3
Property Model
Our property model is inspired by finite-state machines, that we extend with the ability
to memorize data using an environment. For the sake of simplicity, we do not take trace
slicing in account in this chapter, though we implemented it and take i
A property is a tuple (Q, Σ, init, env0 , ∆, v) where Q is a set of state, Σ is the set of
formal events involved in this property, env0 ∈ Env is the initial environment (Env =
Names → Values, where Names is the set of variable names and Values is the set of
values that can be stored in a variable), ∆ : Q × Σ × (Env × Env → B) × (Env × Env →
Env) × Q is the transition function and v ∈ [Q → V ] is the function that maps states to
verdicts.
A transition is a tuple (qs , e f , g, upd, qd ) where qs is the start state, e f is the formal
event, g is the guard, upd is the “updater” and qd is the destination state.
The guard g : Env × Env → B takes the environment built from the instantiated
event’s parameters, the environment of the monitor and returns a boolean. If it returns
true, the transition is taken. If it returns false, the transition will not be taken. The updater upd : Env × Env → Env returns an environment from the environment built from
the instantiated event’s parameters the environment of the monitor. This function is used
to update the environment of the property.
When an instantiated event ei is triggered, a transition (qs , e f , g, upd, qd ) is taken if
the current state qm of the property is qs , ei matches e f and g(ei , m p ) = true, where mm
is the current memory of the property. If so, the memory and the state of the property
are updated: m0m = upd(ei , m p ), where m0m denotes the new memory of the property and
qd becomes the new current state.
See Example 4.
Example 4 The property illustrated in Figure 1.1 (if we ignore trace slicing, which is
equivalent to assuming that there is only one instance of a queue) is a tuple (Q, Σ, init, env0 , ∆, v)
where:
• Q = {Init, queue_ready, sink}
• Σ = {e f ,be f ore (queue_new), e f ,be f ore (queue_push), e f ,be f ore (queue_pop)},
• init = Init
• env0 = [N 7→ 0, Max 7→ 0]
• v = [Init 7→ true, queue_ready = true, sink = false]
• The transition function:
∆={
(Init, e f ,be f ore (new), [any 7→ true], ([size], env) 7→ env[max := size − 1], ready),
(ready, e f ,be f ore (push), [[N, Max] 7→ N < Max], (any, env) 7→ env[N+ = 1], ready),
(ready, e f ,be f ore (pop), [[N, Max] 7→ N > 0], (any, env) 7→ env[N− = 1], ready),
(ready, e f ,be f ore (push), [[N, Max] 7→ N >= Max], (any, env) 7→ env, sink),
(ready, e f ,be f ore (pop), [[N, Max] 7→ N <= 0], (any, env) 7→ env, sink),
}
14
The first tuple makes the property transition from Init to ready when (queue_)new
is called. The guard always returns true so the transition is taken unconditionally.
The updater stores the maximum number of elements in the queue in the environment of the monitor. This maximum is computed from the size parameter of the
event new. The two next tuples make the monitor stay on the ready state when it
is correct to (respectively) add or remove elements from the queue. In each case,
the updater updates the number of elements in the queue in the environment of
the monitor. The two last tuple detect that an element is (respectively) added or
removed though the queue is (respectively) full or empty, and makes the property
transition from ready to sink.
3.4
Evolution of Configurations
In this section, we describe how the system evolves, that is, we give rules to build the
sequence of configurations of the system. First, we define several function used in these
rules in Section 3.4.1. Then, we give the semantic rules in Section 3.4.
Limitation. For the sake of simplicity, we limit the set of events the system can handle to events that can be retrieved using breakpoints. In particular, watchpoints, that can
be used to track memory accesses, are not considered.
3.4.1
Functions Used in the Rules
setBP, setInstrBp
Breakpoints are used to monitor function calls. Therefore, functions that set and unset
breakpoints are needed.
We define the following functions: setInstrBp (sets a breakpoint in the memory of
the program), setBP (sets a breakpoint in the memory of the program and keeps track of
it in the memory of the debugger), unsetBP (find a breakpoint by its address and remove
it from the memories of the program and the debugger).
The function setInstrBp replaces the word at the given address in the memory of
the program by a breakpoint instruction (denoted B) in the memory of the program at
the given address. When this instruction is encountered during the execution of the
program, the execution is suspended and the debugger takes control over it.
Formally, setInstrBp(m p , addr) = m0p with m0p such that
(
B
if addr = a
∀addr, m0p [addr] =
m p [addr] otherwise
Writing a breakpoint instruction is not sufficient. We also need to keep the word we
replace in memory, so when the execution is resumed from this breakpoint, the instruction corresponding to this word can be executed. When the breakpoint is removed, the
instruction will replace back the special breakpoint instruction.
Several breakpoints can be set at the same address. For instance, the monitor and the
user might want to set a breakpoint on the same function. We need to keep track of all
of them in order in the structures of the debugger. We therefore use a list to save them.
We define the function setBP that sets a breakpoint and keeps track of it. We also
indicate if the breakpoint was set by the human user or by the monitor, so that breakpoints set by the user do not call the monitor and breakpoint set by the monitor are not
seen by the user.
setBP(m p , B, addr, isUserBP) = (B 0 , m0p ) with m0p = setInstrBp(m p , addr) and B 0 =
(a, m p [addr], isUserBP) · B.
unsetBP
We also define unsetBP, which role is to unset a breakpoint and stop keeping track of it.
unsetBP(m p , B, a, isUserBP) = (m0p , B 0 , b) such that



b is the first breakpoint
( in B matching (addr, _, isUserBP)


instr s.t. (_, instr, _) = b
∀addr, m0p [addr] =

m p [a] otherwise



B 0 = B from which b ∈ B is removed
Remark. The functions setInstrBp, setBP and unsetBP match the behavior of a real
debugger using software breakpoints. A software breakpoint is implemented using a
trap instruction. When several breakpoints are set at a single address, the debugger sets
one trap instruction at this address in the program’s memory and keeps track of all the
breakpoints in its internal structures.
When the current state of the property changes, breakpoints corresponding to events
monitored by the old state must be unset, and breakpoints corresponding to events monitored by the new state must be set.
setBPs, unsetBPs
In Algorithm 1, we define the function setBPs used to set breakpoints needed for the
new state. It returns a new program’s memory and a new list of breakpoints from the
old memory of the program, the new state and the symbol table. The function unsetBPs
is described in a similar way, but unsets breakpoints needed for the old state. Functions NEEDS B REAKPOINT and EVT T OA DDRESS are used by the functions setBPs and
unsetBPs.
16
Algorithm 1 setBPs, unsetBPs
1: function NEEDS B REAKPOINT(e)
2:
return type(e) = FunctionCall
3: end function
4: function EVT T OA DDRESS(e, Sym)
5:
if type(e) = FunctionCall then
6:
return Sym[name(e)]
7:
else
8:
ERROR
9:
end if
10: end function
11: function UNSET BP S(m p , B, qm , Sym)
12:
m0p ← m p
13:
for all e ∈ enabled(qm ) do
14:
if needsBreakpoint(e) then
15:
(m0p , B 0 ) ← UNSET BP(m0p , B 0 , EVT T OA DDRESS(e, Sym), false)
16:
end if
17:
end for
18:
return (m0p , B 0 )
19: end function
20: function SET BP S(m p , B, qm , Sym)
21:
m0p ← m p
22:
for all e ∈ enabled(qm ) do
23:
if needsBreakpoint(e) then
24:
(m0p , B 0 ) ← SET BP(m0p , B 0 , EVT T OA DDRESS(e, Sym), false)
25:
end if
26:
end for
27:
return (m0p , B 0 )
28: end function
removeAllBPs
A checkpoint is snapshot of the program memory and the current program counter.
It must not contain any instruction B. Setting a checkpoint (in rule 3.5.7 in the next
section) thus requires that all breakpoints are removed from the memory before saving
the memory of the program. We define the function removeAllBPs, which iterates over
the list of breakpoints of the debugger and replaces each instruction B by the original
instruction.
• removeAllBPs(m p , ε) = m p and
• ∀B 6= ε, removeAllBPs(m p , B) = removeAllBPs(m0p , B 0 ) where
(
instr if a = addr
B = (addr, instr, b) · B 0 & m0p [a] =
m p [a] otherwise
restoreAllBPs
Conversely, when a checkpoint is restored (Rule 3.5.8), as the memory doesn’t contain
any breakpoints, current breakpoints must be restored in the memory of the program.
We therefore define the function restoreAllBPs which iterates over the list of breakpoints
and sets the instruction B at the relevant addresses.
restoreAllBPs(m p , B) = m0p with
(
BREAK if ∃(addr, instr, b) ∈ B : addr = a
∀a, m0p [a] =
m p [a]
otherwise
3.5
Semantic rules
In this section, we give the semantic rules specifying the precise behavior if the system.
First step of the execution. When starting the execution of the system, the monitor is initialized. That is, breakpoints corresponding to the initial state of the property
are set.
m0p , B 0 = setBPs(m p , B, qm , Sym)
load monitor
((m p , pc), (I, B, C ), (qm , mm , ms )) −−−−−−−→ ((m0p , pc), (I, B 0 , C ), (qm , mm , ms ))
(3.5.1)
When the command load monitor is issued in the debugger in interactive (I) mode,
the current memory of the program m p and the list of breakpoints of the debugger B are
updated. They become respectively m0p and B 0 where (m0p , B 0 ) = setBPs(m p , B, qm , Sym):
18
They contain the breakpoints needed for the current state of the monitor, which is the
initial state.
The execution of the program is interrupted. If the user interrupts the execution of the program (see Figure 3.2), the debugger transitions from passive (P) to
interactive (I). The rest of the configuration does not change.
InterruptPassive
(P, (P, B, C ), M) −−−−−−−−→ (P, (I, B, C ), M)
(3.5.2)
The execution of the program is resumed. This rule is the inverse of the previous one. If the user continues the execution of the program (e.g. by issuing the
command continue, see Figure 3.2), the system transitions from I to P. Again, the rest
of the configuration does not change.
cmd continue
(P, (I, B, C ), M) −−−−−−−→ (P, (P, B, C ), M)
(3.5.3)
Normal execution. If the system is in passive mode and the instruction about to be
executed is not a special instruction from the debugger (e.g. it is not a breakpoint), the
program executes normally as if there were no debugger and no monitor. The program
counter and the program memory are updated by the function runInstr, which runs the
instruction to be executed. The instruction stop ends the execution.
(m0p , pc0 ) = runInstr(m p , pc)
((m p , pc), (P, B, C ), M) → ((m0p , pc0 ), (P, B, C ), M)
m p [pc] ∈
/ DBGInst
m p [pc] 6= stop
(3.5.4)
Given the current memory of the program m p and the current program counter pc,
if the debugger is in passive mode and the current instruction is not a debugger instruction nor the instruction stop, the current instruction is run, resulting in a new program
memory m0p and a new program counter pc0 . the memory of the program and its program
counter are updated in the configuration of the system. Nothing else in the configuration
is affected.
The user sets a breakpoint. When the debugger is in its interactive mode (I),
the user can set a breakpoint. They are two main ways of setting a breakpoint: giving
the address in the program’s memory where the breakpoint should be set, and giving a
symbol name (typically a function name), which is transformed into an address in the
program’s memory using the symbol table.
We first give the semantic rule for setting a breakpoint at an address.
(m0p , B 0 ) = setBP(m p , B, a, true)
break a
((m p , pc), (I, B, C ), M) −−−−→ ((m0p , pc), (I, B 0 , C ), M)
[a ∈ Address]
(3.5.5)
If the user issues a command to set a breakpoint at address a, the function setBP
updates the current memory of the program m p and the list of breakpoints B of the debugger. The resulting memory m0p and breakpoint list B 0 are stored in the configuration
of the system.
The rule for setting a breakpoint on a symbol name is identical, except we need to
look up the symbol in the symbol table.
(m0p , B 0 ) = setBP(m p , Sym(name), B, true)
break name
((m p , pc), (I, B, C ), M) −−−−−−−→ ((m0p , pc), (I, B 0 , C ), M)
(3.5.6)
If the user issues a command to set a breakpoint at symbol name, the function, the
configuration is updated in the same way as the previous rule. The difference is that the
address, not given by the user, is deduced from the given symbol name using the symbol
table Sym, which is part of the definition of the program (like the function runInstr).
The user sets a checkpoint. When the debugger is in interactive mode (I), the
user can set a checkpoint. When the user sets a checkpoint, several things are saved: the
program’s memory, the program’s current address, the property’s state, the property’s
environment.
Breakpoint instructions in the memory of the program must be removed before saving the program’s memory.
n = min N,
(CN = ⊥
Ck
k 6= n
∀k, Ck0 =
(removeAllBPs(m p , B), mm , pc) k = n
checkpoint
(P, (I, B, C ), M) −−−−−−−→ (P, (I, B, C 0 ), M)
(3.5.7)
When the user sets a checkpoint, the smallest identifier n that is not used for a
checkpoint is used. The new checkpoint list, C 0 , is the same as the current checkpoint
list C , except for index n. The checkpoint is stored in Cn0 . In this checkpoint, the current
memory of the program, the current state of the monitor, its memory and the current
program counter are stored. Breakpoints are removed from the memory of the program
before storage in the checkpoint.
The user restarts a checkpoint. When the debugger is in interactive mode (I),
the user can restore a checkpoint. The program’s memory, the current address, the
20
property’s state and the property’s memory are restored. Breakpoints are also reset in
the new memory of the program.
(m p,tmp , pc0 , q0m , m0m ) = Checkn
m0p = restoreAllBPs(m p , B)
restart n
((m p , pc), (I, B, C ), (qm , mm , ms )) −−−−−−→ ((m0p , pc0 ), (I, B 0 , C ), (q0m , m0m , ms ))
(3.5.8)
When the user restarts a checkpoint, the current memory of the program, the current
program counter, the current state of the monitor and its current memory are replaced
by the respective components of the checkpoint. The current breakpoints are set in the
newly restored memory of the program, as the program memory stored in the checkpoint
does not contain any breakpoints. This behavior matches the behavior of GDB and
LLDB.
The program’s execution encounters a debugger’s instruction. When the
program’s execution encounters a special instruction, the debugger has to check if the
instruction corresponds to a user’s breakpoint or to a monitor’s breakpoint. In the first
case, the system transitions to the I state. In the second case, the event is applied.
Remark. In real systems, the trap corresponding to the special instruction is caught by
the operating system. The operating system suspends the execution of the program and
informs the debugger that a trap happened. This behavior is close to what is described
here. Here, we do not model the operating system so traps are not described.
Handling a breakpoint. In Algorithm 2, we define HANDLE BP. If the breakpoints
was set by the user, the state of the system is returned as is, except for the state of the
debugger, which becomes interactive. If the breakpoint belongs to the monitor, breakpoints from the current state are removed, the breakpoint is interpreted as an event, the
event is applied as a before event using the function APPLY E VENT defined in Algorithm 4, the original instruction is run, the event is applied as an after event using the
function APPLY E VENT and the breakpoints for the new current state are set.
Applying an event. The function applyEvent takes an event and updates the state of
the system. It first use the transition function to get the new state and the new memory of
the monitor. It then applies the scenario using the function λSC (applyScenario) defined
in Algorithm 4.
Applying the scenario. The scenario can set and unset breakpoints, save and restore checkpoints and update its own memory. As a consequence, it has the power to
update the whole system. The scenario is applied only if the current state has been
updated (Line 2 of Algorithm 4). Then, for each entry of the scenario, if the event
corresponds to the entry, the action that is attached to it is run with the function runAction (Line 7). The function runAction is not defined precisely here and depends on the
language chosen to implement scenarios.
Algorithm 2 HANDLE BP
1: function HANDLE BP((m p , pc), (P, B, C ), (qm , mm , ms ), (Btmp , (addr, instr, isUserBP)))
2:
if isUserBP then
3:
qd ← I
4:
pc0 ← pc
5:
m0p ← m p
6:
q0d ← qd
7:
B0 ← B
8:
C0 ← C
9:
(q0m , m0m , m0s ) ← (qm , mm , ms )
10:
else
11:
(m p,tmp , B 00 ) ← unsetBPs(m p , B, qm , Sym)
12:
l ← bpToEvts(m p,tmp , pc, mm , Sym)
13:
for all e ∈ l s.t. isBefore(e) do
14:
((mdp , pcd ), (B d , C d ), (qdm , mdm , mds )) ← APPLY E VENT(
15:
(m00p , pc0 ), (B 00 , C ), (q0m , m0m ), S, e
16:
)
17:
end for
18:
(m00p , pc0 ) ← runInstr(m p,tmp , pcd )
19:
for all e ∈ l s.t. not isBefore(e) do
f
20:
((m p , pc f ), (B f , C f ), (q0m , m0m , m0s )) ← APPLY E VENT(
21:
(mdp , pc0 ), (B d , C ), (qdm , mdm ), S, e
22:
)
23:
end for
f
24:
(m0p , B 0 ) ← setBPs(m p , B f , q0m , Sym)
25:
end if
26:
return (qd , pc0 , m0p , q0d , B 0 , C 0 , (q0m , m0m , m0s ))
27: end function
22
(_, Btmp , (addr, instr, isUserBP)) = unsetBP(B, pc, _)
((m0p , pc0 ), (qd , B 0 , C 0 ), M 0 ) = HANDLE BP(
(m p , pc),
(P, B, C ),
M,
(Btmp , (addr, instr, isUserBP))
)
((m p , pc), (P, B, C ), M) → ((m0p , pc0 ), (qd , B 0 , C 0 ), M 0 )
m p [pc] = B
(3.5.9)
The function bpToEvts generates a list of events from a breakpoint. The type, the
name, the parameters and the values of each event from the list is determined by the
current program counter, the symbol table and the memory of the program and is given
by the instruction set of the program. If the program counter is at the first instruction of
a function, a before event of type FunctionCall is generated. If the program counter is
at the last instruction of a function, an after event of type FunctionCall is generated.
The function bpToEvts specifies for each generated event whether the event is a
before event or an after event.
Algorithm 3 getValueInPrgm
1: function GET VALUE I N P RGM (p, m p , pc, Sym)
2:
if p matches (arg i), i ∈ N then
3:
getParamN(i, pc)
4:
else if p matches (∗v) then
5:
m p [getValueInPrgm(v, pc)]
6:
else if p matches (&arg i), i ∈ N then
7:
getAddrParamN(i, pc)
8:
else if p matches (n), n ∈ DSym then
9:
m p [Sym(n)]
10:
else
11:
ERROR
12:
end if
13: end function
The execution of the program is done step by step. If the system is in interactive mode and the user issues the command step, the system executes the instruction
at the current address. If the instruction is a breakpoint, the breakpoint is handled if it
is a monitor breakpoint and the original instruction is executed. If a user breakpoint is
found, it is ignored, but we take care of keeping it in the debugger’s internal structures
Algorithm 4 APPLY E VENT and APPLY S CENARIO
1: function APPLY S CENARIO(S, (m p , pc), (qd , B, C ), (qm , mm , ms ), (q0m , m0m ), evtBefore)
2:
if qm = q0m ∨ S = ε then
3:
return (m p , pc), (qd , B, C ), (qm, mm, ms )
4:
end if
5:
(ba, lt, Qs , a) ← head(S)
6:
if ((ba = before) = evtBe f ore) ∧ ((lt = leaving ∧ qm ∈ Qs ) ∨ (lt =
entering ∧ q0m ∈ Qs )) then
7:
(m0p , pc0 ), (q0d , B 0 , C 0 ), (q0m , m0m , m0s ) ← runAction((m p , pc), (qd , B, C ), a)
8:
else
9:
(m0p , pc0 ), (q0d , B 0 , C 0 ), (q0m , m0m , m0s ) ← (m p , pc), (qd , B, C ), (qm , mm , ms )
10:
end if
11:
return APPLY S CENARIO(tail(S), (m0p , pc0 ), (q0d , B 0 , C 0 ),
12:
(qm , mm , ms ), (q0m , m0m ), eventIsBefore
13:
)
14: end function
15: function APPLY E VENT((m p , pc), (qd , B, C ), (qm , mm ), e)
16:
(q0m , m0m ) ← ∆m ((qm , mm ), e)
17:
return (
18:
(q0m , m0m ),
19:
APPLY S CENARIO (S, (m p , pc), (qd , B, C ), (qm , mm , ms ), (q0m , m0m ), isBe f ore(e))
20:
)
21: end function
while updating the monitor and its breakpoints. In Algorithm 5, we define the function
HANDLE S TEP BP used in the following rule.
((m0p , pc0 ), (I, B f , C f ), (q0m , m0m , m0s )) = HANDLE S TEP BP( (m p , pc),
(I, Btmp , C ),
(qm , mm , ms ))
cmd continue
m p [pc]=B
((m p , pc), (I, B, C ), (qm , mm , ms )) −−−−−−−→ ((m0p , pc0 ), (I, B f , C f ), (q0m , m0m , m0s ))
(3.5.10)
In Appendix C, we illustrate some of these semantic rules by running a toy program
with property checking. In Appendix D, we giving the sketch of a proof of Proposition
to illustrate these semantic rules in another way 1.
Proposition 1 The execution of a program is not affected by the property evaluation (the
monitor).
24
Algorithm 5 HANDLE S TEP BP
function HANDLE S TEP BP((m p , pc), (I, Btmp , C ), (qm , mm , ms ))
(, Btmp , (addr, instr, isUserBP)) ← unsetBP(B, pc)
if isUserBP then
cmd continue
m p [pc]=B
((m p , pc), (I, Btmp , C ), (qm , mm , ms )) −−−−−−−→ (
f
(m0p , pc0 ), (I, Btmp , C f ), (q0m , m0m , m0s )
)
f
B f ← (Btmp , (addr, instr, isUserBP)) · Btmp
else
((m0p , pc0 ), (I, B f , C f ), (q0m , m0m , m0s )) ← HANDLE BP(
(m p , pc),
(I, Btmp , C ),
(qm , mm , ms ),
(Btmp , (addr, instr, isUserBP))
)
end if
end function
4
Implementation and Evaluation
In order to evaluate our approach in terms of usefulness, efficiency, performance), we
developed a proof-of-concept of our approach. In this chapter, we present our implementation. In Section 4.1, we give an brief overview of check-exec. In Section 4.2, we
describe the syntax used in check-exec to write properties. We then explain how to use
check-exec in Section 4.3. We give a brief description of the organization of its source
code in Section 4.4. We conclude the chapter by an evaluation of our approach using
check-exec in Section 4.5.
4.1
Check-exec: a GDB extension
GDB Process
Check-exec
Monitored Process
(inside Python)
rea
set b
ts
kp oin
Monitor 1
event
Monitor n
event
Breakpoint reached
Breakpoint reached
Program execution
Figure 4.1: Architecture of check-exec
Our proof of concept is named check-exec1 and extends GDB using its Python interface. Check-exec provides a graphical and animated view of the property being checked.
The view is optional but can ease understanding the current state of the property and,
as a consequence, the program. Check-exec also lets the developer control the monitors
and access their internal state (property instances, current states, environments).
1 check-exec
can be downloaded at http://check-exec.forge.imag.fr/.
In this work, we focused on programs written in C and C++. However, managing
other programming languages supported by GDB is possible.
check-exec is written in Python and is executed by GDB when it handles breakpoints
in the monitored process that were set by check-exec. When a breakpoint is reached, the
state of the property is updated and the execution is resumed. It can run one or several
monitors, each evaluating instances of one property independently. Each monitor sets
and deletes breakpoints according to the events that are relevant to its current state. A
diagram of the execution of a program with the tool is presented in Figure 4.1.
4.2
Syntax of Properties
Check-exec provides a syntax for writing properties in the automaton model presented
in Section 3.3, with a slight modification to make properties more concise and their
evaluation more efficient by reducing repetitions. Transitions can be written with two
destination states: a success state, used when the guard returns SUCCESS, and a failure
state, used when the guard returns FAILURE. The guard can also return NOT RELEVANT,
which means that the transition will not be taken. An informal grammar is given in
Figure 4.3.
First, the optional keyword slice on gives the list of parameters on which the
execution trace should be sliced for the evaluation of the property.
slice on queue
Then, an optional initialization block initializes the monitor’s memory using a Python
code block:
initialization {
N = 0
}
Then, comes the list of states containing the mandatory state init. Each state has a
name, an optional annotation indicating whether the state is accepting or not, an optional
action name attached to the state and its transitions.
Each transition comprises the monitored event, the event’s parameters used in the
guard, the guard (optional), the success block and the failure block (optional). Success and failure blocks comprise an optional block of Python code, an optional action
name, and the name of a destination state. The guard is a block of Python code which
should not have any side effect and which should return True if the guards succeeds
(SUCCESS), False if the guard fails (FAILURE) and None if the transition should not be
taken (NOT RELEVANT). An example of a property written for check-exec, checking
whether an overflow happens in a multi-threaded producer-consumer program, can be
seen in Figure 4.2.
28
s l i c e on q u e u e
initialization {
N
= 0
max = 0
}
state i n i t accepting {
transition {
success {
e v e n t queue_new ( queue , s i z e : i n t )
max = s i z e − 1
} queue_ready
}
}
s t a t e queue_ready accepting {
transition {
e v e n t q u e u e _ p u s h ( queue , p r o d _ i d ) {
r e t u r n N < max
}
success {
N = N + 1
p r i n t ( " nb elem : "+ s t r (N ) ) ;
} queue_ready
failure {
p r i n t ("% d made %d o v e r f l o w ! "
% ( prod_id , queue ) )
} sink
}
transition {
e v e n t q u e u e _ p o p ( queue , p r o d _ i d ) {
return N > 0
}
success {
N = N − 1
p r i n t ( " nb elem : "+ s t r (N ) ) ;
} queue_ready
f a i l u r e sink
}
}
s t a t e s i n k non−a c c e p t i n g s i n k _ r e a c h e d ( )
Figure 4.2: Description in the format of check-exec of the property represented in Figure 1.1
s l i c e on
param ,
+ ?
initialization {
Python code
} ?
s t a t e state_name non− ? a c c e p t i n g ? action_name ( ) ? {
t r a n s i t i o n {
b e f o r e | a f t e r ? e v e n t event_name ( param , * ) {
Python code returning T r u e F a l s e or None
} ?
success {
Python code
} ? action_name
(
)
?
state_name
?
failure {
Python code
}
?
action_name
(
)
?
state_name
?
} *
} +
Figure 4.3: Informal grammar for the automaton-based property description language
in check-exec
4.3
Using Check-exec
A typical usage session begins by launching gdb and loading check-exec. Here, we
suppose that GDB was configured to load check-exec automatically at the beginning
of the session:
$ gdb ./my-application
Then, the user loads a property. If actions, which are written as Python and referred
to in the property by their name, are needed, they can be loaded at the same time. If
several properties must be loaded, the following line can be repeated for each property.
(gdb) check-exec load-property correct-behavior.prop actions.py
A scenario can also be loaded:
(gdb) check-exec load-scenario default-scenario.sc
30
Figure 4.4: During the execution of the property given in Figure 4.2, the following
graphs can be seen respectively before property’s initialization, on property’s initialization, while the property is verified and when the property becomes falsified. Light
red, red, brown and gray respectively correspond to non accepting state, a current non
accepting state, a transition taken during the last state change and a state which was
current before the last state change. Graphs are automatically drawn using Graphviz
and colors animated during the execution.
Then, the user starts the program execution. It is also possible to display the graph
of the property with the show-graph subcommand (see Figure 4.4).
(gdb) check-exec show-graph
(gdb) check-exec run-with-program
...
[check-exec] Initialization:
[check-exec]
N: 0
[check-exec]Current state: init
[check-exec]
N: 0
queue.c: push!
[check-exec] Current state: init
...
queue.c: push!
[check-exec] GUARD: nb push: 63
[check-exec] Overflow detected!
[check-exec] Current state: sink
monitorinterface.py
De
fine
sanI
nt
e
r
f
ac
et
o
mani
pul
at
emoni
t
or
s
gdbcommands.py
Sour
c
e
df
r
om GDB
graphdisplay.py
graphdisplayer.py
Di
s
pl
a
yani
mat
e
dpr
ope
r
t
i
e
s
monitor.py
De
fine
st
heMoni
t
orc
l
as
s
property.py
De
fine
st
hePr
ope
r
t
y
c
l
as
s
breakpoint.py
Handl
e
sbr
e
akpoi
nt
s
,
i
nt
e
r
f
ac
e
st
oGDB
scenario.py
De
fine
st
heSc
e
nar
i
o
c
l
as
s
property_parser.py scenario_parser.py
Par
s
e
spr
ope
r
t
i
e
s
Par
s
e
ss
c
e
nar
i
os
Figure 4.5: Organization of check-exec’s code
[check-exec]
N: 63
[check-exec: Execution stopped.]
(gdb)
Check-exec provides more fine-tuned commands to handle cases when properties
and actions need to be loaded separately, or when properties and the program need to be
run at different times. A list of commands is given in Appendix A
4.4
Architecture of Check-exec
The organization of the code is pictured in Figure 4.5. The central part of check-exec
is the Monitor class, defined in monitor.py. A Monitor object is instanced for each
property checked at runtime. Files property.py and (resp.) scenario.py define the
classes Property and (resp.) Scenario. When the developer’s properties and scenarios
are read from files, the result is stored in instances of these classes. Theses instances are
used to build monitors.
Trace slicing and checkpointing the property are handled in the class Property.
When a checkpoint is set, the property must be checkpointed at the same time as the
program. In order to make a checkpoint, the whole state of the property needs to be
32
copied. Because of trace slicing, this can be costly: each slice has its own current state
and its environment. In order to allow scenarios to set checkpoints and drop them with a
limited overhead, we implemented checkpointing of the property using copy-on-write:
the state of a slice is copied in the checkpoint only if (and just before) it is updated.
Checkpointing the program is implemented in GDB using a fork, which is also copyon-write.
check-exec and its monitors are controlled with the GDB command line interface
using commands defined in gdbcommands.py. gdbcommands.py defines the interface
between the GDB user and check-exec. Theses commands expose a part of the interface defined in the monitorinterface.py. This interface is meant to remain a stable
interface to access monitors. It does not give access to the internal structures that are
exposed by the Monitor class and that are not relevant for the end user. This interface is
given in Appendix B.
breakpoint.py defines the interface between the monitors and GDB. It defines
methods to handle breakpoints in a way that is not specific to a particular debugger.
graphdisplay.py defines the graphical view of running monitors. If the view is
enabled, check-exec shows the property as a graph using Graphviz. As the current state
of the monitor changes, the graphical view is updated: the current state is shown in
green if it is accepting, in red if it is not accepting. Taken transitions are represented in
brown.
4.5
Evaluation
We evaluated our approach using check-exec in five scenarios in order to measure its
usefulness, its efficiency and its limitations in terms of performance2 .
4.5.1
Multi-Threaded Producer-Consumers
In this section, our goal is to check whether our approach is realistic in terms of usability. We evaluate the approach on this use-case: a developer works on a multi-threaded
application in which a queue is filled and emptied by different threads and a segmentation fault happens in several cases. We wrote a program3 following this use-case,
deliberately introducing a synchronization error, as well as a property (see Figure 1.1)
on the number of additions in a queue in order to detect an overflow. The size of the
queue is a parameter of the event queue_new. The function queue_push adds an element in the queue. A call to this function is awaited by the transition defined at line 15
2A
video and the source codes needed for reproducing these benchmarks are available at http:
//check-exec.forge.imag.fr/eval/.
3 The source code of this program can be found at http://check-exec.forge.imag.fr/eval/
prod_cons.
of Figure 4.2. The program is run with check-exec. The execution stops in the state
sink (defined line 39 of Figure 4.2). In the debugger, we have access to the precise line
in the source code from which the function is called, as well as the complete call stack.
Under certain conditions (that we artificially triggered), a mutex is not locked. After
fixing this, the program behaves properly.
4.5.2
Micro-benchmark
slowdown factor
Overhead of property-checking at runtime with check-exec
Time between two events (milliseconds)
Figure 4.6: Instrumentation overhead with check-exec. The curve check-exec-arg
corresponds to the evaluation of a property which retrieves an argument from calls to
the monitored function.
When writing a property, the developer must be aware of the overhead of the instrumentation in function of the temporal gap between events it handles.
In this second experiment, we evaluate this overhead in function of the frequency of
the events. We wrote a C program calling a NOP function in a loop. In order to measure
the minimal gap between two monitored events for which the overhead is acceptable,
we simulate the gap between these events by a loop of a configurable duration. Results
of this benchmark using a Core i7-3770 @ 3.40 GHz, under Ubuntu 14.04 and Linux
3.13.0, are presented in Figure 4.6. With 0.5 ms between two events, we measured a
slowdown factor of 2. Under 0.5 ms, the overhead can be significant. From 3 ms, the
slowdown is under 20 % and from 10 ms, the slowdown is under 5 %. We noticed
that the overhead is dominated by the process of handling breakpoints when they are
encountered during the execution. The absolute overhead by monitored event, in the
manner of the overhead of an argument retrieval, is constant. We measured the mean
cost of encountering a breakpoint during the execution. We obtained 95 µs on the same
machine and around 300 µs on a slower machine (i3-4030U CPU @ 1.90 GHz).
34
4.5.3
Multimedia Players and Video Games
We evaluate our approach in a more realistic scenario, on widespread multimedia applications: the VLC and MPlayer video players and the SuperTux 2D platform video game.
A property makes the monitor set a breakpoint on the function that draws each frame to
the screen for these applications, respectively ThreadDisplayPicture, update_video and
DrawingContext::do_drawing. For SuperTux, the function is called around 60 times
per second. For the video players, it is called 24 times per second. In each case, the
number of frames per second (FPS) is not affected and the CPU usage remains moderated: we get a overhead of less than 10 % by the GDB process. These results correspond
to our measurements in Section 4.5.2: there is a gap of 16 ms between two function calls
which is executed 60 times per second. Thus, our approach does not lead to a significant
overhead for multimedia applications when the events occur at the same frequency as
frames.
4.5.4
Downloading a File
We measure the overhead of the instrumentation in a I/O intensive context. We download a 862 MiB file with wget from a local HTTP server using the SimpleHTTPServer
class of the Python standard library. We monitor every call to the function read. The
download stream is redirected to /dev/null to avoid write accesses. Without checkexec, it takes 6.0 seconds (133 MiB/s). With check-exec, it takes 27 seconds (31.4 MiB/s),
corresponding to a slowdown factor of 4.5. During this download, wget does 110294
calls to read, corresponding to a mean gap of 0.05 ms between two events. These
results are similar to those obtained and described in Section 4.5.2.
4.5.5
Opening and Closing Files, Iterators
We evaluate the overhead with widespread applications as perceived by the user. We
ensure that all open files are closed with the Dolphin file manager, the NetSurf Web
browser, the Kate text editor and the Gimp image editor. Despite some slowdowns,
caused by frequent disk accesses, they remain usable.
Likewise, we check that no iterator over hash tables of the GLib library (GHashTableIter)
that is invalidated is used. Simplest applications like the Gnome calculator remain usable but strong slowdowns are observed during the evaluation of this property, event
for mere mouse movements. In Chapter 6, we present possible ways to mitigate these
limitations.
4.5.6
Dynamic Instrumentation on a Stack
We measure the effects of the dynamic instrumentation on the performance. A program
adds and removes, alternatively, the first 100 natural integers in a stack and we check
that the integer 42 is taken out of the stack after being added. A first version of this
property takes advantage of the dynamic instrumentation. In this version, the call to
the function which removes an element is watched only when the monitor know that 42
is in the stack. A second version of the property makes the monitor watch every event
unconditionally. In this example, the execution is 2.2 times faster with the first version.
36
5
Related Work
To our knowledge, combining debugging and monitoring has not been done before in
the literature in the way we do in this work. In this chapter, we present existing works
that are related or close to our contribution in each domain. Our contribution has also
been inspired by some of these works.
5.1
Monitoring
Similar work has already been done in Java (e.g. the Jassda framework [3] which use
CSP-like specifications, LARVA [10, 9] and JavaMOP) and other programming languages which main implementation is a virtual machine. Events, in these works, are
captured using introspection and debugging capabilities of these virtual machines.
JavaMOP [5] is the reference implementation by the authors of the trace slicing
method used in the paper. As our implementation, it can be used to check (parameterized) properties at runtime. It applies to Java program and makes use of the monitoring
features of the Java Virtual Machine, contrary to our model, which is based on a debugger and applies to a set of languages which is not restricted to the set of languages based
on the JVM.
Ducassé and Jahier’s work [13] is also based on event tracing. In their proposal, the
debugged (C) program is run in another process than the monitor, which is connected to
a tracer producing execution traces of the monitored program. However, their work is
focused on trace analysis: debugging is not interactive and program’s execution remains
unaffected.
RiTHM [31] is an implementation of a time-triggered monitor, that is, a monitor
which ensures predictable and evenly distributed monitoring overhead at run-time by
handling monitoring at predictable moments in the execution. Monitoring is done by
adding instrumentation generated from properties that must be evaluated to the code of
the C program to monitor.
RV-Monitor [29] is a efficient tool aimed at finding bugs and bad software practice
in Java programs. It has been used to check simultaneously more than 150 properties
written by reading the Java API documentation of common packages.
5.2
Debugging
A debugger has been written to type check program written in C [28]. Their approach is
roughly to tag memory cells with types and break on inconsistency (e.g., when a double
is stored in a cell pointed by a int* pointer).
Valgrind is a framework to instrument binaries and check them for defects. It provides a way to detect memory related defects by a dynamic binary recompilation and
instrumentation process of the software’s machine code and by running it on a simulated
CPU [33]. It provides a more comprehensive detection of memory related defects than
our approach [32], however our approach can be used to detect certain memory leaks
more efficiently by writing a rule which checks that each manually allocated memory
block is freed.
SLAM is a project aimed at checking good API usage. This project is restricted to
system softwares, mainly drivers, on Windows. Unlike our work, it is based on static
analyzing. [1, 2]
Our work brings improvements and new possibilities over existing approaches. Basing our development on a debugger like GDB allows potential support for multiple programming languages and avoid the necessity of learning a completely new tool and the
need to change existing workflow. The visual representation of the property during the
execution can ease program debugging or monitoring. Interactive debugging is also
possible: our tool can check properties during the program’s execution and let the user
inspect the program’s internal state with the debugger when a property breaks.
38
6
Conclusion and Future Work
6.1
Conclusion
This report presents an approach combining monitoring and debugging as two complementary approaches to program correctness.
In monitoring, the program receives and outputs events. Properties on these events
are verified or enforced. Detecting a bug is possible: if a property on the correctness of
the program breaks at runtime, a bug is present in the program. However, a limitation
of monitoring is that it does not provide a way to understand bugs.
In debugging, the program has an internal state that can be studied and modified.
Interactive debugging is a way to understand a bug and find its cause. However, debugging has no support for bug discovery: a programmer uses a debugger at a time when
the bug is already known.
Our approach aims at taking the best of both techniques by seeing the program as a
system that can be monitored to find bugs and, at the same time, as a system that can
be debugged interactively to understand the bugs that were found. When a bug is found
using monitoring, the debugger can be used in a traditional way to understand it.
In this report, we described this approach in details, we provided a theoretical framework that eases the reasoning about the notion of join execution of the monitor, the
debugger and the program. We also presented check-exec, a proof-of-concept implementation of this approach and used this implementation to evaluate our approach. Our
experiments showed that though the property checker can slow down the execution of
the program considerably when events are temporally close to each other, performance
are acceptable beyond a reasonable threshold (Section 4.5.2). We found that this approach is applicable in realistic use-cases with software like video games and video
players 4.5.3. Our current implementation shows limitations in terms of performance
under other use-cases. In the next section, we present ideas to mitigate this issue.
The results presented in this report open several interesting perspectives described
in the next section.
6.2
Future Work
In this section, we present some perspectives opened by this work. We would like to
leverage possibilities offered by watchpoints, diversify supported event types, explore
other ways of instrumenting the execution, explore possibilities opened by checkpoints
and work on the validation of our approach.
Watchpoints. We implemented basic support for watchpoints in check-exec. The
next step is to experiment with them and find out how we could take advantage of them.
This would, for instance, let us monitor read and write accesses to a C array and look
for illegal modifications while iterating over it using an iterator. However, software
watchpoints are slow, according to [19] and our preliminary experiments. We need to
find a way to monitor a large number of addresses efficiently.
Event Types. Our main event type is the function call. A way to make our approach
more powerful is to find and include other kinds of events in our model. System calls
are an example of event type we have not taken in account yet for technical reasons.
They might be of interest for checking properties on drivers or programs dealing with
hardware.
Instrumentation. Handling breakpoints is costly [4]. Other ways of instrumenting
could provide better efficiency and other kinds of events. Instrumentation could be done
using code injection [31, 30]. Injecting a part of the property checker in the monitored
program could improve performance by avoiding round trips between the debugger and
the program: communications between the debugger and the program would be limited
to the bare minimum (for example, when the scenario requires the execution to be suspended to let the user interact with the debugger), while keeping the current flexibility
of the approach.
Checkpoints. We implemented basic support for checkpoints, allowing the developer to restore a previous state of the execution. We also implemented scenarios. The
next step is to take advantage of both checkpoints and scenarios. A possibility is to
define scenarios that detect the right moments to set checkpoints and to possibly restore
them. One could, for instance, write a scenario that sets checkpoints whenever the evaluation of the property enters a state that is next to a non-accepting state. More generally,
the possibility offered by scenarios should be explored more in depth.
Record and replay. Record and replay is a powerful technique for finding bugs.
Once a buggy execution is recorded, the bug can be studied and observed again by
running the recording. The recording can be inspected inside a debugger like a regular
execution. It might be interesting to combine our approach with reverse debugging.
40
Validation of the Approach. For the moment, our approach has been evaluated
on small, simple examples. Next step is to validate it in a more concrete situation, by
finding a real bug in a widespread application with it and show that it indeed eases both
discovery and understanding of the bug. An obstacle to this step is that one needs to be
familiar with the code of the application that is involved in order to write properties that
are of interest to developers. We consider using a database referencing bugs and try to
find one of them with our approach.
Another idea that is yet to be explored is verifying good practice rules and good
API usage at runtime. We think that API designers and library writers could leverage
our approach by providing properties with their APIs and their libraries. This would
provide a means to check that their APIs are used correctly and make their usage safer.
This would also be a means to document these APIs and these libraries.
A
check-exec Command List
check-exec activate
check-exec checkpoint
check-exec
check-exec
check-exec
check-exec
check-exec
checkpoint-restart
cmd-group-begin
cmd-group-end
delete
exec
check-exec get-current
check-exec load-functions
check-exec load-property
check-exec load-scenario
check-exec new
check-exec run
check-exec run-with-program
check-exec set-current
check-exec show-graph
Activates all the commands monitor related commands
Sets a checkpoint for the program and
each managed monitor
Restores a checkpoint
Begins a group of commands
Ends a group of command
Deletes a monitor
Executes an action in the current monitor. Can be used to call methods of the
current monitor’s interface defined in Appendix B
Prints the name of the current monitor
Loads a user defined functions file
Loads a property file and possibly a function file in the given monitor
Loads a property file and possibly a function file in the given monitor
Creates a monitor that will also become
the current monitor
Runs the monitor
Running the monitor and the program at
the same time
Sets the current monitor
Shows the graph of the monitor in a window and animates it at runtime
B
Check-exec’s Monitor Interface
The following is the documentation of the MonitorInterface class. Its methods can be
used progammatically, some of them can be used from GDB’s shell using the check-exec exec
command. For instance, check-exec exec get_current_states prints the current
states of the current monitor.
debugger_shell(self)
Raises an exception, making check-exec interrupt the execution so the
user can interract with the debugger’s shell.
get_current_states(self)
Returns the set of current states of the property.
WARNING: Follow the same precautions as for
the get_env_dict() method.
get_env_dict(self, s)
Returns the Python dictionary representing the environment of the
property.
WARNING: It is unspecified whether a modification to this dictionary
will be reflected in the environment, and whether a modification to
the environment will be reflected in the dictionary.
Call get_env_dict() each time you need to get the most up-to-date
environment.
If needed, use copy() to be sure to keep a static version of the
environment correponding to the moment when this method is called,
and the set_env_dict() and set_env_value() methods to modify the
environment.
get_env_keys(self, s, as_iterator=True)
Returns the keys of the property’s environment, as an iterator or a
list, whether as_iterator is True or False, repectively.
get_env_value(self, s, key)
Returns the value of the given variable in the property’s
environment. Raises if the key is not present in the environment.
get_states(self)
Returns the set of the states of the property.
WARNING: Follow the same precautions as for
the get_env_dict() method.
print_monitor_state(self)
Prints the monitor’s current state.
register_event(self, event_type, callback)
Register a callback for this event type.
Possible events:
- state_changed(new_states)
new_states is the set of the new current states
- transition_taken(transition, point)
transition is the object representing the transition
point is either "success" or "failure"
- event_applied
The event type is given by its name and the parameters passed to the
callback is what is given in parenthesis.
set_current_state(self, state)
Sets the current state of (the root slice of) the property.
set_env_dict(self, s, new_env)
Sets the environment of the property.
WARNING: Please don’t modify the given dictionary after passing it to
this method. This causes undefined behavior. Use copy() if needed when
passing the dictionary to set_env_dict(). new_env is not guaranteed to
be a reference to the actual new dictionary
set_env_value(self, s, key, value)
46
Sets the value of the given variable in the property’s
environment.
set_globals(self, g)
Sets the dictionary in which functions will be found, if needed,
when e.g. calling (un)register_event.
%
set_quiet(self, b=’True’)
Sets the monitor quiet or not.
set_transition_debug_function(self, fun_name=None)
Specifies a user’s function to call whenever a monitored action not
taken in account in the current states of the property is called.
no argument means the default: no user function is called when it
happens.
step_by_step(self, b=’True’)
Set step by step monitor
stop_execution(self)
Raises an exception making the execution of the monitor stop the
program’s and the monitor’s execution.
unregister_event(self, event_type, callback)
Unregister a callback for this event type.
See also register_event.
Some commands are not accessible from check-exec exec: get_env_dict,
set_globals, get_env_keys, stop_execution, set_current_states, debugger_shell,
set_env_dict.
C
Executing an Example of a Program
We run the program in Figure C.1 while checking the property in Figure C.2. We don’t
use any scenario.
The initial state of the system is ((m p,0 , pc0 = 5), (I, ε, ε), (Init, [], ε).
We first load the monitor (Rule 3.5.1). A breakpoint is set on function f because the
there is a transition from the initial state which can be taken if f is called.
((m p,0 , pc0 = 5), (I, ε, ε), (Init, [], ε)
load monitor
−−−−−−−→
((m p,1 , pc0 = 5), (I, [(2, nop, false)], ε), (Init, [], ε)
In the current memory of the program m p,1 , the nop instruction of the program has
1
2
3
4
5
6
7
8
function f ( ) :
nop
function g ( ) :
nop
a := 1 ( t h i s i s the f i r s t
f ()
g()
stop
i n s t r u c t i o n of t h e program )
Figure C.1: A toy program
Init
before call to f
has_f
before call to g
has_g
Figure C.2: A toy property that do not use any memory.
been replaced by the instruction B. The breakpoint has been saved in the memory of the
debugger.
The users makes the program run by itself by issuing the command continue.
Rule 3.5.3 applies. The debugger enters passive mode.
((m p,1 , pc0 = 5), (I, [(2, nop, false)], ε), (Init, [], ε)
cmd continue
−−−−−−−→
((m p,1 , pc0 = 5), (P, [(2, nop, false)], ε), (Init, [], ε)
Then, normal execution happens (Rule 3.5.4). The instruction a := 1 is run.
((m p,1 , pc0 = 5), (P, [(2, nop, false)], ε), (Init, [], ε)
normal execution
−−−−−−−−−→
((m p,2 , pc1 = 6), (P, [(2, nop, false)], ε), (Init, [], ε)
In m p,2 , the variable a takes the value 1. The current program counter is now at the
next line.
Normal execution happens again (Rule 3.5.4). The execution enters function f .
((m p,2 , pc1 = 6), (P, [(2, nop, false)], ε), (Init, [], ε)
normal execution
−−−−−−−−−→
((m p,2 , pc2 = 2), (P, [(2, nop, false)], ε), (Init, [], ε)
The program counter is now at line 2. The original instruction nop has been replaced by a breakpoint. Rule 3.5.9 applies. First, the breakpoint is unset by the function
unsetBP. We have:
Btmp , (addr, instr, isUserBP) = (ε, nop, false)
Then, the function HANDLE BP is called. As the breakpoint was set by the monitor,
it is not a user breakpoint, so the “else” path of the function is taken. All breakpoints
relative to the current state of the monitor are unset from the current memory of the
program. In our case, this refers to the only breakpoint that has been set. The bpToEvts
function generates a before event e which type is FunctionCall, which name is f and
which parameters and values are an empty list. The before version of the event e is
applied, setting the current state of the monitor to has_f. A breakpoint on line 4 is set to
monitor calls to function f. The breakpoint on line 2 does not exist anymore.
The instruction nop is run and makes the program leave the function.
((m p,2 , pc2 = 2), (P, [(2, nop, false)], ε), (Init, [], ε)
monitor’s breakpoint
−−−−−−−−−−−→
((m p,3 , pc3 = 7), (P, [(4, nop, false)], ε), (has_f, [], ε)
The current program counter is now at line 7. Normal execution happens again (Rule 3.5.4).
The execution enters function g.
50
((m p,3 , pc3 = 7), (P, [(4, nop, false)], ε), (has_f, [], ε)
normal execution
−−−−−−−−−→
((m p,4 , pc4 = 4), (P, [(4, nop, false)], ε), (has_f, [], ε)
The program counter is now at line 4. The original instruction nop has been replaced
by a breakpoint. Rule 3.5.9 applies. The breakpoint on line 4 is unset. The current state
of the monitor becomes has_g. No breakpoints are set, because enabled(has_g) = 0.
/
((m p,4 , pc4 = 4), (P, [(4, nop, false)], ε), (has_f, [], ε)
monitor’s breakpoint
−−−−−−−−−−−→
((m p,5 , pc5 = 8), (P, ε, ε), (has_g, [], ε)
The instruction at pc5 (Line 8) is stop, the execution stops and the configuration is
final (no rules apply).
D
Proof (sketch): the execution of a program
is not affected
In this appendix, we give the sketch of a proof of Proposition 1 (the execution of a program is not affected by the property evaluation). We have the program (m p,0 , pc0 ). We
prove that for any execution that terminates, (m p, f , pc f ) = (m p, f 0 , pc f 0 ) where (m p, f , pc f )
is the final state of the program after execution and (m p, f , pc f ) is the final state of the
program after execution with a monitor, for any (qm,0 , mm,0 ), that is, for any monitor.
We assume that the program never accesses (read or write) its own instructions. We
assume that the users only loads the monitor and launches the program but does not otherwise interact with the debugger. We consider an empty scenario (S = ε). We assume
that there exists an instruction stop that is the unique way to end the execution.
Without the monitor. Without monitor, we consider the debugger starts in passive
mode. No monitor is loaded, so rule 3.5.1 does not apply. The execution of the program
will not be interrupted so rule 3.5.2 does not apply. The debugger will never be in
interactive mode, so rules 3.5.5, 3.5.6, 3.5.3, 3.5.7, 3.5.8 and 3.5.10 do not apply. The
debugger will not set a debugger’s instruction in the program so rule 3.5.9 does not
apply.
If m p,0 [pc0 ] = stop, no rules apply. In this case, (m p,0 , pc0 ) = (m p, f , pc f ).
Otherwise, Rule 3.5.4 applies until (m p, f , pc f ) such that m p, f [pc f ] = stop is reached,
which happens because the execution terminates.
(m p,1 ,pc1 )=runInstr(m p,0 ,pc0 )
((m p,0 , pc0 ), (P, ε, ε), M) −−−−−−−−−−−−−−−−−→ ((m p,1 , pc1 ), (P, ε, ε), M)
If m p,1 [pc1 ] = stop then (m p,1 , pc1 ) = (m p, f , pc f ). Else:
runInstr
runInstr
((m p,1 , pc1 ), (P, ε, ε), M) −−−−→ . . . −−−−→ ((m p, f , pc f ), (P, ε, ε), M)
With the monitor. We first load the monitor (with M0 = (init, mm,0 , ms,0 )):
load monitor (Rule 3.5.1)
((m p,0 , pc0 ), (I, ε, ε), M0 ) −−−−−−−−−−−−−−−→ ((m0p,0 , pc0 ), (P, B, ε), M0 )
By definition of the function setBPs, m0p0 [a] is equal to m p0 [a] for any address
a, except for addresses stored in B. for these addresses, m0p0 [a] = B and instructions stored in B are instructions that are at these addresses in m p,0 . We have m0p,0 =
setBPs(m p,0 , ε, init, Sym). We also have m p,0 = unsetBPs(m0p,0 , B, init, Sym).
We continue the execution, that is, the debugger is set in passive mode.
continue
((m0p,0 , pc0 ), (I, B, ε), M0 ) −−−−−→ ((m0p,0 , pc0 ), (P, B, ε), M0 )
No monitor is loaded anymore, so rule 3.5.1 does not apply anymore. The execution
of the program will not be interrupted so rule 3.5.2 does not apply. The debugger will
never be in interactive mode again, so rules 3.5.5, 3.5.6, 3.5.3, 3.5.7, 3.5.8 and 3.5.10
do not apply. Rule 3.5.9 will apply when a breakpoint is reached. Otherwise, Rule 3.5.4
(normal execution) applies.
The system is now in the state ((m0p,k , pck ), (P, B, ε), (qk , mm,k , ms,k )). We prove that
m p,k = unsetBPs(m p p, k0 ) and ∀a ∈ Address, m p,k [a] 6= B.
For k = 0 (initial case), we already have
m p,0 = unsetBPs(m0p,0 , B, init, Sym)
As m0 is the initial memory of the program, no B is present in the memory.
For k 6= 0, we suppose that
m p,k−1 = unsetBPs(m0p,k−1 , B, qk−1 , Sym) ∧ ∀a ∈ Address, m p,k−1 [a] 6= B
(D.0.1)
We want to prove that
m p,k = unsetBPs(m0p,k , B, qk , Sym) ∧ ∀a ∈ Address, m p,k [a] 6= B
Case: m0p,k−1 [pck − 1] = B. In this case, Rule 3.5.9 applies: the breakpoint is handled. The function POP BP gets the instruction that was replaced by the instruction B.
This instruction is not an instruction B. This would be the case if more than one breakpoints were set at the same address. This is impossible because of D.0.1 (unsetBPs
removes breakpoints needed to monitor events of enabled(qk−1 ) and there are no breakpoints in the resulting program’s memory). The function APPLY E VENT removes all the
breakpoints from the program, updates the monitor, runs the instruction returned by the
function POP BP and set breakpoints for the new current state.
Therefore,
m p,k = unsetBPs(m0p,k , B, qk , Sym) does not contain any breakpoint
54
where m0p,k is the result of the application of Rule 3.5.9.
As we consider an empty scenario, The system is not affected in any other way. The
effect on the program’s execution is therefore transparent.
When m0p,k−1 [pck ] = B, the original program’s instruction is run with the same data
as an execution without the monitor.
Case: m0p,k−1 [pck ] 6= B. In this case, Rule 3.5.4 applies: the instruction is executed
normally.
((m0p,k−1 , pck−1 ), (P, B, ε), (qk−1 , mm,k−1 , ms,k−1 )) → ((m0p,k , pck ), (P, B, ε), (qk , mm,k , ms,k ))
According to Rule 3.5.4, qk = qk−1 . Assuming that the program does not modify its
own instructions (so the instruction that have just been executed did not modify the code
of the program), m0p,k contains the same set of addresses where there is an instruction B
as m0p,k−1 . Therefore,
m p,k−1 = unsetBPs(m0p,k−1 , B, qk−1 , Sym) does not contain any breakpoint
implies
m p,k = unsetBPs(m0p,k , B, qk , Sym) does not contain any breakpoint
Supposing that the program does not modify or read its own instructions, we have
(m p,k , pck ) = runInstr(m p,k−1 , pck−1 )
That is, the execution is similar regardless of the presence of breakpoints in the
program’s memory.
In both cases, original instructions get executed in order without effect from the
monitor. The whole execution with a monitor is thus equivalent to the execution without
a monitor.
Bibliography
[1] Thomas Ball, Ella Bounimova, Byron Cook, Vladimir Levin, Jakob Lichtenberg,
Con McGarvey, Bohus Ondrusek, Sriram K. Rajamani, and Abdullah Ustuner.
Thorough static analysis of device drivers. In Proceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006, EuroSys ’06, pages
73–85, New York, NY, USA, 2006. ACM.
[2] Thomas Ball and Sriram K. Rajamani. The slam project: Debugging system software via static analysis. In Proceedings of the 29th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’02, pages 1–3, New
York, NY, USA, 2002. ACM.
[3] Mark Brörkens and Michael Möller. Dynamic event generation for runtime checking using the JDI. Electr. Notes Theor. Comput. Sci., 70(4):21–35, 2002.
[4] Martial Chabot, Kévin Mazet, and Laurence Pierre. Automatic and configurable instrumentation of C programs with temporal assertion checkers. In 13.
ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2015, Austin, TX, USA, September 21-23, 2015, pages 208–
217, 2015.
[5] Feng Chen and Grigore Rosu. Mop: an efficient and generic runtime verification
framework. In Proceedings of the 22nd Annual ACM SIGPLAN Conference on
Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA
2007, October 21-25, 2007, Montreal, Quebec, Canada, pages 569–588, 2007.
[6] Feng Chen and Grigore Rosu. Parametric trace slicing and monitoring. In Tools
and Algorithms for the Construction and Analysis of Systems, 15th International
Conference, TACAS 2009, Held as Part of the Joint European Conferences on
Theory and Practice of Software, ETAPS 2009, York, UK, March 22-29, 2009.
Proceedings, pages 246–261, 2009.
[7] Yoonsik Cheon and Gary T. Leavens. A simple and practical approach to unit
testing: The JML and junit way. In ECOOP 2002 - Object-Oriented Programming,
16th European Conference, Malaga, Spain, June 10-14, 2002, Proceedings, pages
231–255, 2002.
[8] Edmund M. Clarke, Orna Grumberg, and Doron A. Peled. Model checking. MIT
Press, 2001.
[9] Christian Colombo, Adrian Francalanza, and Rudolph Gatt. Elarva: A monitoring
tool for erlang. In Sarfraz Khurshid and Koushik Sen, editors, Runtime Verification
- Second International Conference, RV 2011, San Francisco, CA, USA, September
27-30, 2011, Revised Selected Papers, volume 7186 of Lecture Notes in Computer
Science, pages 370–374. Springer, 2011.
[10] Christian Colombo, Gordon J. Pace, and Gerardo Schneider. LARVA — safer
monitoring of real-time Java programs (tool paper). In Seventh IEEE International
Conference on Software Engineering and Formal Methods, SEFM 2009, Hanoi,
Vietnam, 23-27 November 2009, pages 33–37, 2009.
[11] Patrick Cousot and Radhia Cousot. Abstract interpretation: A unified lattice model
for static analysis of programs by construction or approximation of fixpoints. In
Conference Record of the Fourth ACM Symposium on Principles of Programming
Languages, Los Angeles, California, USA, January 1977, pages 238–252, 1977.
[12] Philip Daian, Yliès Falcone, Patrick O’Neil Meredith, Traian-Florin Serbanuta,
Shin’ichi Shiriashi, Akihito Iwai, and Grigore Rosu. Rv-android: Efficient parametric android runtime verification, a brief tutorial. In Ezio Bartocci and Rupak
Majumdar, editors, Runtime Verification - 6th International Conference, RV 2015
Vienna, Austria, September 22-25, 2015. Proceedings, volume 9333 of Lecture
Notes in Computer Science, pages 342–357. Springer, 2015.
[13] Mireille Ducassé and Erwan Jahier. Efficient automated trace analysis: Examples
with morphine. Electr. Notes Theor. Comput. Sci., 55(2):118–133, 2001.
[14] Jakob Engblom. A review of reverse debugging. In System, Software, SoC and
Silicon Debug Conference (S4D), 2012, pages 1–6. IEEE, 2012.
[15] Yliès Falcone. Étude et mise en œuvre de techniques de validation à l’exécution.
PhD thesis, 2009.
[16] Yliès Falcone. You should better enforce than verify. In Howard Barringer, Yliès
Falcone, Bernd Finkbeiner, Klaus Havelund, Insup Lee, Gordon J. Pace, Grigore Rosu, Oleg Sokolsky, and Nikolai Tillmann, editors, Runtime Verification First International Conference, RV 2010, St. Julians, Malta, November 1-4, 2010.
58
Proceedings, volume 6418 of Lecture Notes in Computer Science, pages 89–105.
Springer, 2010.
[17] Kiril Georgiev and Vania Marangozova-Martin. Mpsoc zoom debugging: A deterministic record-partial replay approach. In 12th IEEE International Conference
on Embedded and Ubiquitous Computing, EUC 2014, Milano, Italy, August 26-28,
2014, pages 73–80, 2014.
[18] Klaus Havelund and Allen Goldberg. Verify your runs. In Bertrand Meyer and
Jim Woodcock, editors, Verified Software: Theories, Tools, Experiments, First
IFIP TC 2/WG 2.3 Conference, VSTTE 2005, Zurich, Switzerland, October 10-13,
2005, Revised Selected Papers and Discussions, volume 4171 of Lecture Notes in
Computer Science, pages 374–383. Springer, 2005.
[19] Laurie J. Hendren, editor. Compiler Construction, 17th International Conference,
CC 2008, Held as Part of the Joint European Conferences on Theory and Practice
of Software, ETAPS 2008, Budapest, Hungary, March 29 - April 6, 2008. Proceedings, volume 4959 of Lecture Notes in Computer Science. Springer, 2008.
[20] Juha Itkonen, Mika V Mäntylä, and Casper Lassenius. Defect detection efficiency:
Test case based vs. exploratory testing. In First International Symposium on Empirical Software Engineering and Measurement, pages 61–70. IEEE, 2007.
[21] Juha Itkonen, Mika V Mantyla, and Casper Lassenius. How do testers do it? an
exploratory study on manual testing practices. In Proceedings of the 3rd International Symposium on Empirical Software Engineering and Measurement, pages
494–497. IEEE Computer Society, 2009.
[22] Raphaël Jakse, Jean-François Méhaut, and Kevin Pouget. Vérification interactive
de propriétés à l’exécution d’un programme avec un débogueur. In Acte de la
conférence COMPAS, 2016.
[23] Dongyun Jin, Patrick O’Neil Meredith, Choonghwan Lee, and Grigore Rosu. Javamop: Efficient parametric runtime monitoring framework. In Martin Glinz, Gail C.
Murphy, and Mauro Pezzè, editors, 34th International Conference on Software
Engineering, ICSE 2012, June 2-9, 2012, Zurich, Switzerland, pages 1427–1430.
IEEE Computer Society, 2012.
[24] Shmuel Katz. Transactions on aspect-oriented software development i. chapter
Aspect Categories and Classes of Temporal Properties, pages 106–134. SpringerVerlag, Berlin, Heidelberg, 2006.
[25] Gregor Kiczales. AspectJ(tm): Aspect-oriented programming in Java. In Mehmet
Aksit, Mira Mezini, and Rainer Unland, editors, Objects, Components, Architectures, Services, and Applications for a Networked World, International Conference
NetObjectDays, NODe 2002, Erfurt, Germany, October 7-10, 2002, Revised Papers, volume 2591 of Lecture Notes in Computer Science. Springer, 2002.
[26] William Landi. Undecidability of static analysis. ACM Letters on Programming
Languages and Systems (LOPLAS), 1(4):323–337, 1992.
[27] Martin Leucker and Christian Schallhart. A brief account of runtime verification.
J. Log. Algebr. Program., 78(5):293–303, 2009.
[28] Alexey Loginov, Suan Hsi Yong, Susan Horwitz, and Thomas W. Reps. Debugging
via run-time type checking. In Fundamental Approaches to Software Engineering,
4th International Conference, FASE 2001 Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2001 Genova, Italy, April
2-6, 2001, Proceedings, pages 217–232, 2001.
[29] Qingzhou Luo, Yi Zhang, Choonghwan Lee, Dongyun Jin, Patrick O’Neil Meredith, Traian-Florin Serbanuta, and Grigore Rosu. Rv-monitor: Efficient parametric
runtime verification with simultaneous properties. In Borzoo Bonakdarpour and
Scott A. Smolka, editors, Runtime Verification - 5th International Conference, RV
2014, Toronto, ON, Canada, September 22-25, 2014. Proceedings, volume 8734
of Lecture Notes in Computer Science, pages 285–300. Springer, 2014.
[30] Reed Milewicz, Rajeshwar Vanka, James Tuck, Daniel Quinlan, and Peter Pirkelbauer. Lightweight runtime checking of C programs with RTC. Computer Languages, Systems & Structures, 45:191–203, 2016.
[31] Samaneh Navabpour, Yogi Joshi, Chun Wah Wallace Wu, Shay Berkovich, Ramy
Medhat, Borzoo Bonakdarpour, and Sebastian Fischmeister. Rithm: a tool for
enabling time-triggered runtime verification for C programs. In Bertrand Meyer,
Luciano Baresi, and Mira Mezini, editors, Joint Meeting of the European Software
Engineering Conference and the ACM SIGSOFT Symposium on the Foundations
of Software Engineering, ESEC/FSE’13, Saint Petersburg, Russian Federation,
August 18-26, 2013, pages 603–606. ACM, 2013.
[32] Nicholas Nethercote and Julian Seward. How to shadow every byte of memory
used by a program. In Chandra Krintz, Steven Hand, and David Tarditi, editors,
Proceedings of the 3rd International Conference on Virtual Execution Environments, VEE 2007, San Diego, California, USA, June 13-15, 2007, pages 65–74.
ACM, 2007.
60
[33] Nicholas Nethercote and Julian Seward. Valgrind: a framework for heavyweight
dynamic binary instrumentation. In Proceedings of the ACM SIGPLAN 2007 Conference on Programming Language Design and Implementation, pages 89–100,
2007.
[34] Jean-François Roos, Luc Courtrai, and Jean-François Méhaut. Execution replay
of parallel programs. In 1993 Euromicro Workshop on Parallel and Distributed
Processing, PDP 1993, Gran Canaria, Spain, 27-29 January 1993, pages 429–
434, 1993.
[35] Oleg Sokolsky, Klaus Havelund, and Insup Lee. Introduction to the special section
on runtime verification. International Journal on Software Tools for Technology
Transfer, 14(3):243–247, 2012.