Bugs, au Naturale.

Bugs, au Naturale.
%
100 ral
u
Nat
%
100 ral
u
Nat
Premkumar Devanbu
DECAL Laboratory
University of California, Davis
public class FunctionCall {
public static void funct1 () {
System.out.println ("Inside funct1");
}
public static void main (String[] args) {
int val;
System.out.println ("Inside main");
funct1();
System.out.println ("About to call funct2");
val = funct2(8);
System.out.println ("funct2 returned a value of " + val);
System.out.println ("About to call funct2 again");
val = funct2(-3);
System.out.println ("funct2 returned a value of " + val);
}
public static int funct2 (int param) {
System.out.println ("Inside funct2 with param " + param);
return param * 2;
}
}
%
100 ral
u
Nat
ICSE 2012, “On the Naturalness of Software”
Hmmmm….
Tiger, Tiger,
Burning Bright…
Meanwhile, back
in Redmond..
(or Bangalore,
or Shangahi,
or Sunnyvale..)
TIGER!! RUN!!!
Why is your
feature
behind
Schedule?
Code, Code,
Code, Code,
Code…
On the Uniqueness of Code (FSE 2010)
“Let us change our traditional attitude to the construction of
programs: Instead of imagining that our main task is to instruct a
computer what to do, let us concentrate rather on explaining to
human beings what we want a computer to do...
First,
Some Differences
Cross-Entropy per token
Brown
Java
The Skeptic asks..
10
Is it just that C, Java, Python... are syntactically
simpler than English?
7.5
5
2.5
0
1-gram
2-gram
3-gram
4-gram
5-gram
Is buggy code odd?
Buggy vs. Not.
Title
0.7
0.61
10
5
0
non_buggy
buggy
fixed
Effect Size (Cohen’s D)
Line Entropy
15
0.5
0.53
0.43
0.36
0.4
0.29
0.25
0.2
0.0
0.18
0
17
33
50
0.18
67
Defective Line Count
83
100
Defective line count
Title
90
Percent of total bugs
72
67.5
10
60
56
45
Line Entropy
15
81
44
35
5
0
24
22.5
15
0
0
17
33
50
67
83
low
duration
medium
duration
high
duration
100
Defective Line Count
Does it work?
How to tell?
• Problem: Line-level!
• Cost-sensitive measures more suitable.
• Comparable to static analysis warnings!
…measured using Cost-effectiveness curve.
Findings Summary
• More cost-effective than logistic regression
at 5% inspection budget, but not at 20%.
• Cost-effectiveness similar to FindBugs and
PMD.
• Entropy-based ordering improves costeffectiveness of PMD & FindBugs