R: THE GOOD, THE BAD, AND THE UGLY John D. Cook

Transcription

R: THE GOOD, THE BAD, AND THE UGLY John D. Cook
R: THE GOOD, THE BAD,
AND THE UGLY
John D. Cook
M. D. Anderson Cancer Center
Personal background
What is R?
•
•
•
•
•
Open source statistical language
De facto standard for statistical research
Grew out of Bell Labs’ S (1976, 1988)
Influenced by Scheme, Fortran
Quirky, flawed, and an enormous success
No really, what is R?
“You don't have a soul, Doctor.
You are a soul.
You have a body, temporarily.”
Comparison to Excel
Comparison to Emacs
http://batsov.com/articles/2012/05/28/a-true-emacs-knight/
R in data analysis
Languages used in Kaggle.com data analysis competition 2011
Source: http://r4stats.com/popularity
R in bioinformatics (2012)
http://bioinfsurvey.org/analysis/programming_languages/
So what is using R like?
"Using R is a bit akin to smoking.
The beginning is difficult, one may
get headaches and even gag the
first few times.
But in the long run, it becomes
pleasurable and even addictive.
Yet, deep down, for those willing to
be honest, there is something not
fully healthy in it.“
-- Francois Pinard
“… R has a unique and
somewhat prickly syntax
and tends to have a steeper
learning curve than other
languages.”
Drew Conway
John Myles White
So why do statisticians use R?
“The best thing about R is that it was written by statisticians.
The worst thing about R ...”
Bo Cowgill, Google
What are statisticians like?
• Different priorities than software developers
• Different priorities than mathematicians
• Learn bits of R in parallel with statistics
R is a DSL
• To understand a DSL, start with D, not L.
• The alternative to R isn’t Python or C#,
it’s SAS.
• People love their DSL,
and will use it outside of its domain.
Why a statistical DSL?
•
•
•
•
•
Statistical functions easily accessible
Convenient manipulation of tables
Vector operations
Smooth handling of missing data
Patterns for common tasks
Some advantages of R
• Batteries included, one namespace
– Contrast Python + matplotlib + SciPy + IPython
•
•
•
•
•
Designed for interactive data analysis
Easier to program than, e.g., SAS
Open source, interpreted, portable
Succinct notation for querying and filtering
Succinct notation for linear regression
Examples
Set all NA elements of x to 0.
x[ is.na(x) ] <- 0
z <- log( x[y > 7] )
Examples
Fit a linear regression model to w
as a function of x, y, and z,
including a constant term and
all first order interaction terms except xz.
model <- lm(w ~ (x + y + z)^2 – x:z)
Least squares fit to w = a + b x + c y + d z + e xy + f yz
Simple regression
growth
12
10
8
11
6
7
2
3
3
tannin
0
1
2
3
4
5
6
7
8
Regression example
> data <- read.table("example.txt", header=T)
> attach(data)
> names(data)
[1] "growth" "tannin"
> model <- lm( growth ~ tannin )
> summary(model)
...
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 11.7556
1.0408 11.295 9.54e-06 ***
tannin
-1.2167
0.2186 -5.565 0.000846 ***
...
Residual standard error: 1.693 on 7 degrees of freedom
Multiple R-squared: 0.8157,
Adjusted R-squared:
0.7893
F-statistic: 30.97 on 1 and 7 DF, p-value: 0.0008461
Motor Trend metadata
Motor Trend data
Gas mileage
Example from
“R in Action”
by Robert Kabacoff
Code for plot
library(ggplot2)
transmission <- factor(mtcars$am,
levels = c(0, 1),
labels = c("Automatic", "Manual"))
qplot(wt, mpg,
data
=
color
=
shape
=
geom
=
method =
formula =
xlab
=
ylab
=
main
=
mtcars,
transmission,
transmission,
c("point", "smooth"),
"lm",
y ~ x,
"Weight",
"Miles Per Gallon",
"Regression Example")
Language features
•
•
•
•
•
•
Dynamically typed
First-class functions, closures
Objects (two ways!)
Vector-oriented
Pass by value
Everything is nullable (two ways!)
Vectorization example
# generate and store one million random values
x <- rnorm(1e6)
y <- sum(x)
Good R style, bad C style
# save memory by generating one random value at a time
s <- 0
for ( i in 1:1e6 ) s <- s + rnorm(1)
Good C style, bad R style
Some Bad and some Ugly
Speed
Maybe 100x slower than C++,
though it varies greatly.
Tool support
Limited compared to, e.g.,
Visual Studio from 1995.
Safety
Designed for interactive use,
not production.
Hussaini Hanging Bridge (Pakistan)
Misuse
R users often only know R
and use it when inappropriate.
Guide to the Bad and the Ugly
The R Inferno
by Patrick Burns
126 pages
http://www.burns-stat.com/
pages/Tutor/R_inferno.pdf
The book I wish someone would write
s/JavaScript/R/
Photo by David Walsh, http://davidwalsh.name
Lessons from R
• Data analysis is very different from
system programming.
• People will put up with a lot
to get their work done.
• People will use a familiar tool over
a better tool if at all feasible.
Resources
• http://www.r-project.org/
• http://www.johndcook.com/
R_language_for_programmers.html
• “The Art of R Programming”
by Normal Matloff
• @RLangTip

Similar documents