Teaching as an Alternative Scientific

Teaching as an Alternative Scientific Pursuit
Bill Wood
Department of MCD Biology and
Science Education Initiative
University of Colorado, Boulder
SDB Meeting Re-Boot Camp
San Francisco, 7/23/09
How do you view your teaching currently?
E.g.
An unwelcome chore that gets in the way of my
research?
The most fulfilling part of my job as a faculty
member?
etc.
Different views of teaching
A way to find out more about how your students learn
A new area of professional scholarly work?
Two stories . . .
Scientific Teaching
- what is it?
Set clear objectives for instruction
Develop an assay for its effectiveness
Develop methods for achieving objectives
Refine methods based on assayed outcomes
First story:
The Science Education Initiative at U. Colorado, Boulder
Five participating science departments:
Chemistry and Biochemistry
Earth Sciences
Integrative Physiology
MCD Biology
Physics
All strongly research-oriented
All teaching many undergraduates
SEI Goal: improve effectiveness of science teaching
Process
In each department, for each large course:
• Establish specific learning goals through faculty
consensus
• Create validated assessments for measuring
attainment of learning goals
• Create and utilize pedagogically effective materials
and teaching approaches that are:
• aligned with the learning goals
• based on educational research
• improved each year based on assessment results
Wieman's "Holy Trinity"
1. Establish specific learning goals through
faculty consensus
Learning goals (beyond the syllabus):
• specify what students should be able to do by the
end of each course
• guide 60-70% of instruction (remainder discretionary)
• Result: the departmental curriculum is redefined
in terms of specific learning objectives, which can
be assessed by performance-based criteria
Aim for higher Bloom's levels !
Bloom's Levels of Understanding
6. Evaluation: think critically about and defend a position
5. Synthesis: transform, combine ideas to create something new
4. Analysis: break down concepts into parts
3. Application: apply comprehension to unfamiliar situations
2. Comprehension: demonstrate understanding of ideas, concepts
1. Factual Knowledge: remember and recall factual information
Bloom's Levels of Understanding
6. Evaluation: think critically about and defend a position
Judge, Justify, Defend, Criticize, Evaluate
5. Synthesis: transform, combine ideas to create something new
Develop, Create, Propose, Design, Invent
4. Analysis: break down concepts into parts
Compare, Contrast, Distinguish
3. Application: apply comprehension to unfamiliar situations
Apply, Use, Compute, Solve, Predict
2. Comprehension: demonstrate understanding of ideas, concepts
Restate, Explain, Summarize, Interpret, Describe
1. Factual Knowledge: remember and recall factual information
Define, List, State, Name, Cite
Adapted from Allen, D. and Tanner, K., Cell Biol. Educ. 1: 63-67 (2002)
Bloom's Levels of Understanding
6. Evaluation: think critically about and defend a position
What students really need to learn how to do!
5. Synthesis: transform, combine ideas to create something new
4. Analysis: break down concepts into parts
Some, but not many questions on MCAT, GRE exams
3. Application: apply comprehension to unfamiliar situations
2. Comprehension: demonstrate understanding of ideas, concepts
Most questions on introductory biology exams!
1. Factual Knowledge: remember and recall factual information
Example from our introductory genetics course
Syllabus: Patterns of Mendelian inheritance
Pedigree analysis
Modes of inheritance
Learning goals:
After completing this course, students should be able to:
1. Analyze phenotypic data and deduce patterns of inheritance
from family histories.
Course level (one of 9 course-level learning goals):
a) Draw a pedigree based on information in a story problem.
b) Distinguish between dominant, recessive, autosomal, Xlinked, and cytoplasmic modes of inheritance.
c) Calculate the probability that an individual in a pedigree
has a particular genotype.
Topic level (3-5 topic-level goals for each courselevel goal):
(All Bloom's Level 3 or higher)
2. Create validated assessment tools for measuring
attainment of learning goals
These assessments:
• are aligned with learning goals
• validated through student interviews, input from
outside experts, and statistical analysis of
results from large numbers of students
• are administered as pre- and post-tests, to measure
normalized learning gain
<g> = 100 x (post-test - pretest / 100 - pretest)
Now you're in a position to do
Scientific Teaching!
In the literal sense:
Make course changes,
introduce innovative
approaches, and measure
the effects on learning.
3. Create and utilize materials and teaching
approaches proven to be pedagogically effective
All involve Active Learning activities
in addition to or in place of lecturing that:
• are cooperative, involving students working in groups
• are coupled with immediate feedback
• require students to recall, think about, apply, and
verbalize important concepts, rather than simply
record facts for later memorization.
Theory behind why active learning works
Alternative views of education
Transmissionist vs Constructivist
• I know a lot about this topic, and you need to learn it, so I will
transmit my knowledge to you by telling you about it.
• I know a lot about this topic that you need to learn about, and
I also know something about how people learn, so I will create
situations and present challenges for you that will make it easier
for you to efficiently construct knowledge about this topic for
yourself.
Evidence for the constructivist view is
compelling, yet most large courses are still
taught using a transmissionist approach.
Some examples of constructivist activities in class:
• Clicker questions (challenging, with peer discussion)
• Concept mapping
• Problem solving (in groups)
• Analysis of case studies (in groups)
• Analysis and explanation of research data
All can help answer the question, for the student
or the instructor:
How am I doing ?? (formative assessment)
Clickers
Maternal-effect lethal mutants
P0
+/+
F1
m/+
F2
F2 embryo will:
+/+
live
mutagenize
Question: If m is a
strict maternal-effect
recessive mutation:
A) m/m embryo will live.
m/+
live
m/m
B) m/m embryo will die.
?
An example of a clicker question, following a
20-minute lecture explanation of the
physiology behind maternal effects of many
embryonic lethal C. elegans mutations.
Maternal-effect lethal mutants
P0
+/+
F1
m/+
F2
F2 embryo will:
+/+
live
mutagenize
A) m/m embryo will live.
m/+
live
initial individual answers
n=70
Question: If m is a
strict maternal-effect
recessive mutation:
m/m
B) m/m embryo will die.
?
Typical result, every year,
in our developmental
biology course.
What to do, when half the
class doesn’t get it?
Best strategy: Let
students debate the
question and then re-vote.
Maternal-effect lethal mutants
P0
+/+
F1
m/+
F2
F2 embryo will:
+/+
live
mutagenize
A) m/m embryo will live.
m/+
live
initial individual answers
n=70
Question: If m is a
strict maternal-effect
recessive mutation:
m/m
B) m/m embryo will die.
?
after group discussion
Peer instruction works!
E. Mazur, Peer Instruction, A Users
Manual, Prentice-Hall, 1996)
Don't forget to discuss what happened!
interact with
Concept mapping
Transcription
Factors
interact with
RNA
Polymerase
catalyzes
activate or
inhibit
bind to
binds
to the
Transcription
initiated at
Promoter
upstream or
downstream of
Regulatory
elements
Concept map of transcription
Examples of results from the scientific
teaching approach
Item Difficulty Index (P)
P values (mean fraction correct answers) on each of the
25 GCA questions, pre- and post-tests, grouped by
learning goal
LG2
LG1
LG3
LG4
LG5
LG7
LG9
LG6
LG8
Question number
n = 607 students
Pre-test
Post-test
increment
Smith, MK, Wood, WB, Knight, JK (2008) The Genetics Concept Assessment:
a New Concept Inventory for Gauging Student Understanding of Genetics.
CBE-Life Sci. Educ. 7: 422-430.
Number of students
Comparison of student normalized learning gains in
traditional and interactive-engagement courses
traditional
interactive
interactive
% normalized learning gain
Developmental biology,a required course for majors, ~70 junior and senior undergraduates,
taught in Fall '03, Spr. 04,and Spr. 05. From JK Knight and WB Wood. Teaching more by
lecturing less. Cell Biol Educ 4: 298-310 (2005).
Second story:evidence for the value of group work
Why does peer discussion improve student
performance on in-class concept questions?
The story of Prof. Tin Tin Su
Question: Do students learn during the discussion, or
are they simply influenced by their knowledgeable
peers to choose the right answer?
Experiment using isomorphic questions, Q1 and Q2:
Q1
Students vote
individually,
correct answer
and distribution
not revealed.
Peer
discussion
Q1ad
Students re-vote,
correct answer
and distribution
still not revealed.
Smith MK, Wood WB, et al. (2009). Why peer discussion improves student
performance on in-class concept questions. Science 323: 122-124.
Q2
Isomorphic
question:
students vote
individually,
then correct
answers and
distributions
revealed.
Percent Correct
Mean individual improvement from Q1 to
Q2 for 16 isomorphic question pairs
100
90
80
70
60
50
40
30
20
10
0
Q1
Q1ad
Q2
Mean Q2 score is significantly higher than mean Q1 score (16% ± 1%SE)
Data from one of the Colorado majors genetics courses, 350 students
On average, students who corrected their initial
response to Q1 after discussion did much better on
Q2 than students who did not correct their initial
response
All Students
Q1
52%
correct
41%
correct
Q1ad
Q2
48%
incorrect
84%
correct
77%
correct
59%
incorrect
23%
44%
incorrect correct
56%
incorrect
Conclusion:
Most students are learning from peer discussion
But how??
Transmissionist view: the stronger students explain
the correct reasoning to the weaker students, who
therefore now understand it (Mazur).
Constructivist view: in the process of actively
discussing and defending different points of view,
students arrive at a correct understanding by
themselves.
Percent correct
Mean individual improvement from Q1 to Q2
for question pairs of different difficulty
100
90
80
70
60
50
40
30
20
10
0
NG = 50%
Q1
NG = 36%
NG = 47%
Q1ad
Q2
Easy
Medium
Difficult
(5 questions)
(7 questions)
(4 questions)
NG: normalized gain from Q1 to Q2.
Chi-square analysis on responses to each of the
four difficult question pairs
Model: Q1-correct students are randomly distributed among
the participating groups.
All students in these non-naïve groups, and only these
students, answer Q2 correctly.
Observed
Predicted
Observed
Total
correct
on Q1
correct
on Q2
24 (12%)
64
102
203
33.3
<0.01
44 (16%)
114
147
277
15.9
<0.01
50 (18%)
122
141
275
5.1
=0.02
52 (20%)
125
185
258
56.3
<0.01
correct
students
on Q2 participating
χ2
p
Conclusion:
Most students are learning from peer discussion
But how??
Transmissionist view: the stronger students explain
the correct reasoning to the weaker students, who
therefore now understand it (Mazur).
Constructivist view: in the process of actively
discussing and defending different points of view,
students arrive at a correct understanding by
themselves.
Student surveys support the constructivist explanation
Survey question (n=328 responding): When I discuss
clicker questions with my neighbors, having someone in the
group who knows the correct answer is necessary in order
to make the discussion productive (agree/disagree).
47% of students disagreed.
Student surveys support the constructivist explanation
Comments from these students included:
"Often when talking through the questions the group
can figure out the questions without originally knowing
the answer, and the answer almost sticks better that
way because we talked through it instead of just hearing
the answer."
"Discussion is productive when people do not know the
answers because you explore all the options and
eliminate the ones you know can't be correct."
Prof. Tin Tin Su . . .
was senior author on the first classroom research paper to be
published as a Report in Science:
MK Smith, WB Wood, JK Knight, W Adams, C Wieman, and TT Su, Why
peer discussion improves student performance on in-class concept
questions. Science 323: 122-124 (2009).
has changed her attitude toward teaching:
"After teaching the same course for two years in a row, I always used
to get bored with it. Now I can't wait for next fall, because there's
another experiment I want to do!"
is lead PI on a pending CCLI grant application to NSF,
proposing to study the effects of active learning activities on
longer term retention of key concepts in genetics.
Acknowledgements: colleagues in MCD Biology
Jennifer Knight, Ph.D. - SEI Coordinator
Jia Shi - Science Teaching Fellow
Michelle Smith - Science Teaching Fellow
Twelve participating faculty members
The University of Colorado Science Education Initiative
http://www.colorado.edu/sei/