Not Afraid of "Rate My Professors":

What is one burning
question you have about
feedback?
Please write it down and
drop it in the basket at the
front of the room.
Thanks!
Not Afraid of
"Rate My
Professors":
How to Ask Your Students What They
Think About the Course and Use
Feedback to Improve Learning During
the Term
Nanda Dimitrov, Ph.D.
Teaching Support Centre
Centre for Research on Teaching and
Learning in Higher Education
Feedback on
teaching is an
on-going
conversation
between
you and your
students
Feedback Throughout the Course
Check in with students
at least at these key
milestones during the
semester
Before
the
course
End of
the
semester
Plus get feedback on
student learning
throughout the course
6-8
weeks in
First day
of class
3 weeks
in
What forms of feedback have you
tried in your class?
End of the
course
Before the
course
6-8
weeks in
First day
of class
3 weeks
in
Value of midterm feedback
›  “Receiving
feedback from student ratings
administered during the first half of the term
positively related to improving teaching as
measured by student ratings at the end of
term- meta analysis of 17 studies (Cohen
1980)
›  Midterm feedback using the Teacher
Behaviour Inventory leads to improvement of
classroom teaching – significant increase in
ratings of Overall Teaching Effectiveness at
the end of term. (Murray, 2007 ;Murray &
Smith, 1989) Types of feedback on …
Your
teaching
approach
Assessments
The learning
environment
Learning and
learning
activities
Great Resources on Feedback
http://cft.vanderbilt.edu/
guides-sub-pages/studentfeedback/#inclass
Great Resources on Feedback
Brown University
› 
Great Resources on Feedback
Classroom Assessment
Techniques
University of Michigan
›  http://www.crlt.umich.edu/gsis/p4_8
Great Resources on Feedback
http://www.flaguide.org/
Learning Assessment for STEM disciplines
Including
Assessment of
Mathematical
Thinking (Math CATs)
Write down 3 questions you
want to ask your students …
›  Question
1 – About your teaching approach
›  Question 2 – About learning
›  Question 3 – About assessment in your course
REVISE DURING THE DISCUSSION
Start the feedback conversation –
Get to know your audience
›  Before
class online or on Day 1
›  Surveys, index cards, forum discussions
›  How
do you learn best?
›  What types of things hinder your learning?
›  Why are you taking this class?
›  What do you hope to learn?
›  How will you use what we learn together?
›  What has been your most memorable learning
experience at university so far?
›  Is there anything you want me to know about you
as a learner?
Feedback on your teaching approach
Stop Start Continue
›  Owl Polls
›  Polls everywhere, clicker questions
›  Surveymonkey/Fluidsurveys
›  Student ambassadors
›  In class survey
›  Index cards; sticky notes
›  Twitter style feedback
on paper – 140 chrs
›  Critical Incident
Questionnaire (S. Brookfield)
› 
Explain the purpose of
feedback to students
›  Clearly state the purpose and eventual uses of the evalua2on to the class › 
› 
› 
Anonymous Read only by you (instructor) Not graded! ›  Ask for specific, construc2ve feedback ›  Offer students examples of such feedback ›  Summarize feedback for class ›  Respond and make changes Models of Intercultural and Cosmopolitan
Learning(Online) – Graduate Course
Hello everyone, We have been working together for over three weeks. I am asking for your
feedback at this point so that I can adjust the learning activities to best meet your
needs. Please provide feedback on the course in a 3 question surveymonkey
survey.
The survey is outside of owl so that it can be completely anonymous. Please let
me know how the course is going for you so far, what I can do to help you meet
your learning goals make this a great learning experience for you. This feedback method is called STOP - START - CONTINUE. In the survey below
please share what I or we as a group should stop doing, start doing or continue
doing to help make the course a great learning experience. I will try to
incorporate your suggestions within the next week if possible.
You can reach the survey by clicking:
https://www.surveymonkey.com/s/78C88KV
Thank you so much for your feedback.
Stop Start Continue
›  What do you like about this class (what should I con$nue doing)? ›  What could be improved in this class/what else would help you learn (what should I start doing)? ›  What don’t you like about this class (what should I stop doing)? ›  Consider reframing as: › How might I beJer support your learning? Alternative questions for SSC
›  What
should we START doing in the class i.e. what else would help make this a
great learning experience for you?
›  What should we STOP doing - what about
the course learning experience is not
helping you learn?
›  What should we CONTINUE doing in the
course - What aspects of the course are
you learning the most from? Early Semester Feedback
University of
Minnesota,
Twin Cities
Early semester feedback
Please rate on a scale of 1 to 5
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
Your understanding of what is expected of you in the course
The instructor’s clarity in presenting or discussing course material
The instructor’s use of examples or illustration
The instruction’s ability to speak clearly and audibly.
The instructor’s success in getting you interested or involved.
The instructor’s availability to answer questions and provide help.
The instructor’s respect and concern for students
Your comfort asking questions and expressing opinions in class.
Helpfulness of feedback on
Degree to which assessments (exams, quizzes) measure your
knowledge and understanding
Early Semester Feedback Form - University of Minnesota, Twin Cities
Open feedback
›  The
best aspect of the course for me so
far has been….
›  Because….
›  One
thing I would like to suggest to
improve the learning experience is ….
Midterm Feedback- Three Things
›  What
are three important things you have
learned so far?
›  What are three aspects of the class that
have helped your learning so far?
›  What are three things that you wish were
different?
›  Three
Things (Craig Nelson, Indiana
University - Bloomington)
Interpreting Student Feedback
›  Analyze the feedback ›  Look for trends ›  Look for comments you can feasibly act upon ›  Don’t let one student comment ruin your day ›  Tell students how you will incorporate their feedback – and do it! Think_Pair_Share the Question
about Your Teaching Approach
›  3
minutes
›  What do you hope to find out?
›  What question would you ask your class?
I am teaching…
Are they learning?
Classroom Assessment Techniques
CATs are…
§  modest classroom experiments designed to gain instructor’s
valuable feedback on what, how much and how well their
students are learning
§  Also designed to give teachers insights into how students
respond to particular teaching approaches
Characteristics of CATS
Learner Centered
Teacher-Directed
Mutually Beneficial
Formative
Context-Specific
Ongoing
Rooted In Good Practice
CATs give us insight into the
gap between we teach and
what students learn
›  What
areas of the material are they
struggling with?
›  What skills have they mastered?
›  Are they ready for an extra challenge?
›  What else would help them learn?
Examples of CATs
›  The
Minute Paper
›  The Muddiest Point
›  The One Sentence Summary
›  Applications Cards
›  Directed Paraphrasing
›  Exit ticket
›  (Angelo and Cross, 1997)
›  More
examples at
http://www.crlt.umich.edu/gsis/p4_8
CAT Cycle
10. Design a
follow-up CAT
1. Choose the
focus class
2. Focus on
assessable
goal or
question
3. Choose &
plan a CAT
9. Evaluate
CAT’s
effect on
T&L
4. Teach target
lesson
8. Communic.
response
7. Interpret
results;
formulate
response
5. Assess
student
learning
6. Analyze
student
feedback
CATS for STEM disciplines
›  FLAG
website
http://
www.flaguide.org/
›  MATH CATS:
›  Fault
finding and fixing
›  Plausible Estimation (Fermi problems)
›  Creating Measures
›  Convincing and proving
›  Reasoning from evidence
Feedback on a single class
Entry and Exit Tickets (Brown University)
Name one important thing you learned in class today.
›  What did you think was accomplished by the small group
activity we did today?
›  Write/ask one question about today’s content—
something that has left you puzzled.
›  Read this problem… and tell me what your first step
would be in solving it.
›  One of the goals of this class is to have all participants
contribute to the seminar. How well do you think this was
achieved today?
› 
Exit tickets
›  I
used the blackboard extensively today. Was
its organization and content helpful to you in
learning? Why or why not?
›  Which of the readings you did for class today
was most helpful in preparing you for the
lesson? Why?
›  We did a concept map activity in class today. Was this a useful learning activity for you? Why
or why not?
›  Of
what we learned today, what would
you NOT like to have on the exam?
Feedback on an assignment
›  What
did you learn from
this assignment?
›  How did the assignment
help you learn?
›  How could the
assignment be
improved?
›  E.g. value of weekly
reading reflections
Think_Pair_Share the Question
about Learning or Assessment
5 minutes
›  What do you hope to find out?
›  What question would you ask
your class, when and in what
form?
›  Provide feedback on each
other’s questions
› 
Tools to collect feedback
Polls tool in Sakai Owl
›  Owl-
Polls tool (can be anonymous)
Polls Everywhere
Peer observation
›  Participate
in Teaching Squares
›  Ask a colleague to sit in on your class
›  Ask a TA to sit at the back of the class
and comment on how students are
engaged
End of Course
Feedback
Ask the questions you
need to improve the
course for the next time
Example: Final Reflection Forum in an Online course
Please make one final post and share
(1) which learning activities, readings or models you found most
useful in the course and why and
(2) of what we learned, what are you most likely to apply in
your own teaching.
Thank you for your feedback!
References
› 
Cohen, P. A. (1980). Effectiveness of student-rating feedback for improving
college instruction: A meta-analysis of findings. Research in Higher Education,
13(4), 321-341. › 
George, J., & Cowan, J. (1999). A handbook of techniques for formative
evaluation: Mapping the student’s learning experience. London: Kogan Page.
› 
Lenze, L. F., & Warner, M. S. (October, 1995). Summative evaluation and
formative feedback. Update, 1, National Education Association.
› 
Marsh, H. W., & Roche, L. A. (1997). Making students’ evaluations of teaching
effectiveness effective: The critical issues of validity, bias, and utility. American
Psychologist, 52, 1187-1197.
› 
Murray, H. G. (2007). Low-inference teaching behaviors and college teaching
effectiveness: Recent developments and controversies. In R. P. Perry & J. C.
Smart (Eds.), The scholarship of teaching and learning in higher education: An
evidence-based perspective (pp. 145-200). Dordrecht, The Netherlands:
Springer.
› 
Murray, H.G., and Smith, T.A. (1989). Effects of Midterm Behavioral Feedback
on End of-term Ratings of Instructor Effectiveness. Paper presented at annual
meeting of the American Education Research Association, San Francisco.
Great books on learning & feedback
›  Ambrose,
S.A., Bridges, M. W., DiPietro, M. Lovett,
M.C., Norman, M. K. (2010). How Learning Works:
Seven Research Based Principles for Smart
Teaching. San Francisco: Jossey Bass.
›  Angelo,
A.T. & Cross, K.P. (1997). Classroom
Assessment Techniques:A Handbook for College
Teachers, San Francisco:Jossey Bass.
›  Nilson,
L. (2010). Teaching At Its Best: A Research
Based Resource for College Educators. 3rd
edition. San Francisco:Jossey Bass.