Peer Review - Access Digital and Distributed Learning

Peer Review - Students
constructing feedback
Fiona McCloy Office for Digital Learning Wednesday 25 March 2015 ulster.ac.uk Aim & Outline
Session Aim
Educational benefit of using peer
review in higher education and
how this can be applied at Ulster.
Session Outline
Definition, technology, process,
demo, benefits, case study, tips.
What is peer review?
“Peer review refers to scenarios
where students construct a
feedback response in relation to
the work of other students.
This would usually be a written
response based on an evaluative
judgement of the work against
some criteria.”
(Nicol, 2013)
Nicol, D (2013) Peer review: putting feedback
processes in students’ hands, Perspectives on
Pedagogy and Practice, Journal of the Centre for
Higher Education Practice, Ulster University
Role of technology?
• 
• 
• 
• 
• 
• 
Distribution options
Scalable
Convenient
Confidential
Easily add criteria
Easily collate results
PEER Toolkit Project, Jisc, 2011
reap.ac.uk/PEERToolkit.aspx
Technologies
Institutional tools
•  Blackboard Self & Peer Assessment
•  Turnitin PeerMark
Other tools
• 
• 
• 
• 
WebPA
Calibrated Peer Review
PeerWise
Aropa
Peer Review Process
Student Perspective
Designing Peer Review
Instructor perspective
Consider:
• 
• 
• 
• 
• 
Timings – submission period, evaluation period, review period
How many papers will each student review?
Will students review their own?
Distribution of papers – random, instructor or student selects?
Will the activity be marked? Marks allocated for taking part? How
many marks allocated?
•  Will the activity feed forward, so student has the opportunity to act
upon the feedback?
•  Will students be able to review if they haven’t submitted something?
•  Is it anonymous?
Tool Comparison
Self and Peer
Assessment
Turnitin
PeerMark
Grading option
No grading option
Preview option
Not available
Not available
Student pairing option
Not available
Students can choose papers to review
Not available
Can annotate directly onto paper
Not available
Layering with GradeMark feedback
User friendly interface for students
Turnitin PeerMark Tool
Document Viewer
Turni4n PeerMark Document Viewer screenshot (h>ps://www.turni4n.com/help/cv/pm-­‐help-­‐instructor.html) Peer Review
PeerMark Demo
Turnitin PeerMark video. Available via Vimeo: http://vimeo.com/30517483
“..if we want students to develop
critical thinking, judgement and
autonomy in assignment
production they should be
provided with high-level evaluative
experiences similar to those of
experts.
Peer review, students evaluating
and commenting on each other's
work, is one way to achieve this”
PEER Toolkit Project, Jisc, 2011
reap.ac.uk/PEERToolkit.aspx
Learning Benefits
•  Producing feedback is cognitively challenging and requires the
students to engage at a higher level with the substance and skills of
their discipline;
•  Students must rehearse and reconstruct their own understanding of
a particular topic;
•  Criteria and standards used to assess student work are more likely
to become internalised;
•  Through critically analysing and evaluating the outputs of others,
students are put into the same decision space as experts which can
help support acquisition of the tacit knowledge that experts use when
tackling a task.
Nicol, D (2013) Peer review: putting feedback processes in students’ hands, Perspectives
on Pedagogy and Practice, Journal of the Centre for Higher Education Practice, Ulster
University
Case Study
Review design specification using PeerMark [n=82]
Nicol, Thomson and Breslin, 2013
Produce engineering draft design specification.
Review two peers; review their own. Rubric provided. Two
questions: comment on convincingness of design rationale;
identify a worthwhile improvement that could be made and reason.
Marks for participating in review activity, not on quality.
Students were positive about experience. Active process that
helped them to improve their own work.
Helped them be more critical, detached, analytical and logical
to reviewing their own work.
Nicol, D (2013) Peer review: putting feedback processes in students’ hands, Perspectives
on Pedagogy and Practice, Journal of the Centre for Higher Education Practice, University
of Ulster
Mapping to Ulster Principles
Clarify good
performance
Time and effort
on task
Timely high
quality feedback
Opportunities to
act on feedback
Positive motivational
beliefs
Self-assessment
& reflection
Interaction
& dialogue
Ulster Principles of Assessment and Feedback for Learning, Ulster University, 2011
http://ee.ulster.ac.uk/assessment_and_feedback/
Peer Review Tips
• 
• 
• 
• 
• 
Involve students
Communicate expectations and alleviate fears
Start small in first year then increase complexity
Carefully consider learning design/activity structure
Carefully construct questions and criteria/rubric –
could involve students in development of questions
•  Feed forward –provide opportunity for students to
improve their work
•  Recommend low stakes marks for participation – no
evidence peer grading improves individual performance
ulster.ac.uk