Assessment for Learning and Program Improvement

Assessment for Learning and
Program Improvement
Kevin Barry, Director
for Teaching and Learning
Universidad La Salle
Arequipa, Peru – October 26, 2013
Workshop Outcomes
 
After participating fully you will be able to:
 
Compare and contrast purposes for assessment
 
Identify key aspects of Utilization Focused Evaluation
 
Use taxonomies to inform articulation of goals
 
Align major assignments/evaluations to goals
 
 
Consider a Scholarship of Teaching and Learning (SoTL)
approach
Consider applying assessment principles at the program
level
for Teaching and Learning
Summative Assessment
 
Focused primarily on judgment
 
Not concerned with contributing to learning process
 
Dominant form of assessment seen in higher
education policy
for Teaching and Learning
1
Formative Assessment
 
Focused on assessment as tool to enhance learning
 
Typically focused on improvement on current task
for Teaching and Learning
Assessment for Learning
 
Focused on developing transferable capacity for
learning
 
Impact extends beyond current task/course
for Teaching and Learning
2
Utilization-Focused Evaluation (UFE) begins
with the premise that evaluations should be
judged by their utility and actual use.
Patton, 2008, p. 37
UFE does not prescribe any specific content,
method, or theory. It is a guiding framework, as
opposed to a methodology.
3
4
The Seven Principles for Good Practice in
Higher Education
 
Good Practice
 
 
 
 
 
Communicates High Expectations
Encourages Student-Faculty Contact, Cooperation Among
Students, and Active Learning
Gives Prompt Feedback
Emphasizes Time On Task
Respects Diverse Talents and Ways of Learning
Chickering and Gamson (1987).
for Teaching and Learning
5
6
Why Articulate Student Learning Outcomes?
 
Communicate high expectations
 
Identify the most important outcomes
 
Add transparency for the students
 
Form the basis for designing assessments/
assignments
 
Facilitate assessment of effectiveness
for Teaching and Learning
The Cognitive Process Dimension
From A Taxonomy for Learning, Teaching & Assessing: A Revision of
Bloom’s Taxonomy of Education Objectives. Anderson & Krathwohl (Eds.)
for Teaching and Learning
7
Taxonomy of Significant Learning
From Creating Significant
Learning Experiences: An
Integrated Approach to Designing
College Courses. L. Dee Fink
for Teaching and Learning
The Interactive Nature of Significant Learning
From Creating Significant
Learning Experiences: An
Integrated Approach to Designing
College Courses. L. Dee Fink
for Teaching and Learning
8
When Writing Student Learning Goals
 
Use specific language
 
Students will be able to …
... Describe, Analyze, Argue, Solve, Create, Compare, etc.
 
Avoid vague or passive language
 
“Know” and “understand” are valuable but vague
 
Avoid passive language – “Students will be exposed to …”
for Teaching and Learning
9
La telenovela: CAPP 23507, FTT 20260, ILS 20901, LAST 20402
Spring 2014
Professor: Kevin Barry
Office: 353 DeBartolo Hall – Email: [email protected] – Phone: 631-4443
Office Hours: Tue & Wed 3:30-4:30 or appt.
La telenovela:
género, significado cultural y producción estudiantil
Materiales:
Ø Telenovelas: cómo son cómo se escriben, Eduardo Adrianzén Herrán (Telenovelas)
Ø Grammar of the Shot, Roy Thompson (gos)
Ø Web Resources: www.nd.edu/courses/capp/capp23507.01/www
Ø Listserv: [email protected]
Recommended: Spanish/English dictionary. Good online dictionaries include: www.diccionarios.com and
http://www.wordreference.com.
Course Description:
In this course you will study the telenovela in an integrated multidisciplinary learning environment. You will learn the
formulas of the classic telenovela and its archetypical characters, and look into the cultural impact of the telenovela in Latin
America and the world. In addition, you will explore the linguistic and technical aspects of screenwriting, production, acting
and post-production through the creation of a class "telenovela."
Student Learning Outcomes:
Upon successful completion of this course you will be able to:
• Describe the origin of the telenovela genre, the cultural significance of the telenovela in Latin America, and common
formulas used in telenovela plots.
• Describe, compare and contrast archetypical characters used in telenovelas, identify the archetypes of characters in
telenovelas viewed and develop characters that conform to these archetypes.
• Describe and be able to identify and apply a variety of production and post-production techniques including types of
shots, transitions and best practices.
• Operate video and audio equipment in a location-based production environment.
• Write, produce, plan, direct, shoot, act in, and edit a mini telenovela.
Teaching and Learning Methods:
We will use a variety of methods in this class with the focus being on active student engagement and the successful completion
of the student produced telenovela. Methods used include:
• Acting activities to explore acting practices appropriate for the telenovela and to develop oral skills and acting abilities
and increase comfort in front of the camera.
• Readings and viewing of Telenovelas accompanied by reflection questions that aid discussion preparation and
facilitate the connection of reading concepts to the viewed material.
o You are strongly encouraged to record your ideas/questions as you read.
• Student creation, production and post-production of commercials in small groups and of a telenovela as a class with
the application of the concepts learned as a primary focus.
o Assigning of primary roles in telenovela creation, production and post-production to give an authentic
production experience
10
11
Questions for Formulating Significant Learning Goals From http://www.deefinkandassociates.com/GuidetoCourseDesignAug05.pdf "A year (or more) after this course is over, I want and hope that students will ." Foundational Knowledge • What key information (e.g., facts, terms, formulae, concepts, principles, relationships, etc.) is/are important for students to understand and remember in the future? • What key ideas (or perspectives) are important for students to understand in this course? Application Goals • What kinds of thinking are important for students to learn? • Critical thinking, in which students analyze and evaluate • Creative thinking, in which students imagine and create • Practical thinking, in which students solve problems and make decisions • What important skills do students need to gain? • Do students need to learn how to manage complex projects? Integration Goals • What connections (similarities and interactions) should students recognize and make…: • Among ideas within this course? • Among the information, ideas, and perspectives in this course and those in other courses or areas? • Among material in this course and the students' own personal, social, and/or work life? Human Dimensions Goals • What could or should students learn about themselves? • What could or should students learn about understanding others and/or interacting with them? Caring Goals • What changes/values do you hope students will adopt? • Feelings? • Interests? • Ideas? Learning-­‐How-­‐to-­‐Learn" Goals • What would you like for students to learn about: • how to be good students in a course like this? • how to learn about this particular subject? • how to become a self-­‐directed learner of this subject, i.e., having a learning agenda of what they need/want to learn, and a plan for learning it? 12
activities to enhance student learning
based on kolb’s learning dimensions
Most of us spend little time thinking about how we
learn; we just know that we do—and haven’t we been
doing it for years anyway? When we do think about
learning, we assume that everyone learns in a like manner. A common fallacy held by some university faculty is
to assume that students learn in the same way they do. In
practice, this assumption often degenerates to an unspoken feeling that “all good students think and learn like I
do.” Consequently, students with different learning styles
are often dismissed as either lacking intelligence or being
non-cooperative (Harb 1995). This means that most
courses are usually taught in the teacher’s learning style.
Educational psychologist David Kolb suggests a theory of experiential learning to provide a framework for
examining some of the many ways in which people learn:
In the model below, these dimensions are presented
in four quadrants. Within each quadrant are course
activities (e.g., assignments) that can help facilitate
learning within each of the dimensions. The learning
activities shown in the oval below display passive activities in the center and progressively more active learning
strategies toward the outer edge. Research suggests that
students acquire more knowledge and retain it longer
when more active learning strategies are employed (such
as those at the outer perimeter of the oval). Teachers
seeking to enhance their students’ learning should consider including in their courses a mixture of active strategies that appeal to diverse ways of learning.
Bibliography
Harb, John N., Pam K. Hurt, Ronald E. Terry, and Ken J. Williamson,
Perceptions
Teaching through the Cycle, 2nd ed. (Provo, Utah: BYU, 1995).
Concrete Experience: sensing/feeling
Abstract Conceptualization: thinking
Kolb, David A., Experiential Learning: Experience as the Source of Learning
and Development (Englewood Cliffs, New Jersey: Prentice-Hall, 1984).
Svinicki, Marilla D., Learning and Motivation in the Postsecondary Classroom
Processing Modes
(Bolton, Massachusetts: Anker Publishing Company, 2004).
Reflective Observation: watching
Active Experimentation: doing
Svinicki, Marilla D. and Nancy M. Dixon, “The Kolb Model: Modified for
Classroom Activities,” College Teaching 35, no. 4 (1987): 146.
Direct experience
xper
ienc
e
AC
Recall of experience
n
io
at
In-class experience (lab)
TI
LE
Simulations
Films/tapes
Field
work
Lecture examples
Projects
Labs Homework
Case
studies
Lecture
Simulations Examples
PASSIVE
LEARNING
Rhetorical
questions in
class
Thought
questions for
readings
Discussion
Brainstorming
Logs
Journals
IN
RN
Lecture analogies, descriptions
EA
Text reading
L
VE
n
io
at
AC
AC
Model-building exercises
TIV
er
v
Model critiques
Papers, project proposals
G Thru Reflecti
v
NIN
e
O
AR
bs
G Thru Active Ex
N G Thru Concrete E
VE
pe
ri m
en
t
ACT
NI
LEAR
IVE
on
zati
E LE
i
l
a
ARNIN
ptu
G Thru Abstract Conce
focus on faculty fall 2006 3 13
TI
Examples of Course Skeletons
Course:
Biological Research
(capstone for majors)
I want my students to use the scientific method for original research and
to communicate their results with the scientific community.
1
9
2
10
3
11
4
12
5
13
6
14
7
15 Original Scientific experiment
8
in scientific format, plus oral report
for Teaching and Learning
14
La telenovela: historia, significado cultural y producción estudiantil
Calendario del curso
Semana 1
1/17 – Rompehielos/Ejercicios comunicativos. Orientación a los materiales y al equipo.
Tarea: Lectura: Telenovelas 24-29 (que es una TN) (35 min.), Telenovelas 91-99 (arquetipos I) (35 min.), MM 12-21
(introducción y tipos de planos)(35 min.)
1/19 – Ejercicios comunicativos. Ver La usurpadora 1. Discusión de los varios elementos dentro de la telenovela y la
lectura.
Tarea: Ver La usurpadora 2- 5 (120 min.). Lectura: Telenovelas 100-123 (arq. II- V) y completar las preguntas de
discusión (130 min.).
Semana 2
1/24 – Discusión de la telenovela. Discusión del público de la telenovela. Discusión del diálogo. Introducción al
guión.
Tarea: Escuchar: NPR – Telemundo… (10 min.), Telenovelas 53-70 (las ideas) (70 min.), Lectura: MM 22-41 (los
planos sencillos) (30 min.)
1/26 –Ver algunas publicidades. Introducción al guión grafico. Comenzar a crear una publicidad (lluvia de ideas,
bosquejo, comenzar el guión). Practicar el rodaje.
Tarea: Ver La usurpadora 6-9 y completar las preguntas de discusión (145 min.). Escribir un guión de una de las
publicidades de la lluvia de ideas (65 min.). Lectura: Manual de la Canon GL2 16-22, 31, 35-36, 38
(visualización), 47, 51 (30 min.).
Semana 3
1/31 – Información sobre el mercadeo. Continuar la creación/redacción de las publicidades. Practicar el rodaje
Tarea: Terminar el guión para la publicidad (80 min.). Manual de la Canon GL2 52-54, 57, 69-70, 76-81 (40 min.)
2/2 – Intercambiar guiones de las publicidades. Practicar el rodaje.
Tarea: Preparar un guión gráfico de la publicidad (60 min.). Ver La usurpadora 10-13 y completar las preguntas de
discusión (145 min.). Lectura: MM 42-57 (introducción al montaje)( 35 min.),
Semana 4
2/7 – Ver La usurpadora 14 (fin). Discutir La usurpadora. Redacción del guión gráfico y preparación para el rodaje
de las publicidades (¿qué se necesita? localidades, actores, equipo, etc.).
Tarea: Final Cut Pro – Técnicas básicas de capturar (40 min.). Terminar de grabar las publicidades. (80 min.)
2/9 – Orientación al “Multimedia Studio”. Rodaje de las publicidades.
Tarea: Ver Amor real 1-4 y completar las preguntas de discusión (145 min.). MM 58-75 (5 tipos del montaje)(40
min.). Final Cut Pro – Cómo hacer el primer corto (40 min.).
Semana 5
2/14 – Demostración y práctica del montaje.
Tarea: Lectura: Telenovelas 135-147 (dialogo) (60 min.). Final Cut Pro – Claves a la rapidez (15 min.). 2-3
ideas/cambios al diálogo aburrido (15 min.). Lectura: MM 76-99 (prácticas del trabajo I)(40 min.).
2/16 – Demostración y práctica del montaje. (9:00-10:45)
Tarea: Lectura: Telenovelas 71-78 (biblia) (35 min.). Borrador de la biblia, 1-2 páginas, (60 min.). Ver Amor real 58 y completar las preguntas de discusión (145 min.).
Semana 6
2/ 21 – Ver Amor real 9. Traer 5 copias de la biblia individual para comentar, sintetizar y crear una biblia colectiva
15
La telenovela: historia, significado cultural y producción estudiantil
Tarea: Todos – rodar las últimas escenas del episodio 4. Los montadores – terminar el montaje bruto del episodio 4.
Música – continuar con la creación/selección de la música (2 hrs.)
4/6 – Ver el montaje bruto del episodio 4. Discusión sobre el trabajo hasta ahora.
Tarea: Rodar lo que necesitan. Montaje en grupos de los episodios asignados. Música – terminar con la
creación/selección de la música (4 hrs.)
Semana 13
4/11 – Ver la telenovela estudiantil. Charlar sobre lo que todavía tienen que hacer.
Tarea: Traducción del dialogo de los episodios (en grupos). (2 hrs.)
4/13 –Montaje en grupos, añadir la música.
Tarea: Rodaje final de lo que necesitan para refinar la telenovela. (4 hrs.)
Semana 14
4/18 – Ver la telenovela estudiantil. Charlar sobre lo que todavía se necesita hacer.
Tarea: Continuar con el montaje refinado (grupos). Promotores – escribir los créditos. (2 hrs.)
4/20 – TCEs. Continuar con el montaje refinado (grupos).
Tarea: Montadores – terminar el montaje refinado, añadir créditos. Promotores – mandar anuncios sobre la
exhibición pública.
Semana 15
4/25 – Ver la telenovela estudiantil con publicidades. Hacer cambios si son necesarios.
Tarea:
4/27 – Ver episodio completo de una telenovela.
Tarea:
4/29 –
Exhibición Pública de la telenovela estudiantil
Semana 16
5/2 – Repaso
MIERCOLES 5/10
EXAMEN FINAL
10:30 – 12:30
16
Scholarship of Teaching and Learning (SoTL)
 
Is scholarly inquiry into student learning
 
Typically involves action research
 
Advances the practice of teaching
 
Uses data to evaluate effectiveness
 
May result in better learning
for Teaching and Learning
General Chemistry at Notre Dame – Problems
 
Too many “at risk” students were dropping the
course and the possibility of a STEM career.
 
Students were frustrated with the large lecture course
format
for Teaching and Learning
17
General Chemistry SoTL – Solution
 
Designed an alternate section for “at risk” students
 
 
 
Reduced section size.
Added engagement and collaborative work to enhance
learning
Student pairs answer conceptual questions in class
 
Added recitation sessions for group problem solving
 
Maintained common exam with traditional sections
for Teaching and Learning
General Chemistry SoTL – Implementation
 
Enrolled at risk students in alternate sections
 
Gathered Feedback – Student surveys, Common
exam, Video of recitation sections
 
Improved recitation after first round
 
Used common exams with traditional sections as a
control.
for Teaching and Learning
18
General Chemistry SoTL – Outcomes
 
Collected data on student success/retention
 
Looked at success in this course and future science courses
 
50% more at risk students continue in science
 
Greater success in future science courses
http://gallery.carnegiefoundation.org/collections/castl_he/djacobs/
for Teaching and Learning
19
Program Assessment
 
Utilization Focused Evaluation approach?
 
 
 
 
Greater need to focus on steps 1-4
Steps 6 & 7 must be broader but maintain focus
Sampling will likely be necessary
Remember that evaluations should be judged on their
actual use
for Teaching and Learning
Program Assessment Examples
 
Romance Languages
 
Engineering – Responding to ABET criteria
 
University Learning Outcomes for Undergraduates
for Teaching and Learning
20
to communicate effectively,
clearly and cohesively in
both written and oral discourse
GOALS
last revised 12-19-07
OBJECTIVES
write clearly, precisely, and cohesively using the
disciplinary conventions and methodologies that
constitute effective literary analysis; frame and
sustain an argument that includes both the exposition and analysis of information; demonstrate
competence, fluency and clarity of expression,
with minimal interference from English, and with a
good range of vocabulary and syntax; incorporate
a wide range of expressions and rhetorical forms
(persuasion, etc.) with attention to register and
finer shades of meaning.
express themselves clearly, accurately and effectively in a variety of communicative contexts and
situations; sustain an advanced level of spoken
interaction with native speakers in academic,
professional and informal settings; prepare and
deliver a well-organized, logical oral presentation;
formulate and support ideas and opinions; respond to complex lines of reasoning
understand complex and/or extended literary
works in Spanish from all periods and genres;
consider intention and effect from the perspectives of the author, the text, and the reader; discern finer points, allusions, irony and other nuances in meaning even when not clearly articulated;
relate formal aspects of the text to its theme
understand spoken language, even when delivered at native speed; comprehend a range of
recorded audiovisual materials in Spanish (film,
television, online media, etc.), including those with
idiomatic usage; follow propositionally abstract
and linguistically complex lines of argument; synthesize and paraphrase the ideas and details of
extended speech and academic lectures
Students will be able to:
We aim for a common
minimum of Advanced
High for graduating majors.
The majority of our students for whom Spanish is
not their first language
reach at least Advanced
mid of the ACTFL Proficiency guidelines
We aim for and typically
achieve a common minimum of Advanced Plus
on the ACTFL Proficiency
Guidelines among our
graduating majors.
STANDARDS
Preliminary Working Draft
Course embedded assignments, exams and papers
essay exams,
Multi-draft writing assignments through 300 level
Class discussions (observation); embedded assignments (oral presentations)
Course embedded assignments: exams and papers
Class discussions (observation by faculty)
DIRECT
Spanish Program Learning Outcomes and Assessment
INDIRECT
ASSESSMENT
Writing assignments, exams, papers all in Spanish;
peer-tutoring through Writing Center
Discussions in the program
are conducted exclusively
in Spanish; co-curricular
opportunities such as Tertulia, Spanish Club, Theater workshop, service learning, exchange through
ISSA also help reinforce
spoken interaction
Readings are all in Spanish
Discussions in the program
are conducted exclusively
in Spanish; co-curricular
opportunities such as Tertulia, Spanish Club, Theater workshop, service learning, exchange through
ISSA also help reinforce
spoken interaction
University of Notre Dame Department of Romance Languages and Literatures
STUDENT LEARNING OUTCOMES
COMPREHENSION
PRODUCTION
Listening
Reading
Speaking
Writing
SKILLS
21
Standardized proficiency
tests mid-curriculum and at
time of graduation: BYU
exam used for placement
online listening comprehension programs such as
Langmedia; ACTFL oral
proficiency tests for first
majors; transcript analysis ;
random evaluation of representative writing samples; summative individual
or collective portfolios;
proposed expansion and/or
more structured incorporation of peer tutoring address needs of upper division; proposed smaller
classes for bridge courses;
proposed expansion and/or
more structured incorporation of co-curriculars
Spanish has been using
standardized criteria of
evaluation through 310 for
the past two years, and
now in process of reflecting
upon 310 and beyond
through analytic and holistic scoring.
PROPOSED (DRAFT)
ABET Objectives and Outcomes — Department of Civil & Environmental Engineering & Earth Sciences
10/22/13 3:36 PM
Department of Civil & Environmental
Engineering & Earth Sciences
HOME > EDUCATION > Undergraduate Programs > ABET Objectives and Outcomes
ABET Objectives and Outcomes
Program Educational Objectives and Outcomes: Curriculum in Civil Engineering
1. Program Educational Objectives
The curriculum in Civil Engineering is designed to cultivate and promote among our undergraduates high levels of scholarship, desire to seek understanding, a
solid foundation in professional and personal responsibility, desire for service, and an ability/willingness to serve as leaders within both the engineering
profession and society.
This curriculum results from the following program objectives. The graduates of our program will:
Demonstrate that they are applying physics, chemistry, biology, geology, mathematics and/or
engineering science through participation in graduate school, engineering practice, or other
professional practice.
Be responsible for identifying, formulating and executing solutions to multidisciplinary problems encountered
within either the practice of civil engineering or other professional endeavors.
Communicate (either orally or in writing) in the professional and/or public domain.
Enter the engineering profession through advanced study and/or professional practice,
or pursue an alternative professional career path through advanced study
Apply personal, professional and ethical considerations in completing engineering design
or job requirements.
Identify the impact of engineering on society and the natural and built environments.
Demonstrate continuing activities focused on learning past the undergraduate education.
Contribute to service efforts to the community in both the professional and private realm.
Recognize the importance of, and continue to pursue, professional engineer (PE) licensure.
2. Learning Outcomes
The Learning Objectives outlined above will be assessed through a series of specific learning outcomes. These outcomes are listed below:
Graduates will possess an ability to apply knowledge of mathematics, science, and engineering.
Graduates will possess an ability to design and conduct experiments, and to analyze and interpret data in at least two of the major civil engineering specialty
disciplines.
Graduates will possess an ability to identify, formulate, and solve engineering problems in both individual and team environments, particularly in the design of
a system, component, or process to meet desired needs.
Graduates will possess an understanding of professional ethical responsibility.
Graduates will be able to communicate effectively, both orally and in writing, and both individually and as members of multi-disciplinary teams.
Graduates will have a knowledge of contemporary issues in civil engineering, and the broad education necessary to understand the impact of engineering
solutions in a global and societal context.
Graduates will recognize the need for and an ability to engage in life-long learning and the importance of professional licensure.
Graduates will possess an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice.
http://ceees.nd.edu/education/undergraduate-program/abet-objectives-and-outcomes
Page 1 of 2
22
23
Bibliography
Ambrose, Susan A. & others (2010). How Learning Works: Seven Research-Based Principles for
Smart Teaching. San Francisco: Jossey-Bass.
Anderson, Lorin W. and Krathwohl, David R. Eds. (2001). A Taxonomy for Learning, Teaching,
and Assessing: A revision of Bloom’s Taxonomy of Educational Objectives. Addison
Wesley Longman, Inc.
Bain, Ken. (2004). What the Best College Teachers Do. Harvard University Press. Cambridge
Massachusetts. London, England.
Boud, D. & Falchikov, N. (Eds.) (2007). Rethinking Assessment in Higher Education: Learning
for the longer term. Routledge, New York, NY.
Chickering, Arthur W., and Gamson, Zelda F. (1987). “Seven Principles for Good Practice In
Undergraduate Education,” AAHE Bulletin (March 1987, pp. 3-7).
Fink, L. Dee (2003). Creating Significant Learning Experiences: An Integrated Approach to
Designing College Courses. San Francisco: Jossey-Bass.
Kurfiss, Joanne (1987). Critical Thinking. ASHE-ERIC Higher Education Report. Washington,
DC: The George Washington University, School of Education and Human Development.
Patton, M. Q. (2012). Essentials of Utilization-Focused Evaluation. Sage Publications, Inc.,
Thousand Oaks, CA.
Stevens, Dannelle D., & Levi, Antonia J. (2012). Introduction to Rubrics: An Assessment Tool to
Save Grading Time, Convey Effective Feedback, and Promote Student Learning, 2nd
Edition. Sterling, Virginia: Stylus.
Walvoord, Barbara E., & Anderson, Virginia J., (2009). Effective Grading: A Tool for Learning
and Assessment in College, 2nd Edition. Jossey-Bass Inc, San Francisco, CA.
Wiggins, Grant, & McTighe, Jay (2005). Understanding by Design, 2nd Edition. Association for
Supervision and Curriculum Development, Alexandria, VA.
24