now - Bett Middle East 2016

Measuring what
counts: Data-driven
decision making through
learning analytics to
enhance life skills
April 2016
Mohamad Ridwan Othman
Vice President, AGILE
Taylors Education Group
Not everything that counts can be
counted, and not everything that
can be counted counts.
William Bruce Cameron, 1963
Taylor’s Education Group
2016
Higher Education Division
47 years in Higher Education
Taylor’s
University
Taylor’s
College
Sri
Hartamas
Taylor’s
College
Subang
Jaya
British
University
Vietnam
Schools Division
65 years in Schools
Garden
International
School
Nexus
International
School
Putrajaya &
Singapore
Australian
International
School
Malaysia
Est. 2008
& 2007
Est. 2000
Taylor’s
International
School
KL
Taylor’s
International
School
Puchong
Est. 1991
Est. 2014
Est. 1969
Est. 1951
AGILE was set up in 2015 to elevate and make consistent, the learning and teaching quality across all these
institutions and also for the learning community across the region
THES2016.taylors.edu.my
Our vision is to “To
Educate the Youth of the
World to Take Their
Productive Place as
Leaders in the Global
Community.”
Coined about 5 years
ago, one of the key
outcome was for Taylors
to educate for the
Head, Heart and Hand
i.e. a Holistic Education
Why ?
#1
Generation Shift and
Change in Student Profiles
Born 1995 2009
Gen Z Connected Locally and Globally
•
As global consumers, Gen Z’s have interconnectedness that make them demographically
homogenous. Universally, they are exposed to the same brands and marketing, as
geographical location has become irrelevant
•
As they come of age, businesses, employers and even governments will want to steel
themselves for this demanding generation: They want things fast, flexible and in tune with their
beliefs on a mobile device.
General Response:

21st Century skill-sets and attributes. MOE Malaysia's 2015 Blueprint for HE

Many schools are exposing their students to overseas study trips, mission trips, internships and
community projects

Social/Collaborative learning (Problem based Learning, Project-based Learning) paradigms

Students trying their hand at being entrepreneurs or taking on part-time jobs

Parents can help them cope with harsh realities by not shielding them too much

Connect with personal touch via mobile
#2
New World of Everywhere
Everyone Everything Technology
Current and
Near
Future
Past/Current
Mode
Content
Content-based
Teacher-centric
Textbook
Courseware
Learner
Receptor
Education 1.0 Learners
• Receiving
• Responding
• Regurgitating
Teacher
Didactic
Communicator
Knowledge transfer
Reference: https://usergeneratededucation.wordpress.com/tag/education-3-0/
Emerging Model
Learning-based
Learner-centric
L2L
Content
Textbook
Courseware
eContent
Internet
Web
OER
Video Lectures
Teacher
Facilitator
Curator
Education 2.0 Learners
• Communicating
• Connecting
• Collaborating
Education 3.0 Learners
• Connectors
• Creators
• Constructivists
Learner
Learner
• Personal Learning
Networks
• Community of
Practice
Reference: https://usergeneratededucation.wordpress.com/tag/education-3-0/
#3
Change in Work Place Expectations
&
Disconnect of Employers and
Graduates
Source: https://robertjgardner.files.wordpress.com/2013/07/vuca-copy.png
5 Key Trends in the Work-place

New behaviors entering the workplace


Mobility


a new generation of digitally savvy and enabled
workers;
Globalization


being able to get your job done anywhere,
anytime, and on any device;
Millennials


being more public, sharing, collaborating, etc;
living and working in a world without boundaries;
Technology

big data, the cloud, internet of things, wearables,
and the like
Source: Jacob Morgan: 5 Ways People Over 50 Can Stay Relevant in the Future of Work
Training Net-Gen Students for Jobs Yet to be Invented
It’s not the content but the pedagogical method
Education 2.0/3.0
Education 1.0
Source: Dancing with Robots: Human Skills for Computerized Work
http://content.thirdway.org/publications/714/Dancing-With-Robots.pdf
Author: Jenna Filipkowski, PhD
([email protected])
Publication date: June 30, 2015 v.2
Collaboration U: Business and University Partnerships To
Secure Talent Pipelines

Employers are struggling to find college graduates with the
necessary skills for their jobs
significant and constantly
growing skills gap in today's workforce" and
"there appears to be a
between higher
education institutions that are preparing
the next generation of workers and the
employers who expect to hire them."
Employers are encountering a "
disconnect
Source: Human Capital Institute (2015), Collaboration U: Business and University Partnerships To
Secure Talent Pipelines. http://www.hci.org/hr-research/collaboration-u-business-and-universitypartnerships-secure-talent-pipelines
Changes in/of Learning

Innovation



Agile & responsive
Risk taking

Learner via the Teacher
Lifelong Learner
Fail early, fail fast
Problem finding

Beyond problem
solving

Sense-making
Learning Organization
Learning Society
13% OF PEOPLE
ARE ENGAGED IN
THEIR WORK
Source: Charlene Li, Altimeter, TED Talk 2014, Giving up control: leading in the digital era
Our Response
WHAT
The
to do next is easy
when you know
the
WHY
Outcome
TEG Holistic Education Framework
Attributes
Dimensions
Process
Educational Experience
(Value Proposition)
Academic
Excellence
Life Skills
1. Assumed
2. Obvious
Emotional
Wellbeing
Core
Purpose
Dimension
Source : Team Analysis
To Educate Youth of the World to Take Their Productive Place as
Leaders in the Global Community
Dimensions
Three (3) dimensions have been identified
with eight (8) attributes supporting the
delivery of holistic education
Academic Excellence
Life Skills
1.
2.
Discipline Specific Knowledge
Thinking & Problem Solving
Attribute
3.
4.
5.
6.
Emotional Wellbeing
Lifelong Learning
Communication Skills
Personal competencies (Self-awareness + Self-management – Goleman)
Social competencies (Social awareness + r’ship management – Goleman)
7.
Entrepreneurialism (Create, Capture and Deliver Value)
8.
Global perspectives
From Ad-Hoc,
Supplementary
To Integration within
Curriculum
MEASURE
How do we
the efficacy of the programmes
to inculcate HE?
Integration approach

Requires a holistic way to also measure the Life
Skills and Emotional Well Being portion.

Academic Excellence
easy
PROBLEM:isTIME

CONSUMING
andto ensure accuracy,
How do we measure,
easily,
RESOURCE
validity and reliability?
INTENSIVE

Rubrics aiding academics and allowing for
qualitative analysis would be helpful.
The ‘Promise’ of Learning Analytics
and Data-Driven Decision Making
“Why is it that the output of powerful learning analytics reporting
processes, acknowledged by institutional leaders as giving new insight
• Institutional
Culture,
into organizational patterns
and practices,
fail to influence institutional
• Innovation
and Change
Resistant….
planning and strategic
decision-making
processes?
• Approaches to motivate social and
cultural
change to institutional culture within
….this may be the result of lack
of attention
higher education, lack of understanding of the degree to which
individuals and cultures resist innovation and change, and lack of
understanding of approaches to motivating social and cultural change.”
Macfadyen, L. P., & Dawson, S. (2012). Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic
Plan. Educational Technology & Society, 15 (3), 149–163.
Critical success factors
Biggest ‘Problem’ with Analytics in Education
Great Engine but no or very little mapping
BI force-fit into Edu Model
Cumbersome & Difficult to administer
Time consuming to map to:
• Academic’s concerns,
• Departments concerns
• Dean’s and Edu Leaders concerns
• Not easy to integrate with other Data sources
•
•
•
•
Choice for this pilot in Taylor’s
Edu
Mapping
Great Engine but no or very little mapping
BI force-fit into Edu Model
Cumbersome & Difficult to administer
Time consuming toEasily
map to:
Integrated
• Academic’s concerns,
• Departments concerns
• Dean’s and Edu Leaders concerns
• Not easy to integrate with
other Data sources
Pedagogy
•
•
•
•
Centric
Edu Mapping done with Academic Rigour:
Professor Peter Mortimore
Professor Knut Roald
Professor Kjell B. Hjertø
Professor Louise Stoll
Professor Andy Hargreaves
Professor Viviane Robinson
Professor Michael Fullan
Professor Lorna Earl
Professor Jan Merok Paulsen
Yngve Lindvig, Louise Stoll, Michael Fullan, Andy
Hargreaves, Jarl Inge Wærness, Lorna Earl and
Anders Fosnes. Toronto, 2015
Professional quality
Input
Parental Support
Parental Expectations
Parental
socioeconomic
background
Input grades
Output
Intrinsic Motivation
Self-efficacy
Satisfaction
Tests and/or grades
School Attainment
Key capacities
Self-Assessment
Digital skills
Communication skills
Decision-making skills
Citizenship
Student-Teacher Relations
Formative assessment
Student Involvement
Teacher Feedback
Order in Class
Teacher Encouragement of
Creativity
Career counselling
Scientifically tested
student survey
Student behaviour
Mastery-approach goal items -MAP
Mastery-avoidance goal items – MAV
Performance-approach goal items - PAP
Performance-avoidance goal items - PAV
Collectivism
Individualism
Win-loose
Commitment
Learning environment
Social well-being
Bullying
Trust
Safety
Conflicts
Conflicts – (Emotional Relationship
Conflict)
Conflicts – (Emotional Task Conflict)
Conflicts – CT (Cognitive Task
Conflict)
Conflicts - (Cognitive Relationship
Conflict)
Emotional intelligence
EQ – SEA (Self Emotion
Appraisal)
EQ - OEA (Other Emotion
Appraisal)
EQ – UOE (Use of Emotion)
EQ – ROE (Regulation of
Emotion)
Why student survey?
Recent research has
shown that student
perception data are
predictive of student
achievement gains and
in many cases are more
reliable than classroom
observations and student
growth measures
Students are at a good vantage point
to make judgements about classrooms
because they have encountered
many different learning environments
and have enough time in a class to
form accurate impressions. Also, even if
teachers are inconsistent in their day-to
day behaviour, they usually project a
consistent image of the long-standing
attributes of a classroom environment
Unlike value-added data,
student survey data provide
actionable feedback that
teachers and supervisors
can use to target specific
areas for improvement
(Schulz, Sud and Crowe 2014)
(Fraser 1998)
Example of Survey Questions
Emotional
intelligence
Self-emotion appraisal (SEA)
•
I have a good sense of why I have certain feelings most of the time.
•
I have good understanding of my own emotions.
•
I really understand what I feel.
•
I always know whether or not I am happy.
Others’ emotion appraisal (OEA)
•
I always know my friends’ emotions from their behavior.
•
I am a good observer of others’ emotions.
•
I am sensitive to the feelings and emotions of others.
•
I have good understanding of the emotions of people around me.
Use of emotion (UOE)
•
I always set goals for myself and then try my best to achieve them.
•
I always tell myself I am a competent person.
•
I am a self-motivated person.
•
I would always encourage myself to try my best.
Regulation of emotion (ROE)
•
I am able to control my temper and handle difficulties rationally.
•
I am quite capable of controlling my own emotions.
•
I can always calm down quickly when I am very angry.
•
I have good control of my own emotions.
Pilot Approach and Specifics

3000+ students in Taylor’s College Subang Jaya and Taylor’s College Sri Hartamas

4 key programmes

A Levels

South Australian Matriculation (SACE)

Canadian Pre-University

International Baccalaureate

Feb to April 2016 – Administer anonymous surveys via the system

April to May 2016 – Analysis of Results

Sharing with all teachers in TC

Train the teachers in slicing, filtering and managing results in the system

Presentation of Detailed Findings in THES2016, end of May 2016

Review and Fine-Tuning for roll-out in TC

Non-anonymous and Aggregrated at Programme level

Integrated with other systems such as the LMS (Student actual grades)
Possible Analysis and Data
For every factor, academics
in TC can easily plot twofactor graphs like this, at
both the Institution Level and
even at their own
programme level. (Also by
Gender etc.)
Ability to see the different
student groups that maybe
require the different and
necessary interventions.
Possible Analysis and Data (Future)
Its also possible to
drill down to the
individual students
and give a detailed
map such as this for
every one.
Correlations
between these
data and other
data such as
Student Grades in
the SMS or LMS can
also be shown.
Evidence
Using evidence and customized educational technology systematically for
releasing the potential in every learner: Findings presented at ICSEI Jan
2016
1.
Assemble and present data from multiple sources leads to better
professional learning
2.
In-depth processes with a broad evidence perspective leads to better
professional learning
3.
Use of digital reflection tools leads to better professional learning
4.
This type of professional learning leads to better student outcome,
measured with a broad set of indicators
Paper presented at the annual conference of the International Congress for School Effectiveness and
Improvement, January 2016, Glasgow.
“The dialog maps is an engaging way to inquire into our class-
climate. We are challenged to expand the way we think about learning
and well-being.”
Student
“For the first time the management was inquiring into our
professional learning activities, instead of just evaluating our
results.”
School leader
“Using the process designer with relevant prototypes led to better
prepared teachers that discuss teaching quality instead of data
quality.”
School Leader
“This is probably… No, this is for sure the most sophisticated and
elegant educational solution I’ve ever seen. And the thinking behind it is
very much in line with ours.”
North American, Deputy Education Minister
“After seeing the products and mission, we need to further develop our
strategy and change the focus of our agenda for buying tools for
schools.”
Group of Educational Directors, European Country
“These products is an excellent way to broaden the focus in education
to include well-being, 21 century skills and holistic learning.”
Asian group of Educational Directors
THES2016.taylors.edu.my
25-27 April 2016
Delivering Holistic Edu,
Acquiring of LifeSkills is
key for
students now
Mohamad Ridwan Othman
[email protected]
THES2016.taylors.edu.my
Choosing the right tools
helps us measure and finetune
Conexus is one possible
approach. Happy
Exploring! Do come to
THES2016 for further
results