SEE PROGRAM MODERATION WORKSHOP 2014

SEE PROGRAM
MODERATION WORKSHOP
2014
LWA is a contracted supplier of assessment validation services, moderation workshops and professional development
services under the Skills for Education and Employment Program funded by the
Department of Industry
© LWA 2014
1
AGENDA
© LWA 2014
9.00 – 9.30
9.30 – 9.40
9.40 – 10.10
Registration, on arrival tea and coffee
Welcome, introductions, overview and purpose of the day
SESSION 1: IV IN THE SEE
10.10 – 10.55
SESSION 2: MODERATION

PTAs
10.55 – 11.15
11.15 – 11.35
Morning tea
SESSION 2: CONTINUED

Group feedback and discussion of PTA moderation
11.35 – 12.55
SESSION 3: MODERATION

Learning tasks

Group feedback and discussion of Learning moderation
12.55 – 1.30
1.30 – 3.00
Lunch
SESSION 4: MODERATION

Oral communication tasks
3.00 – 3.15
3.15 – 3.45
Afternoon tea
SESSION 4: CONTINUED
Group feedback and discussion of Oral Communication moderation
3.45 – 4.00
Wrap up and evaluation forms
2
SESSION 1: IV IN THE SEE
Handout 1: What is needed for verification
 Do you have any questions about what you need to provide?
© LWA 2014
3
IV REQUIREMENTS - PTA
 KPI 3 Accurate Assessment
Were the assessment tasks appropriate for the ACSF level?
 PTA kit appropriately aligned to ACSF indicators
 PTA kit contains a sufficient range of tasks to adequately assess ACSF indicators
Is there sufficient evidence that the provider has assessed the client accurately against
the ACSF?
 Student response demonstrates exit competence
 Sufficient assessor analysis of student performance against the ACSF indicators claimed
© LWA 2014
4
IV REQUIREMENTS – IPA/LOA
 KPI 3 Accurate Assessment
Were the assessment tasks appropriate for the ACSF?
 Tasks appropriately aligned to the ACSF
 Task materials are appropriate for use as assessment tasks
Is there sufficient evidence that the provider has assessed the client accurately against
the ACSF?
 Student response demonstrates exit competence
 Sufficient assessor analysis of student performance against the ACSF
 Range of text types demonstrated
 KPI 4 Quality Training
Are the tasks relevant to the client’s learning needs?
 Tasks are appropriate for the client cohort
 Tasks are appropriate for adult learners
© LWA 2014
5
TODAY WE ARE MODERATING,
NOT VERIFYING
 Tasks and client responses stand alone for moderation purposes
 Consistency in the interpretation and application of the ACSF
 Build on professional knowledge
 Support teaching and assessment practices
© LWA 2014
6
SESSION 2
Moderation Activities:
PTAs
© LWA 2014
7
SESSION 2: PTA TASKS 1, 3, 4
Use the tasks to decide:
 Is the allocated ACSF core skill and level accurate?
 Is the task sufficient to support the indicator?
 Does the client response demonstrate the indicator?
© LWA 2014
8
PTA 1 :TASK, LEARNING & OC
TASK
© LWA 2014
TASK: Aligned

Detailed instructions to the assessor

Appropriate prompts to the assessor

Both learning indicators comprehensively addressed

Excellent prompts to the assessor to observe .02 evidence throughout the PTA

More familiar questions could be used at the start of the interview, e.g. introductory
comments relating to personal details

Modified questions may be required for Language clients, e.g. personal details,
background
9
PTA 1: CLIENT RESPONSE, LEARNING
2.01
Client Sample: Demonstrated
•
•
2.02
© LWA 2014
Client identifies some goals and areas of learning need e.g. numeracy, work related skills
Evidence of a learning goal and the investigating of relevant learning pathway e.g. has
investigated requirements for roadside slashing business
• Evidence of barriers e.g. family, financial skills
• Assessor has recorded detailed client quotes
Client Sample: Not demonstrated

Evidence more aligned to 1.02

Further questions could have been asked e.g about ‘White Card’ training, to prompt
client to expand on strategies used to gain this qualification

Source of information about web search unclear

Evidence could have been strengthened if assessor had noted any strategies used by
the client during the assessment
10
PTA 1: CLIENT RESPONSE, OC
3.07
Client Sample: Not demonstrated
Evidence suggests potential level 3 but it is not clear for example what are the client’s
actual words and what has been paraphrased
•
More in depth discussion of some topics, for example, the CFS may have allowed for
gathering of stronger evidence at level 3 than a short response for every question e.g.
What is involved with being a CFS volunteer? Tell me about the training required for
the CFS. What type of person is suited to the CFS?
•
Quotes captured need to demonstrate appropriate levels of complexity
•
Assessor comments need to be supported by reference to specific quotes. The
reporting sheet included assessor comments such as ‘used a range of
tenses…vocabulary sufficiently broad’ but these are too general
Client Sample: Demonstrated
•
3.08
•
•
•
© LWA 2014
Client was able to engage in conversation on familiar and some less familiar contexts
Assessor comments need to be supported by reference to specific quotes of questions
asked by the assessor and to the client’s ability to maintain a conversation e.g. turn
taking
Where assessor noted ‘asked questions to clarify discussed volunteer responsibilities’ a
few examples of actual questions may have provided stronger evidence
11
PTA 3: TASK & CLIENT RESPONSE, LEARNING
TASK TASK: Aligned

Consider review of layout of Assessment Interview Form to provide more space
for recording responses

Suggest starting the Assessment Interview Form with questions relating to
highly familiar, personal details to put low level clients at ease

Prompts are too long if read as a block of text

1.01
1.02
© LWA 2014
Assessor could have extended the conversation to demonstrate full extent of
competence
Client Sample: Demonstrated

Evidence supports reported indicator

Further questioning, prompting may have allowed for higher competence to be
demonstrated
Client Sample: Demonstrated

Evidence supports reported indicator

Further questioning, prompting may have allowed for higher competence to be
demonstrated

Assessor could have made observations of strategies throughout the PTA
12
PTA 3: TASK & CLIENT RESPONSE, OC
1.07
1.08
© LWA 2014
Client Sample: Demonstrated

Quotes on ‘Assessment Interview Form’ support reported indicator

Extensive supporting quotes are recorded on the ‘scribed assessor’ sheet

Ensure all assessor comments pertain to indicator performance features
Client Sample: Demonstrated

Assessor comments supported with referenced examples

Evidence is strengthened with quoted examples of requests for clarification and
repetition
13
PTA 3: TASK & CLIENT RESPONSE, READING
TASK TASK: Aligned
•
Pre
1.03
Pre
1.04
© LWA 2014
It would be useful to include a range of texts in a PTA kit relevant to the client
groups
Client Sample: Demonstrated

Assessor comments address Writing and Numeracy performance features

Ensure comments address the performance features of the indicator being
claimed
Client Sample: Demonstrated

Some assessor comments give evidence for learning
14
PTA 4: TASK & CLIENT RESPONSE, LEARNING
TASK
TASK OC & LEARNING
Assessment Interview Form: Aligned
• Suggest starting the Assessment Interview form with questions relating to highly familiar,
personal details to put low level clients at ease
• Prompts are too long if read as a block of text
• Assessor could have extended the conversation to demonstrate full extent of competence
A self-assessment form: Not aligned
• Not an appropriate task for a low level client
• Limited application against ACSF Performance features
• Assessor noted ‘high level of support ‘was provided. What form did the support take e.g. was
the client expected to read the task but found it too difficult?
Core LLN Skill Assessment Summary Sheet: Aligned
• Assessor has provided examples of client performance against the ACSF Focus Areas
Pre
1.01
Client Sample: Demonstrated
Pre
1.02
Client Sample: Demonstrated
© LWA 2014

Assessor notes together with client performance support reported indicator

Assessor notes together with client performance support reported indicator

Assessor observed strategies throughout the PTA
15
PTA 4: CLIENT RESPONSE OC
Pre
1.07
Pre
1.08
© LWA 2014
Client Sample: Demonstrated

Based on evidence client sample aligned. Evidence on Assessor Notes sheet
suggests more highly familiar questions may have produced evidence of 1.07

No clear delineation of comments addressing of individual indicators
Client Sample: Not Demonstrated

Client responses demonstrated understanding of several questions of varying
complexity thereby indicating 1.08 was achieved

Need to include a few examples of questions that need to be rephrased

Assessor comments span both indicators. The .08 indicator is not clearly
addressed
16
PTA 4: TASK & CLIENT RESPONSE READING
TASK Both text types formatted. No prose text used
Task: Emergency Procedures: Not aligned

Task complexity spans several levels of reading

In order to award Pre level 1, a task aligned to level 1 must be used
Task: Calendar: Not aligned

Task contains complex language

Pre
1.03
Pre
1.04
© LWA 2014
Too many questions. Questions 5 – 11 do not address level 1 reading
performance features
Client Sample: Not Demonstrated
•
•
Comment ‘have difficulties if the text has unfamiliar vocabulary and complex
structure’ indicates Emergency Procedures task is too complex
The client may well have demonstrated competency at level 1 if appropriate texts
were used given that she has answered questions 1 - 4 correctly
17
PTA 4: CLIENT RESPONSE WRITING
TASK Task: aligned
Pre
1.05
Pre
1.06
© LWA 2014

Two appropriate text types were used

Parts of the form exceed level 1 complexity. To ensure consistency between
assessors maybe modify the form so there is a clear distinction between the
sections to show level 1 and level 2
Client Samples: Demonstrated
• Personal details on form copied as noted by assessor indicate the client is still prelevel 1.05
• Although the client demonstrated some 1.06 features (as acknowledged in the
assessor notes e.g. assessor’s comments relating to punctuation) pre-level 1 is
appropriate e.g. the information in the form was copied
18
PTA 4: TASK & CLIENT RESPONSE NUMERACY
TASK Task: Not Aligned
• Tasks are not real life. They consist of a series of decontextualized activities.
• Portions of the task are maths questions (questions 3-7)
• Some of the task activities are beyond level 1 (questions 15, 16, 17, 18, 200)
1.09
1.10
Pre
1.11
© LWA 2014
Client Sample: Demonstrated
• Some comments span indicator levels
• Assessor notes together with client performance supports reported indicators
despite the unsuitable tasks
19
SESSION 3
Moderation Activities:
LEARNING
© LWA 2014
20
SESSION 3: LEARNING TASKS 1-5
Use the tasks to decide:
 Is the allocated ACSF core skill and level accurate?
 Is the task sufficient to support the indicator?
 Does the client response demonstrate the indicator? (If supplied)
 Would you adapt this task to use in your own context? How?
© LWA 2014
21
LEARNING TASK 1: TASK & CLIENT
RESPONSE
MY GOALS & HOW DO YOU LIKE TO LEARN
1.01
© LWA 2014
It is acknowledged that Distance learning presents particular challenges for
assessment and documentation
Tasks: Aligned
• Could be broadened to cover past experiences of successful learning, any barriers
and planning and organising
• Task 2 provides evidence of preferred learning approaches but client may be
confused by the graphic
• Task 2 could be redesigned to allow for demonstration of more focus areas
Client Sample: Demonstrated
• Evidence would be strengthened if assessor noted any relevant details gained in
conversation with the client
• Assessor notes map evidence to focus areas although the notes are very brief
• Assessor comment on strategies is a .02 performance feature
• Assessor could note client’s ability to engage in learning activities with appropriate
scaffolding as additional evidence for learner identity
22
LEARNING TASK 2: TASK & CLIENT
RESPONSE
BOOKLET & STRATEGIES FOR LEARNING
2.01
Tasks: Aligned

Covers a broad range of performance features
Client Sample: Demonstrated
© LWA 2014

Prompting and advice box ticked but no information provided

Variation in scaffolding provided

Assessor comments are succinct yet thorough with mapping of evidence to relevant
performance features

Assessor prompting with Qs 1 and 7 could have strengthened the evidence

Updated SEE program task Cover Sheet should have been used
23
LEARNING TASK 3: TASK ONLY
LEARNING IN THE WORKPLACE
2.02
© LWA 2014
Task: Aligned
• Depending on responses, the task allows for a range of levels to be demonstrated
• Good task to facilitate evidence from a work placement
• Task could be modified to suit a range of workplace contexts
• Some of the statements could be reworded as questions e.g. statement 1 to allow
for a broader range of responses
24
LEARNING TASK 4: TASK & CLIENT
RESPONSE
RANGE OF ACTIVITIES
2.02
Tasks: Aligned
 Task well constructed covering a broad range of performance features
Client Sample: Demonstrated
• Responses may have been more detailed if the assessor had scribed the student
responses
• Assessor Comments template is well structured
• Not all client responses show strong exit level 2 competence e.g. task 1, task B
question e. However, there is sufficient evidence overall to support the outcome
• Some assessor prompting may have allowed for stronger responses e.g. completion
of task 5
© LWA 2014
25
LEARNING TASK 5: TASK ONLY
MY WORKPLACE – A REFLECTION
3.01
3.02
© LWA 2014
Task: Aligned
• Good task for gathering evidence from a work placement and task could be
modified according to workplace context
• Could be used as part of a project about the work experience. This could
broaden the possible demonstration a wider range of Learning focus Areas as
well as demonstration of other Core Skills
26
SESSION 4
Moderation Activities:
ORAL
COMMUNICATION
© LWA 2014
27
SESSION 4: ACTIVITY, TASK 2
Q & A BOARD GAME
FILM CLIP REMOVED
You will see a group of students playing the Questions and Answers game.
Pre-teaching includes practising question forms.
After watching, discuss in your group:
 What OC outcomes can be assessed with this game?
 Are there any other core skills that can be assessed? How?
 How can the game be adapted to assess different ACSF levels?
© LWA 2014
28
SESSION 4: OC TASK 2
MODERATION: Q & A BOARD GAME
TASK & CLIENT SAMPLE
 Is the allocated ACSF core skill and level accurate?
 Is the task sufficient to support the indicator?
 Does the client response demonstrate the indicator?
 Would you adapt this task to use in your own context? How?
© LWA 2014
29
OC TASK 2: TASK & CLIENT RESPONSE
Q & A BOARD GAME
1.07
Task: Aligned

Creative task with potential to be used effectively across OC levels

Instruction sheet provides guidance on participation (supported by examples on
the white board. See video)

Assessor sheet is very comprehensive and allows for reporting of strong evidence
against the performance features
Client Sample: Demonstrated
© LWA 2014

Teacher notes provide strong demonstration of both the ability to ask questions
and give short responses

Assessor comments support reported indicator well
30
SESSION 4: ACTIVITY, TASK 8
ABC 7:30 REPORT
FILM CLIP REMOVED
This is a segment from the ABC 7.30 Report entitled Casualising the Workforce.
Discuss:
 What OC listening level would you assess with this story?
 Could you also use it to assess speaking? How?
 Could it be used as a springboard to assess other core skills? How?
 Would you use this idea? How?
© LWA 2014
31
SESSION 4: OC TASK 8
MODERATION: 7:30 REPORT, CASUALISING THE WORKFORCE
TASK & CLIENT SAMPLE
 Is the allocated ACSF core skill and level accurate?
 Is the task sufficient to support the indicator?
 Does the client response demonstrate the indicator?
 Would you adapt this task to use in your own context? How?
© LWA 2014
32
OC TASK 8: TASK & CLIENT RESPONSE
CASUALISING THE WORKFORCE
4.08
Task: Aligned
•
•
•
•
•
•
Excellent task relevant to the needs of the learner cohort
Context of task in classroom learning well explained
Text demonstrates appropriate complexity for the indicator
Could be strengthened in Focus Area of Audience and purpose. Consider some
questions to elicit client understanding of the purpose and motives of different
speakers
Could be strengthened in Focus Area of Comprehension. Consider some
question reformulation to allow client to clearly demonstrate ability to relate
pieces of information in an oral text rather than treating them as separate pieces
of information
Evidence strengthened through the provision of a part transcript to give an
indication of the complexity of the listening text. It is not necessary to include the
entire transcript
Client Sample: Demonstrated
•
•
•
© LWA 2014
Demonstrates understanding of complex oral texts (multiple speakers)
Understands abstract nouns, e.g. casualisation
Understands vocabulary specific to the topic
33
SESSION 4: ORAL COMMUNICATION TASKS
1,3,4,5,6,7
Use the tasks to decide:
 Is the allocated ACSF core skill and level accurate?
 Is the task sufficient to support the indicator?
 Does the client response demonstrate the indicator? (If supplied)
 Would you adapt this task to use in your own context? How?
© LWA 2014
34
OC TASK 1: TASK ONLY
AT THE BUTCHER
1.07
Task: Not Aligned
•
•
•
© LWA 2014
Appropriate real life task
Role play cards too heavily prompted to allow for demonstration of exit level
competency
Assessment Criteria sheet does not closely reflect indicator performance features
35
OC TASK 3: TASK & CLIENT RESPONSE
G & S TRANSPORT TICKET
1.08
Task: Aligned
• Authentic relevant task for learner cohort supporting community engagement
• Well structured and scaffolded task
• Student reflection on page 5 could potentially provide some learning evidence
Client Sample: Demonstrated
• Assessor annotations together with quotes of client’s speech strongly support
indicator performance features
• Transcript of transaction provided
• Detail of classroom context provided on page 3 of task
© LWA 2014
36
OC TASK 4: TASK & CLIENT RESPONSE
DENTAL APPOINTMENT & CASUAL CONVERSATION
1.07
Tasks: Aligned
• Clear instructions to learners
• Prompt cards provide appropriate level of support
Client Samples: Demonstrated
• Strong evidence aligned to the relevant performance features both on the Task
Cover Sheet and on the tasks
• Clear transcripts of conversations provided
© LWA 2014
37
OC TASK 5: TASK & CLIENT RESPONSE
STUDENT RANG ME
1.07
Authentic sample: Aligned
Client Sample: Demonstrated
© LWA 2014

Good example of capturing authentic evidence from the student

Evidence supports the indicator claimed
38
OC TASK 6: TASK ONLY
PHONE MESSAGE
2.08
Task: Aligned
•
•
•
•
© LWA 2014
Need to consider students with spiky profiles i.e. will writing ability impact on task
outcomes?
Task designed for use with multiple core skills (Listening, Writing, Reading)
Listening text is appropriate context and complexity
The message pad may be confusing as it does not readily relate to the information
in the text e.g. ‘while you were out’ could be changed to ‘from’
39
OC TASK 7: TASK ONLY
PART TIME & CASUAL WORK
3.08
Task: Aligned
•
•
•
•
© LWA 2014
The task is highly relevant for students planning to enter the workforce
Explanation of context in classroom learning would need to be explained on the
Task Cover Sheet
Question 8 could be reformulated to give students the opportunity to express
their opinion on the benefit of this information to listeners
Some level 4 complexity in the text but scaffolding of questions supports the 3.08
outcome
40
PLEASE FILL IN YOUR
EVALUATION FORM
Contact us:
T: 03 9429 7551
F: 03 9429 7221
www.lwa.net.au
SEE admin: [email protected]
Office admin: [email protected]
Verifiers:
[email protected] [email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
© LWA 2014
41