Supervisors’ oral feedback on HDR student writing A criterion-based approach
Transcription
Supervisors’ oral feedback on HDR student writing A criterion-based approach
Supervisors’ oral feedback on HDR student writing A criterion-based approach Dr Bronwen Dyson [email protected] Supervisors’ oral feedback › For Higher Degree Research students, feedback constitutes: “… “a major, if not the major, form of instruction” (Bitchener, Basturkmen & East (2010, p.82). o Written feedback provided on student drafts. o Oral feedback provided in meetings, either before or after drafting. › Limited research on supervisory feedback, particularly oral feedback. Largest study (Paré, 2010, p. 107) concludes that advisors’ spoken comments are “often ambiguous, enigmatic and coded – that is saturated with meaning, but difficult to understand”. › So, is supervisors’ oral feedback ambiguous & difficult to understand? › Argument: Writing criteria can reveal whether or not there is a lack of clarity - or any other problem - in supervisors’ oral feedback and so should be more widely used. 2 Studies of supervisor oral feedback › Paré (2010, 2011): questionnaire & recorded meetings of large number of Canadian Humanities and Sciences supervisors. › Argues that advisors “struggle to articulate implicit knowledge”, e.g. › I think the information is there but I have two main points about it. One is that I think it should be a bit more focused. ... The other comment is to, I don’t know, firm it up, I suppose. ... I’d like more numbers, I suppose. ... So that my general feeling is that the chapter itself ... should be put within a slightly bigger box for the committee (2011, 61-62). › However: › Analysis is broad-brush. › Learning is not measured, despite the conclusion that the feedback is “difficult to understand”. 3 Studies of supervisor oral feedback (2) › Bitchener, Basturkmen, East & Meyer (2011): questionnaire & written feedback of 35 NZ Humanities, Commerce & Sciences/Mathematics supervisors. › Shows supervisors perceive a range of feedback types e.g. oral + written. › Meetings have various aims & characteristics e.g. talk/listen vs address omissions. › Problems e.g. some do not hold meetings (2/35). › Suggest classification of written feedback using speech acts e.g. “Perhaps do X” = ‘suggesting’ (cf Kumar & Stracke, 2007). › However: › No analysis of actual meeting practices. › No mention of writing criteria, which could provide a more direct link with the writing than speech acts. 4 Research questions › So, little research on supervisors’ oral feedback on thesis writing, none on learning but the suggestion of some problems & the potential of writing criteria not yet explored. › Research questions: (1) How successful is supervisors’ oral feedback, as measured by a set of writing criteria? (2) In what ways could writing criteria guide this feedback? 5 The study Case study 1 Doctoral advisor - male, US-trained. Student - female, Cantonese-speaking, IELTS: 6.5. Case study 2 Doctoral advisor - female, US-trained. Student - female, Mandarin-speaking, IELTS: 8. Data Fortnightly meetings audio-recorded & transcribed. 8 one-hour sessions (4 per case study). 6 Data analysis: the criteria MASUS writing criteria to assess strengths & weaknesses(Bonanno & Jones, 2007) 5 criteria A Use of source material B Structure and development of the argument C Academic writing style D Grammatical correctness E Qualities of presentation o 25 sub-criteria e.g. • B o Genre is appropriate to the task o Clear focused thesis statement o Choice of Theme & New reflects structure … • E • Paragraphing reflects essay structure … 7 Data analysis (2) Student learning • Measured by ‘uptake’, an optional move in which the learner responds to feedback on a gap in his/her knowledge (Ellis, Basturkmen & Loewen, 2001, p. 286). • Uptake an early sign of learning. Moves & episodes • Feedback & learning analyzed as: • Moves: Utterances within a turn or over adjoining turns relating to a single sub-criterion. • Episodes: One or more moves relating to a single Area e.g. A or more than one Area e.g. A & B. 8 Findings: Feedback moves Majority of feedback in Areas A & B (95.78%) A (Use of source material) B Structure and development of the argument) 42.46% (N = 262) 53.32% (N = 329) Most relevant data is employed e.g. I think (author) and (author) obviously are very good for this, I have to see them in here (DA 2, 1:222) Genre is appropriate to the task e.g. So I think this might become a short separate chapter on its own after this long chapter (DA 1, 1:336) 9 Findings: Feedback moves (2) Minority of feedback in Areas C, D & E C: Academic writing style D: Grammatical correctness E: Qualities of presentation 0.16% (N = 1) 0.16% (N = 1) 3.89% (N = 24) Appropriate lexis e.g. Like this uh . her‘herausforderung’ right which I still don't understand right (DA 1, 4: 160) Clause structure e.g. Yeah this is a run-on sentence (DA 2, 1: 119) Other qualities of presentation e.g. It's a bit long I think (DA 1, 4: 48) 10 Findings: Feedback episodes Majority of feedback episodes combine Areas A & B A B C D E A+B Other combinat ions 13.35% (N = 75) 16.90% (N = 95) 0 0 0 61.03% (N = 343) 49% (N = 8.72) DA 2: [B] And then you say, look how a whole style ... has merged in China and outside China, following the success of (title). Right. ... [A] And then I think it would be great to just go through some newspapers and do you know what I mean? Student 2: Yeah. (DA 2, 4: 182-185) 11 Findings: + or - feedback More feedback on inappropriate than appropriate writing Strengths Weaknesses 8.75% (N = 54) 91.24% (N = 563) Critical evaluation of evidence e.g. That was excellent, … I like how you say there's a gap obviously because there's all this stuff on Chinese American literature and nothing on very recent … migrants. (DA 2, 1: 129-131) Focused thesis statement e.g. But I think that what's missing there is why are you looking at the specific points that you talk about? (DA 2, 1: 35) 12 Findings: Effect on learning Majority of uptake in Areas of main feedback. Less uptake than feedback. A+B feedback episodes produce half the uptake. % A B C D E A+B Total N Feedback 43 53 0.2 0.2 3.9 61 617 Uptake 60 0 0 3.2 30 154 37 Feedback & uptake on A: DA 2: So my big question was ... why are you using (author) in your thesis? … Student 2: I think that the context that (author) uses this concept is different from my context. (2, 1: 9-10) 13 Discussion: RQ 1 How successful is supervisors’ oral feedback, as measured by a set of writing criteria? Successful and not-so-successful: ü Fine-tuned awareness of criteria A & B & their sub-criteria Important counter-evidence of Paré’s (2010, 2011) conclusion of ambiguity. Humanities’ language-savvy supervisors. o Little attention to C, D & E Problematic given difficulties at micro-level (e.g. Cooley & Lewkowitz, 1997). Although, written feedback on micro-level (Bitchener, Basturkmen & East, 2010). o Episodes zigzagged across Areas A & B Problematic given difficulties with Area B (Bitchener & Basturkmen, 2006) Bitchener et al. (2011): written feedback ‘piles up’ functions. o More feedback on weaknesses than strengths Hyland & Hyland (2001): written feedback ‘sugars the pill’. o More feedback than uptake Not just due to ‘ambiguity’ as in Paré (2010, 2011), but range of factors. 14 Discussion: RQ 2 In what ways could writing criteria guide this feedback? MASUS could guide supervisors’ feedback by: › Developing skills in providing clear feedback on criteria A & B & their subcriteria. › Encouraging attention to C, D & E since often errors occur & are repeated in these Areas. › Fostering a focus on specific Areas, especially ones which pose particular difficulties e.g. B. › Creating greater balance between feedback on strengths & weaknesses. › Thereby, fostering greater balance between feedback & uptake. 15 Conclusion: a criterion-based approach › Study contributes to the limited research on supervisors’ oral feedback. › Suggests that this feedback is not problematic in ways previously indicated but in some new ways. › Highlights the value of writing criteria, specifically MASUS, as a framework for analyzing & strengthening supervisors’ oral feedback. › Provides a direct link between the oral feedback & the writing. › Limitations: A small sample from Humanities which does not test potential of criterion-based approach. › Future research: Widen the disciplinary lens, trial a criterion-based approach & consider effect on student writing. 16 › Thank you! › [email protected] › Postgraduate Writing › Faculty of Arts & Social Sciences › University of Sydney 17 References › Bitchener, J., & Basturkmen, H. (2006). Perceptions of the difficulties of postgraduate L2 thesis students writing the discussion section. Journal of English for Academic Purposes, 5, 4-18. › Bitchener, J., Basturkmen, H., & East, M. (2010). The focus of supervisor written feedback to thesis/dissertation students. International Journal of English Studies, 10(2), 79-97. › Bitchener, J., Basturkmen, H., East, M., & Meyer, H. (2011). Best practice in supervisor feedback to thesis students. Wellington, New Zealand: The National Centre for Tertiary Teaching Excellence. › Bonanno, H., & Jones, J. (2007). Measuring the Academic Skills of University Students: The MASUS procedure, a diagnostic assessment. Sydney: University of Sydney, Learning Centre. › Casanave, C. P., & Hubbard, P. (1992). The writing assignments and writing problems of doctoral students: faculty perceptions, pedagogical issues, and needed research. English for Specific Purposes, 11(1), 33-49. › Cooley, L., & Lewkowitz, J. (1997). Developing awareness of the rhetorical and linguistic conventions of writing a thesis in English: addressing the needs of EFL/ESL postgraduate students. In A. Duszak (Ed.), Culture and styles of academic discourse (pp. 113-129). Berlin, Germany: Mouton de Gruyter. › Ellis, R., Basturkmen, H. & Loewen,S. (2001). Learner uptake in communicative ESL lessons. Language Learning 51(2), 281-318. › Hyland, F. & Hyland, K. (2001), Sugaring the pill: Praise and criticism in written feedback. Journal of Second Language Writing 10(3), 185-212. › Kumar, V., & Stracke, E. (2007). An analysis of written feedback on a PhD thesis. Teaching in Higher Education, 12(4), 461-470. › Paré, A. (2010). Making sense of supervision: Deciphering feedback. In P. Thomson & M. Walker (Eds.), The Routledge doctoral student's companion: Getting to grips with research in education and the social sciences (pp. 107-115). New York, NY: Routledge. › Paré, A. (2011). Speaking and writing: Supervisory feedback and the dissertation. In L. McAlpine & C. Amundsen (Eds.), Doctoral education: Research-based strategies