The General Chemistry Project
Transcription
The General Chemistry Project
Phase II Analysis of Chemistry Support Initiative Prepared by Ahrash Bissell Ph.D. Project background Dr. Richard MacPhail (chemistry) and Dr. Ahrash Bissell (ASIP) met in Fall 2005 to discuss the general chemistry (Chem 21 and 22) sequence, with particular focus on the professor's desired learning outcomes, the constraints he faced, and his goals for the future. Dr. MacPhail and the ARC staff exchanged information and ideas related to the course goals, the ways in which to best support student learning, and measurements of student success. Because the visitation rate of general chemistry students to the ARC was so high, Dr. Bissell developed tools that could be used in ASIP one-on-one sessions to more effectively convey the learning expectations of the general chemistry courses and to give the students some concrete materials which could help them to practice the needed academic skills. Based on our subjective assessments of student understanding and performance, one of the best tools was something we called "problem manipulation". Problem manipulation is a technique that is grounded in metacognitive theory. Briefly, the method encourages students to generate and evaluate knowledge (appropriate to the course) on their own. It is scaffolded, content-specific, and suitable for group study. It also provides an appropriate means for predicting the types of questions that might appear on an exam. Finally, the method integrates well with other academic skills that students will often learn at the ARC, and is thus well suited to the "whole-course approach" to supporting student learning and the notion that most students have a suite of skills that should be attended to simultaneously. Dr. MacPhail met with the ARC staff to discuss how we might both benefit from embedding the method directly into his course recitation. Project implementation in Spring, 2006 The format that we chose for integrating the method was as follows: 1. Pre-recitation exercise. a. Prior to each week's recitation, the students would receive a single problem to work on their own. The problem was designed to be "advanced", but do-able, and the content was matched to that time-period of the class. b. Included in the pre-recitation problem were a few "canned" manipulations. Hints were given, and the manipulations were designed mostly to illustrate the concept, rather than challenge the students. 2. In-recitation group work. a. In recitation, each class was divided up into groups of four (or so). Each group worked on the same recitation exercise, which closely matched the 1 content of the pre-recitation exercise but was designed to be more difficult. The expectation was that the group members would work together to overcome the challenge of the problem. b. The exercise also consisted of either "canned" manipulations (early in the semester) or "open" manipulations (later in the semester). In either case, the expectation was that the problem would require the group members to work together in order to solve it in the allotted time. This teamwork element was especially crucial when the groups were asked generate their own manipulations later in the semester. 3. Post-recitation exercise. a. After each recitation, the students picked up a post-recitation problem, due the following week (along with the next pre-recitation problem). Again, the content was matched to the other problems, and the difficulty level was designed to be similar to that of the in-recitation problem. b. These problems also contained manipulations, some canned, some open. these problems were also graded, though the grade was mostly based on effort, not performance. There were a few recitations that were devoted to exam review, and one or two recitations where a "fun" chemistry exercise was substituted, but the bulk of the recitations were formatted as illustrated above. At the end of the semester, all of the students in the course were asked to fill out a survey in which they reflected on the recitation format, on their performance in recitation and the course overall, and on the contrast of those recitation sections to other recitation formats they might have had (in science). The return rate on the survey (administered electronically) was very high, approximately 95%. Alterations to this format for Spring, 2007. Based on feedback from the TAs and the surveys from 2006, we made several adjustments for Dr. MacPhail's Chem 22 class in 2007. 1. In addition to the recitation exercises (described above), we also administered take-home quizzes that explicitly asked the students to use and reflect on the problem-manipulation method. The quiz questions were very similar to at least one of the questions on each midterm exam and the final. In addition to giving us (and the students) more feedback on academic performance, this arrangement makes it possible to evaluate the correlation between student improvement in recitation (using the problem-manipulation method) and on the exams (using whatever method they choose). 2. In order to more accurately make the comparison described above, course graders separately recorded the scores for a subset of exam questions that were designed to mimic the recitation problems and were most easily solved by applying the problem-manipulation method. 2 3. Course T.A.s devoted more of the recitation period to metacognitive development. They wanted the students to understand the rationale behind the methods, rather than simply apply the method when asked to. 4. We also altered some of the survey questions to capture student perceptions that were missed the first time and also to reflect the changes made above. Results Subjective assessment of the success of the project In terms of all of the positive outcomes from the process described in the Introduction, this project has been a great success. In addition to this specific project, we have engaged meaningfully with faculty in assessing the characteristics and learning needs of "at-risk" students in chemistry and math. The chemistry faculty involved in this assessment have been enthusiastic and receptive throughout, and they have recognized the work and perspective of the ARC as being key to the ongoing effort. There is continued interest in the evaluation of the "off-cycle" chemistry courses, the development of additional peerled-team-learning support programs, and other collaborative programs as appropriate. Given appropriate personnel and resources, we have every expectation that the ARC can make a real difference in supporting freshman and sophomore students in these STEM courses. Feedback from Dr. MacPhail, Claire Siburt, and the course TAs was also very positive. In general, the students seemed to be more engaged, especially in recitation, as well as more aware of the learning expectations for the course. The TAs enjoyed leading their sections more than they had in previous courses, and they felt invested in the teaching process. Survey results The student responses to the end-of-course survey were also overwhelmingly positive. Survey responses were very similar for both years and are discussed together. Between 77-87% of the students said that they preferred the recitation format they experienced in Dr. MacPhail's class over prior college recitation experiences (in chemistry or comparable classes). Similarly, 88-91% of students said that we should continue to use this recitation format in future semesters. We asked students to consider the impact of the recitation sections and problemmanipulations on their course performance and understanding of the material. 87% of students said that the recitation sections and associated problem-manipulation methodology helped to prepare them for the midterm exams. 90% of students found every component of the recitation format to be helpful (pre-recitation, in-recitation, and post-recitation exercises). In terms of student self-evaluation of the impact of the method on their understanding of how they learn best, the results were more equivocal, with approximately 50% of students agreeing that there was a positive impact. The results 3 were similar for student reports on whether they actually learned the chemistry concepts better. Nonetheless, students did not feel that the format was a burden. Indeed, students really appreciated the group work (86-92% approval), and many students (approx. 60%) indicated that they were already using the method in other courses (or could how it could be useful). More than anything, it appeared that most students appreciated the thought that went into the recitation format, as well as the notion that the professor was giving them guidance for how to learn the material rather than just what to learn. Student performance results There were no obvious overall differences in course performance among the two sections, or even as compared to previous years where this format was not used. However, the course uses a curved grading scale, and with so many students it would in fact be surprising if there was a substantial shift in the performance of every student in the class. Instead, we are most interested in two things: 1. Does every student who truly desires to succeed have access to the knowledge and skills that are necessary to do so? 2. Is there an impact on course performance based on performance on skill mastery and performance in the recitation sections? The first item is harder to quantify, but we have some information that leads us to believe that we came closer to achieving that goal in these courses than in the other chemistry courses. First, professor and TA feedback indicated that most students they met with seemed to understand the demands of the course better than in previous semesters. Moreover, more students were able to identify the specific areas of weakness and were trying to get help on those areas accordingly. Second, the visitation rate to ASIP for students in Dr. MacPhail's sections was quite a bit lower than the other chemistry sections, despite the greater exposure to the ARC in his classes. This reduction is almost certainly due to greater student confidence that they understand the learning expectations and know which areas need improvement. Indeed, students in those classes that did visit ASIP were clearly more aware of the possible reasons for their struggles than students in previous semesters or other sections. Note, however, that there continues to be a concern about student overconfidence in their metacognitive abilities and capacity to self-teach better academic skills. This is an area that needs further attention. The second item is amenable to statistical analysis by comparing student responses to the end-of-semester survey to their actual performances, both in recitation and on course exams. 4