Minnesota State University Moorhead Assessment Report Cover Sheet ( )
Transcription
Minnesota State University Moorhead Assessment Report Cover Sheet ( )
Minnesota State University Moorhead Assessment Report Cover Sheet (An electronic version of this form can be accessed at http://web.mnstate.edu/assess) Note: All non-accredited programs are required to complete this form. Include Assessment Reporting Forms for each learning outcome assessed. Academic Program: Sociology, Criminal Justice, and Geronotology Department: Sociology and Criminal Justice College: Social and Natural Science` Date: 5/2/2013 1. Name(s) of Department Assessment Coordinator and/or Assessment Committee Members Karen Branden, Lee Vigilant, Deb White, Joel Powell 2. List of All Student Learning Outcomes. (List all outcomes, placing an asterisk (*) by the outcomes you are assessing this year.) We did not assess a specific learning outcome this academic year. This was our year of review and planning. The process of review, reflection and planning is scheduled to take place every four academic years. During this process the combination of all assessment results over the three previous academic years in which all SLO’s have been assessed were combined. This information was reviewed to help determine improvements needed in assessment, informed curriculum decisions, and helped maintain quality education in the department. We are currently happy with our process and are making no changes at this time. Student Learning Outcomes: Students completing a major in Sociology, Criminal Justice, and/or Gerontology should be able to: Demonstrate knowledge of important theoretical perspectives in their discipline 1. (Sociology, CJ, or Gerontology). 2. Demonstrate an understanding of scholarly sources of information (i.e. research published in peer-reviewed journals). 3. Critique methods, theory and the work of other scholars. 4. Conduct, interpret, present and write reports about social scientific research using qualitative and/or quantitative methodologies. 5. Demonstrate an understanding of the role of theory in social research. 6. Explain current social phenomena as they relate to major theoretical traditions. 1 Schedule for Assessing Student Learning Outcomes: Academic year Outcomes to be assessed 2010-2011 3, 4 2011-2012 1, 5, 6 2012-2013 2 2013-2014 *Review and Planning 3. Describe how your program has addressed the comments from the Student Learning Outcomes Assessment Committee during the past two academic years? (If you have made changes to your plan, file a revised Assessment Plan Cover Sheet and Assessment Planning Form(s).) We have clarified our rubrics for each student learning outcome based on a request from the assessment committee. 4. If you have received an Instructional Improvement Grant in the past two years, identify the outcomes on which the grant was based and provide a summary here of the results from your grant. 5. Signatures Department Chair or Program Director Dean or Director Required Attachments: 1. Assessment Reporting Forms 2. Records of department meetings when Assessment Report was discussed and approved. 2 Department of Sociology and Criminal Justice Assessment Plan Student Learning Outcomes: Students completing a major in Sociology, Criminal Justice, and/or Gerontology should be able to: 1. Demonstrate knowledge of important theoretical perspectives in their discipline (Sociology, CJ, or Gerontology). 2. Demonstrate an understanding of scholarly sources of information (i.e. research published in peerreviewed journals). 3. Critique methods, theory and the work of other scholars. 4. Conduct, interpret, present and write reports about social scientific research using qualitative and/or quantitative methodologies. 5. Demonstrate an understanding of the role of theory in social research. 6. Explain current social phenomena as they relate to major theoretical traditions. Schedule for Assessing Student Learning Outcomes: Year Outcomes to be assessed Year 1 2 Year 2 3, 4 Year 3 1, 5, 6 Year 4 Review and Planning *The process of review, reflection and planning will take place every four academic years. During this process the combination of all assessment results over the previous three academic years will be combined. This information will be reviewed to help determine improvements needed in assessment, will inform curriculum decisions, and help maintain quality education in the department. Methods of Assessment Assessment of Outcome 1: Demonstrate knowledge of important theoretical perspectives in their discipline (Sociology, CJ, or Gerontology). Final papers of students in each of the following courses will be selected: Sociology 302—Social Theory Sociology 407—Contemporary Theory. Assessment committee member will rank each paper on a scale ranging from 1 (failed to meet expectations) to 10 (exceeds expectations). Assessment committee members will also prepare comments to the following questions in relation to their rankings. 1. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding SLO 1? 2. What concerns do you have after reading student papers? 3. What suggestions do you have for changes that could be made to improve students’ success at meeting the SLO? 1 3 4. After reading the students’ papers, do you feel that the method of assessment provided sufficient information for you to adequately assess how well student met this SLO? If not, what suggestions do you have for improving our method of assessment? The combination of ranked scores and the faculty reviewers’ comments will be shared at a Department Retreat and will form the foundation for discussion regarding ways of improving student success at meeting this SLO. Success with this SLO occurs when the majority of students (more than 60%) meet or exceed expectations. Assessment of Outcome 2: Demonstrate an understanding of scholarly sources of information (i.e. research published in peer- reviewed journals). In one randomly selected upper and one lower-division course in each program, students will be asked to provide written responses to the following questions: 1. Think about the last assignment that you completed in a (Sociology/CJ/Gero) course in which you were required to find and utilize scholarly sources. Please describe the types of sources that you used for the assignment. Be as detailed as possible. For instance, describe the types of books, journal articles, and/or internet sources you used. 2. What resources/ technologies (at MSUM or elsewhere) did you use to find and access the sources described in Question 1? 3. What is a peer-reviewed journal? In your answer, explain how it may differ from other sources of information such as books or webpages. Student responses to question 1 will be scored as follows: (a) listed only scholarly sources, (b) listed a mix of scholarly and nonscholarly sources, (c) listed only nonscholarly sources, or (d) type of sources could not be identified. We then summarized responses according to course level (upper or lowerdivision) For question 2, we compiled a list of types of sources utilized by students For question 3 the scored responses will be based on whether or not the student is able to clearly define the qualities of a peer-reviewed journal. Responses will then be summarized according to course level. Assessment of Outcome 3: Critique methods, theory and the work of other scholars. Final papers of 10 students in each of the following courses will be randomly selected: CJ 400, Soc 410, and Soc 450. Assessment committee member will rank each paper on a scale ranging from 1 (failed to meet expectations) to 10 (exceeds expectations). Assessment committee members will also prepare comments to the following questions in relation to their rankings. 2 4 1. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding the Student Learning Outcome (SLO)? 2. What are some concerns that you have after reading student papers? 3. What suggestions do you have for changes that could be made to improve students' success at meeting the SLO? 4. After reading the students' papers do you feel that the method of assessment provided sufficient information for you to adequately assess how well students met this SLO? If not, what suggestions do you have for improving our method of assessment? The combination of ranked scores and the faculty reviewers’ comments will be shared at the Department Retreat and will form the foundation for discussion regarding ways of improving student success at meeting this SLO. Success with this SLO occurs when a large majority of students (more than 60%) meet or exceed expectations. Assessment of Outcome 4: Conduct, interpret, present and write reports about social scientific research using qualitative and/or quantitative methodologies Final papers of 10 students in SOC 352 and 351 will be randomly select. Assessment committee members will rank each paper on a scale ranging from 1 (failed to meet expectations) to 10 (exceeds expectations). Assessment committee members will also prepare comments to the following questions in relation to their rankings. 1. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding the Student Learning Outcome (SLO)? 2. What are some concerns that you have after reading student papers? 3. What suggestions do you have for changes that could be made to improve students' success at meeting the SLO? 4. After reading the students' papers do you feel that the method of assessment provided sufficient information for you to adequately assess how well students met this SLO? If not, what suggestions do you have for improving our method of assessment? The combination of ranked scores and the faculty reviewers’ comments will be shared at the Department Retreat and will form the foundation for discussion regarding ways of improving student success at meeting this SLO. Success with this SLO occurs when a large majority of students (more than 60%) meet or exceed expectations. Assessment of Outcome 5: Demonstrate an understanding of the role of theory in social research. Final papers of students in each of the following courses will be selected: Sociology 350—Methods and Statistics for Social Research, Sociology 351—Quantitative Methods. Assessment committee members will rank each paper on a scale ranging from 1 (failed to meet expectations) to 10 (exceeds expectations). 3 5 Assessment committee members will also prepare comments to the following questions in relation to their rankings. 1. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding SLO 1? 2. What concerns do you have after reading student papers? 3. What suggestions do you have for changes that could be made to improve students’ success at meeting the SLO? 4. After reading the students’ papers, do you feel that the method of assessment provided sufficient information for you to adequately assess how well student met this SLO? If not, what suggestions do you have for improving our method of assessment? The combination of ranked scores and the faculty reviewers’ comments will be shared at a Department Retreat and will form the foundation for discussion regarding ways of improving student success at meeting this SLO. Success with this SLO occurs when a large majority of students (more than 60%) meet or exceed expectations. Assessment of Outcome 6: Explain current social phenomena as they relate to major theoretical traditions. Final papers of students in each of the following courses will be selected: Sociology 350—Methods and Statistics for Social Research Sociology 351—Quantitative Methods. Assessment committee members will rank each paper on a scale ranging from 1 (failed to meet expectations) to 10 (exceeds expectations). Assessment committee members will also prepare comments to the following questions in relation to their rankings. 1. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding SLO 1? 2. What concerns do you have after reading student papers? 3. What suggestions do you have for changes that could be made to improve students’ success at meeting the SLO? 4. After reading the students’ papers, do you feel that the method of assessment provided sufficient information for you to adequately assess how well student met this SLO? If not, what suggestions do you have for improving our method of assessment? The combination of ranked scores and the faculty reviewers’ comments will be shared at a Department Retreat and will form the foundation for discussion regarding ways of improving student success at meeting this SLO. Success with this SLO occurs when a large majority of students (more than 60%) meet or exceed expectations. 4 6 Department of Sociology and Criminal Justice Assessment Report 2011-2012 Student Learning Outcomes assessed in this cycle: Students completing a major in Sociology, Criminal Justice, and/or Gerontology should be able to: 1. Demonstrate knowledge of important theoretical perspectives in their discipline (Sociology, Criminal Justice, and/or Gerontology). 5. Demonstrate an understanding of the role of theory in social research. 6. Explain current social phenomena as they relate to major theoretical traditions. Methods of Assessment Assessment of Outcome 1: Demonstrate knowledge of important theoretical perspectives in their discipline (Sociology, Criminal Justice, and/or Gerontology). Final papers of students in each of the following courses were selected: Sociology 302—Social Theory (Hughey) Sociology 407—Contemporary Theory (Vigilant ) Department faculty reviewed the papers and prepared comments on the following questions: 5. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding SLO 1? 6. What concerns do you have after reading student papers? 7. What suggestions do you have for changes that could be made to improve students’ success at meeting the SLO? 8. After reading the students’ papers, do you feel that the method of assessment provided sufficient information for you to adequately assess how well student met this SLO? If not, what suggestions do you have for improving our method of assessment? Assessment Rubric for Student Learning Outcome 1 Demonstrate knowledge of important theoretical perspectives in their discipline (Sociology, Criminal Justice, and/or Gerontology). Totals (68): (2) 3 % Failed expectations, (23) 34% Met expectations, and (43) 63% Exceeded Reviewer Comments: Student Learning Outcome 1: Demonstrate knowledge of important theoretical perspectives in their discipline (Sociology, Criminal Justice, and/or Gerontology). 5 7 1. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding SLO 1? Soc 302: Most students met expectations. Soc 350: Overall, 90% of the student papers demonstrated an acceptable knowledge of important theoretical perspectives in their specific fields. Soc 351: Overall, 80% of the student papers demonstrated an acceptable knowledge of important theoretical perspectives in their specific fields. The one paper that did not meet expectations for this SLO simply did not provide a clear and cogent explanation of “social exchange theory” and “critical theory,” and it did not show the relevance of these theoretical traditions to the problem under study. Soc 407: All of the assessed student papers in Contemporary Theory demonstrated an acceptable knowledge of theoretical perspectives in the discipline, and this should come as no surprise since the course itself is a seminar on contemporary theories in sociology. Moreover, in this convenience sample of 10 student-papers, 80% of the students exceeded expectation in SLO number one by succinctly and cogently explicating the fundamental propositions of the theory under study. 2. What concerns do you have after reading student papers? Soc 302: I would like to see more students exceed expectations, but the distribution offers no surprises. This reflects a fairly typical distribution for an upper level class. Soc 350: The cited sources for a few theories were problematic because they were not drawn from primary (or academic) sources, but from the internet (i.e., paper number one employed www.teachnet.com to provide an explication of social norm and social learning theories). Soc 351: Although most students provided an adequate explication of listed theories, the majority of papers can improve by connecting the listed theories to the paper’s testable hypotheses and results. Soc 407: None. 3. What suggestions do you have for changes that could be made to improve students’ success at meeting the SLO? Soc 302: Students generally met the SLO in this class. I do not believe that significant changes are called for. Soc 350: N/A Soc 351: Students should provide cogent rationale for the use of extant theories in their papers, and show how these theories connect to their hypotheses. Soc 407: None. 4. After reading the students’ papers, do you feel that the method of assessment provided sufficient information for you to adequately assess how well student met this SLO? If not, what suggestions do you have for improving our method of assessment? Soc 302: The student papers consideration here are from a sociological theory class, so I would expect them to reflect that. Soc 350: Yes. Soc 351: Yes. Soc 407: Yes. 6 8 Assessment of Outcome 5: Demonstrate an understanding of the role of theory in social research. Final papers of students in each of the following courses were selected: Sociology 350—Methods and Statistics for Social Research (White, Fall 2011) Sociology 351—Quantitative Methods (White, Fall 2011) Department faculty reviewed the papers and prepared comments on the following questions: 5. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding SLO 1? 6. What concerns do you have after reading student papers? 7. What suggestions do you have for changes that could be made to improve students’ success at meeting the SLO? 8. After reading the students’ papers, do you feel that the method of assessment provided sufficient information for you to adequately assess how well student met this SLO? If not, what suggestions do you have for improving our method of assessment? Assessment Rubric for Student Learning Outcome 5 Demonstrate an understanding of the role of theory in social research. Totals (15): (2)13% Failed expectations, (7) 47% Met expectations, (6) 40% Exceeded REVIEWER COMMENTS Student Learning Outcome 5: Demonstrate an understanding of the role of theory in social research. 1. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding SLO 5? Soc 350: Overall, 90% of the student-papers demonstrated an acceptable knowledge of the role of theory in social research, and of this percentage, 40% of the students exceeded expectation in this area by clearly and cogently connecting extant sociological, gerontological, and criminological theories to their testable hypotheses. A few papers failed to demonstrate a clear and strong connection between existing theories and testable hypotheses. Soc 351: Overall, 80% of the student-papers demonstrated an acceptable knowledge of the important role of theory in social research, and of this percentage, half the students exceeded expectations in this area by connecting extant sociological, gerontological, and criminological theories to their testable hypotheses. 2. What concerns do you have after reading student papers? Soc 350: N/A Soc 351: N/A 3. What suggestions do you have for changes that could be made to improve students’ success at meeting the SLO? Soc 350: N/A 7 9 Soc 351: It is standard practice in the discussion section of research reports to connect the findings of one’s research to the findings of other extant studies and/or existing theories. In essence, students should explain how existing theories might shed light on the findings of their research, and, at a minimum, whether their findings support the theories that are listed in their theory section. 4. After reading the students’ papers, do you feel that the method of assessment provided sufficient information for you to adequately assess how well student met this SLO? If not, what suggestions do you have for improving our method of assessment? Soc 350: Yes. Soc 351: Yes. 8 10 Assessment of Outcome 6: Explain current social phenomena as they relate to major theoretical traditions. Final papers of students in each of the following courses were selected: Sociology 350—Methods and Statistics for Social Research (White, Fall 2011) Sociology 351—Quantitative Methods (White, Fall 2011) Department faculty reviewed the papers and prepared comments on the following questions: 5. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding SLO 6? 6. What concerns do you have after reading student papers? 7. What suggestions do you have for changes that could be made to improve students’ success at meeting the SLO? 8. After reading the students’ papers, do you feel that the method of assessment provided sufficient information for you to adequately assess how well student met this SLO? If not, what suggestions do you have for improving our method of assessment? Faculty reviewer comments were shared at a department meeting on April 18th, 2012 and have formed the foundation for discussion regarding ways of improving student success at meeting this SLO. Assessment Rubric for Student Learning Outcome 6 Explain current social phenomena as they relate to major theoretical traditions. Totals (15):(3) 20% Failed expectations, (7) 47% Met expectations and (5) 33% Exceeded REVIEWER COMMENTS Student Learning Outcome 6: Demonstrate knowledge of important theoretical perspectives in their discipline (Sociology, Criminal Justice, and/or Gerontology). 1. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding SLO 6? Soc 350: Overall, 80% of the student-papers provided an acceptable explication of how the social phenomenon under investigation was related to a major theoretical perspective/tradition, and of this percentage, 50% of the students exceeded expectation under this SLO. Soc 351: Overall, 80% of the student-papers demonstrated acceptable knowledge of an important theoretical perspective/tradition in their respective discipline. 9 11 2. What concerns do you have after reading student papers? Soc 350: The two papers that did not meet expectations under SLO number six did not demonstrate a clear and cogent understanding of the theoretical perspectives to begin with, so these papers did not connect theories to the phenomena (smoking in the elderly population for paper number one and video games and adolescent violence for paper number four). In the case of the latter paper, the “theory section” of the manuscript consisted of a simple statement that “People consider the violent video games to be the cause of the negative effects on the person playing, while non-violent video games can stimulate and educate a person in a positive way [sic].” Soc 351: There is a dearth of discussion on how extant theories on the problem under study connect to the findings of the research in the discussion section of the paper. 3. What suggestions do you have for changes that could be made to improve students’ success at meeting the SLO? Soc 350: In the two papers highlighted above, it appears that these students did not go into the assignment with a working understanding of what a proper theoretical statement is, and how a theoretical statement might be connected to a theoretical tradition/perspective. The fact that 80% of the papers met or exceeded this SLO might suggest that these students may not have taken Sociology 302 (Classical Sociological Theory). Soc 351: It would be a good idea for students to connect their listed theories to the findings from their testable hypotheses in the discussion section. Too often, students simply define the theory at the beginning of the paper without making any subsequent connection to their results. Suggestion: It is standard practice in the discussion section of research reports to connect the findings of one’s research to the findings of other extant studies and/or existing theories. In essence, students should explain how existing theories might shed light on the findings of their research, and whether their findings support the theories that are listed in their theory section. 4. After reading the students’ papers, do you feel that the method of assessment provided sufficient information for you to adequately assess how well student met this SLO? If not, what suggestions do you have for improving our method of assessment? Soc 350: Yes. Soc 351: Yes. Department Comments: The measure of a successful SLO is whether or not at least 60% of students met or exceeded expectations. It is clear from the data that this occurred with all SLO’s being assessed. For SLO 1; 97% of students met or exceeded expectations. Concerning SLO 5; 87% of students met or exceeded expectations. With regard to SLO 6; 80% of students met or exceeded expectations. The department reviewed the assessment findings and report and made the following comments. During the department meeting specifically for review of the assessment report the following discussion occurred. Vigilant observed that students have trouble connecting theories to test findings, and that the department should develop ways to teach theory construction to 10 12 Sociology majors. He would like to emphasize the sociology of knowledge in future hires. White noted the increase in majors and student enthusiasm in recent years. She asked faculty to imagine a “dream curriculum” for all three of the department’s programs. Powell asserted that teaching sociology as a habit of thinking could be more effective than trying to “cover the material” in various sociology classes. The Department agreed that curriculum should be a major consideration at the next retreat, and that connections between theory and method will be germane as assessment data have indicated this area for improvement. 11 13 Department of Sociology and Criminal Justice Assessment Report 2010-2011 Assessment of Outcome 3: Critique methods, theory and the work of other scholars. Final papers of 10 students in each of the following courses will be randomly: CJ 400 (Hendrix-Sloan, SP 10), Soc 410 (Humphers-Ginther, SP 10), and Soc 450 (White, SP 10). Names of students will be removed and copies will be provided to all department faculty (to the assessment committee). The Soc 410 papers from spring 2010 were misplaced in the move from Holmquist to Lommen and will be replaced with Soc 410 papers from spring 2011. An assessment questionnaire about expectations was completed by each assessor prior to completing the rubric for each paper. An assessment rubric was used for each SLO. Department faculty will review the papers and prepare comments to the following questions. 5. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding the Student Learning Outcome (SLO)? 6. What are some concerns that you have after reading student papers? 7. What suggestions do you have for changes that could be made to improve students' success at meeting the SLO? 8. After reading the students' papers do you feel that the method of assessment provided sufficient information for you to adequately assess how well students met this SLO? If not, what suggestions do you have for improving our method of assessment? Faculty comments will be shared at the Fall department retreat and will form the foundation for discussion regarding ways of improving student success at meeting this SLO. 12 14 Assessment Rubric for Learning Outcome 3 Class: CJ 400 and Soc 450___ Paper: 10 randomly selected totaling 20_________ Critique of Method Fails to meet expectations 1 2 Meets Expectations Exceeds Expectations 3 4 5 6 7 8 9 CJ 400 Scores 3 10 5 4 3 2 Percentages 10% Failed Soc 450 Scores 1 Precentages 3% Failed 50% Meet 11 10 3 (30) 40% Exceed 5 1 5 57% Meet 2 5 (30) 40% Exceed Critique of Theory Fails to meet expectations 1 2 Meets Expectations Exceeds Expectations 3 4 5 6 7 8 CJ 400 Scores 4 9 4 2 1 Percentages 20% Failed Soc 450 Scores 65% Meet 8 Percentages 0% Failed 8 9 10 (20) 15% Exceed 2 5 60% Meet 3 4 (30) 40% Exceed *Note: One reviewer felt the critique of theory was not part of these papers and did not score this section. Critique of the Work of Scholars Fails to meet expectations 1 2 CJ 400 Scores 3 Meets Expectations Exceeds Expectations 4 5 6 7 8 9 2 9 7 3 3 2 Percentages 6.8% Failed Soc 450 Scores 4 Percentages 13% Failed 53% Meet 6 1 1 33% Meet 13 15 10 3 (30) 37% Exceed 2 7 4 53% Exceed 5 (30) Areas of concern for this paper? Noticeable that it is not a writing intensive course…the paper and the sociology gets better. Exceeds expectations when students can begin to identify gaps in research. Direct quotes (maybe these should be limited) and the reiteration of what authors say were too numerous. Bibliography resources were not cited. What should students be doing? Students not forced to produce their own ideas. Plaigerism…with the literature reviews. Suggestions for needed (instructional) improvements prompted by this paper Literature reviews should be aimed at a social problem. Paper number 5 “Female Drug Dealers” was also paper #8 for soc 450. This paper also had Wikipedia sources not sited. Students should include gaps in research….some did but others did not. The papers do not critique very well. They discuss the theories but do not identify the weak aspects of the theories under discussion. Having students do a literature review opens the door to plagerism. Deficiencies in assessment method suggested by this paper 1. The assignment for the Soc 400 papers were not theoretically focused so one reviewer felt they could not be scored which is why there is a total of 20 instead of 30 under the critique theory section of the rubric. 2. We should get the assignment criteria so we are able to use them to help guide assessment. 3. Our assessment tool should include something positive about the papers. 4. Some students chose topics and focused their attention on individual problems/issues rather than sociological ones. 5. Professors should be congratulated that the overwhelming majority of scores were met or exceeded expectations. Even though it was not part of the tool all assessors felt compelled to make positive comments about the papers. 14 16 Assessment of Outcome 4: Conduct, interpret, present and write reports about social scientific research using qualitative and/or quantitative methodologies Final papers of 10 students in SOC 352 and 351 during Fall 2010 will be randomly select. Names of students will be removed and copies will be provided to all department faculty Department faculty will review the papers and prepare comments to the following questions. 5. Overall, do you feel that most students met, exceeded, or failed to meet your expectations regarding the Student Learning Outcome (SLO)? 6. What are some concerns that you have after reading student papers? 7. What suggestions do you have for changes that could be made to improve students' success at meeting the SLO? 8. After reading the students' papers do you feel that the method of assessment provided sufficient information for you to adequately assess how well students met this SLO? If not, what suggestions do you have for improving our method of assessment? Faculty comments will be shared at the Fall department retreat and will form the foundation for discussion regarding ways of improving student success at meeting this SLO. 15 17 Assessment Rubric for Learning Outcome 4 Class___Soc 351 and Soc 352__ Paper__10 randomly selected from each course totaling 20 Summary of results: The majority of scores were in the meets or exceeds expectation section of the scale. Soc 351 scores, totals and percentages are in bold and Soc 352 scores, totals and percentages are identified in italics. Conduct Research Fails to meet expectations 1 2 Meets Expectations 3 4 5 Soc 351 Scores Total 1 6 Percentages 3% Failed Soc 352 Scores Total 6 7 8 9 10 8 4 3 2 4 (30) 53% Meet 5 Percentages 2 Exceeds Expectations 0% Failed 1 4 43% Exceed 2 7 40% Meet 1 1 9 (30) 60% Exceed Interprets Results Fails to meet expectations 1 2 3 Meets Expectations 4 Soc 351 Scores Total 1 Percentages 10% Failed Soc 352 Scores Total Percentages 2 2 7% Failed Exceeds Expectations 5 6 7 5 8 8 43% Meet 3 5 30% Meet 16 18 8 9 10 2 4 (30) 47% Exceed 1 7 1 1 2 63% Exceed 8 (30) Writes Reports Fails to meet expectations Meets Expectations Exceeds Expectations 1 5 7 2 3 4 Soc 351 Scores Total 1 1 Percentages 17% Failed 3 Soc 352 Scores Total Percentages 6 2 7 2 1 57% Meet 1 0% Failed 6 1 7 1 8 9 10 2 2 2 (30) 26% Exceed 3 30% Meet 1 3 5 1 8 (30) 70% Exceed Areas of concern for this paper? There were simple straight forward hypotheses that were less interesting and less complex than their literature reviews. Suggestions for (instructional) improvements prompted by this paper? Students struggle with smoothing out their papers. Some of the papers did a good job of situating the problem but many did not. Many papers omit a history of the problem. There were obvious normative biases in 352. Some of the papers are low on reflexivity. Some do a good job discussing the method while others do not mention it. Deficiencies in assessment method suggested by this paper? THESE ARE NOT PROBLEMS ISOLATED TO THESE CLASSES. THESE ARE PROBLEMS OF EDUCATION THAT EVERYONE HAS TO DEAL WITH FIXING. 1. Doesn’t get at why students can’t smooth out their papers. 2. The assessment tool should Identify the positives and/or strengths of the papers as well as the weaknesses. 3. Some of the positive comments made by assessors: Some of them did a good job of articulating the social context of the problem. Many of the writers used their own words rather than rely on quotations and ideas of scholars. It is really good that they have to explain the data sets and they did well with it. The papers are overwhelmingly acceptable or exceeding expectations. 17 19 4. These papers were scored lower because they were more ambitious, not less. Promises were made in the introduction that were not lived out in the paper. Can the miniethnographies comfortable rest here for assessment? For this Group of Papers: If you have an ideal paper, or body of work, or group of students to which you are comparing these papers, describe. Soc 351 and 352: 1. The idea of average undergraduate writing over the last 20 years. 2. Seniors in the discipline. Soc 450 and CJ 400: 1. Entry level graduate students writing for first publication (MA chapter prior to completion). 2. Seniors in the discipline. What are your criteria for failing to meet expectations? (Conducting research, interpreting results, writing reports) Soc351 and 352: 1. Research was not conducted (citations absent), scholarship (e.g. blogs, wiki), methods not understood. 2. Fails to connect results with any sociological explanation or fails to seat results in a sociological framework. 3. Does not present in organized and coherent fashion. 4. Inability to write clearly and put ideas together in a smooth fashion. 5. Inability to understand peer reviewed journal articles and use them to discuss research. Soc450 and CJ400: 1. Adopts method without understanding. 2. Recites theory without understanding relationships to research. 3. Cites work only by topic. Does not understand/discriminate legitimate sources from nonlegitimate. What are your criteria for meeting expectations? 18 20 Soc 351 and 352: 1. The research site is understood, the method is understood, some sense of blind spots of the research. 2. Observation go beyond “mere” description of setting. 3. Paper is organized and coherent. Soc 450 and CJ400: 1. Sees an applicable method and recognizes advantages. 2. Sees an applicable theory and explains the value of the perspective to the topic. 3. Selects appropriate work to the topic. What are your criteria for exceeding expectations? Soc 351 and 352: 1. The research site is creative and productive. The observation are thorough and emerge from a sense of method. 2. Connections made to other sites, frames of reference, studies or social theory. 3. Superior writing, organization, expression of the importance of findings to social science. 4. Where appropriate, successfully problematizes prior research OR introduces an interesting, underserved question. 5. Creativity and uniqueness beyond what they are learning. Soc450 and CJ400: 1. Recognizes and applies the contemporary research best fitting the theoretical perspective. 2. Understands and explains the advantages and limitations of method. 3. Understands and explains connections of theory and method. 4. Identifies unifying threads in applicable scholarship. 5. Relates more than one theoretical perspective in terms of vital points of articulation. E.g. Symbolic Interaction and Marxism both contribute because of ________. 6. Literature review is well-integrated/connected research topic. 19 21 Spring 2009 Assessment of Student Learning Outcome 2. Demonstrate an understanding of scholarly sources of information Assessment Committee Summary of Assessment Data Prepared by Deb White, Sue Humphers-Ginther, and Geri Hendrix-Sloan Think about the last assignment that you completed in a Criminal Justice, Gerontology, or Sociology course in which you were required to find and utilize scholarly sources. Please describe the types of sources that you used for the assignment. Be as detailed as possible. For instance, describe the types of books, journal articles, and/or internet sources you used. Lower division courses Percent of students who named only scholarly sources = 71.4% Percent of students who named scholarly and other sources = 27.5% Percent of students for whom we could not determine whether they used scholarly or other sources = 1.1% N = 91 Upper division courses Percent of students who named only scholarly sources = 60% Percent of students who named scholarly and other sources = 20% Percent of students who named only non-scholarly sources: 6.7% Percent of students who named no sources: 13.3% Percent of students for whom we could not determine whether they used scholarly or other sources = 0% N = 30 What is a peer-reviewed journal? In your answer, explain how it may differ from other sources of information such as books or web pages. Lower division courses Percent of student who clearly defined the qualities of a peer-reviewed journal = 53.1% Percent of student who did not clearly defined the qualities of a peer-reviewed journal = 46.9% N = 81 Upper division courses Percent of student who clearly defined the qualities of a peer-reviewed journal = 41.4% Percent of student who did not clearly defined the qualities of a peer-reviewed journal = 58.6% N = 29 20 22