Minnesota State University Moorhead 2007-2008 Assessment Report Cover Sheet (

Minnesota State University Moorhead
2007-2008 Assessment Report Cover Sheet
(An electronic version of this form can be accessed at http://www.mnstate.edu/assess)
Note: All non-accredited programs are required to complete this form. Include Assessment
Reporting Forms for each learning outcome assessed.
Academic Program: Bachelor of Science, Mass Communications
Department: Mass Communications
College: College of Business and Industry
Date: October 19, 2007
1. Name(s) of Department Assessment Coordinator and/or Assessment Committee
Members
Jody Mattern, Mark Strand, Dan Johnson, Martin Grindeland, CT Hanson
2. List of All Student Learning Outcomes. (List all outcomes, placing an asterisk (*) by the
outcomes you are assessing this year.)
1. Recognize the theoretical concepts in one's chosen emphasis.
2. Demonstrate an understanding of the ethics governing one's chosen emphasis.
3. Demonstrate verbal competency in one's chosen emphasis.
4. Demonstrate visual competency in one's chosen emphasis.
5. *Demonstrate ability to work collaboratively.
6. *Demonstrate an ability to create a professional document that meets industry standards.
7. *Demonstrate entry level competence inone's chosen emphasis through successful completion
of an internship or other professional exeprience.
3. Describe how your program has addressed the comments from the Student Learning
Outcomes Assessment Committee during the past two academic years? (If you have made
changes to your plan, file a revised Assessment Plan Cover Sheet and Assessment Planning
Form(s).)
The assessment committee suggested that we provide more specific criteria for the different
categories in the rubric. We revised the internship rubric by including category clarifications and
adding two questions to better follow our student learning outcomes. We revised the teamwork
rubric by adding category clarifications related to amount of time a specific behavior occurred
(some of the time, none of the time, always, never, etc.). These changes were not in place for our
assessment measures for 2006-07, but are in place for future assessments.
4. If you have received an Instructional Improvement Grant in the past two years, identify
the outcomes on which the grant was based and provide a summary here of the results
from your grant.
Instructional Improvement Grant Report
Jody Mattern
Learning Outcome Addressed: Demonstrate an ability to create a professional document that
meets industry standards.
The instructional improvement grant provided funding for ten duty days for the development of a
portfolio class. The description of the project from the grant application follows.
We proposed development of a senior level class for advertising and public relations students
called “Senior Portfolio.” The class has a dual function: 1) to act as a departmental assessment
tool for the learning outcome related to creating professional documents, and 2) to help students
develop a professional-level portfolio for use in job searches after graduation.
During the summer of 2006, I conducted 11 interviews with advertising professionals at local,
regional and national levels. I also reviewed several books for possible use as textbooks and class
resource materials. After developing separate evaluation rubrics for copywriters, graphic
designers, and art directors, I worked with the instructional technology department to develop
special software for use in this class. With this software, I am able to upload student projects,
pair them with a selected evaluator, and send them off in the form of an email.
I also recruited a network of 25 evaluators to evaluate portfolio materials and maintained contact
with them throughout the semester. During the semester, a total of 126 evaluations were
solicited.
After the semester has ended, the average score of all projects evaluated was tabulated.This score
will be compared from year to year to track changes. I also asked evaluators to indicate overall
weaknesses they saw in the student work. I will compile these comments and share with others in
the department who teach courses in the advertising emphasis. For example, several evaluators
have indicated that the typography is weak in many of the pieces they see. I have shared this with
the two faculty members who teach layout and typography and they are exploring ways of
including more work with typography in their courses.
I have included some materials to document what is stated in this report including examples of
the interview questions and answers, a list of evaluators, screen shots of the evaluation software,
examples of evaluation rubrics, and a list of the projects sent out.
5. Signatures
Department Chair or Program Director
Dean or Director
Required Attachments:
1. Assessment Reporting Forms
2. Records of department meetings when Assessment Report was discussed and approved.
Minnesota State University Moorhead
2007-2008 Assessment Reporting Form
(An electronic version of this form can be accessed at http://www.mnstate.edu/assess)
Instructions: Include this form for each student learning outcome assessed during the previous
year. Include Assessment Report Cover Sheet.
Academic Program: Bachelor of Science, Mass Communications
1. Learning outcome assessed (please include the number of the outcome to correspond
with the list on the cover sheet).
5. Demonstrate ability to work collaboratively.
2. Describe assessment measure used for this learning outcome (attach instrument or
rubric)
A teamwork rubric was administered to CT Hanson's classes. Approximately 70 students
completed the evaluation of others in their group. Results were tabulated and averages obtained
for each criteria measured. See Appendix D for results.
3. Expected/satisfactory student results (from assessment plan)
The department expected 80% of its student interns to rate at least 4 on a 5-point scale.
4. Actual results from the past year (attach additional information, if necessary)
The overall average score from fall 06 and spring 07 was 4.6, which exceeds our expectations.
See Appendix D for a breakdown of results by question.
5. Describe and explain available trend data for student performance on this outcome over
the past several years. In other words, describe how the results of this measure have
changed over the past several years.
Since this was the first year we tracked this measure, we will use this figure as a baseline to track
trends in future years.
6. Proposed action in response to results. (Please note if improvements can be made with
existing department resources. If improvements cannot be made with existing department
resources, consider applying for an Instructional Improvement Grant.)
We are satisfied with these results and have determined that no additional steps to improve
scores for this learning outcome are needed other than to revise the rubric to better clarify
categories.
Minnesota State University Moorhead
2007-2008 Assessment Reporting Form
(An electronic version of this form can be accessed at http://www.mnstate.edu/assess)
Instructions: Include this form for each student learning outcome assessed during the previous
year. Include Assessment Report Cover Sheet.
Academic Program: Bachelor of Science, Mass Communications.
1. Learning outcome assessed (please include the number of the outcome to correspond
with the list on the cover sheet).
6. Demonstrate an ability to create a professional document that meets industry standards
2. Describe assessment measure used for this learning outcome (attach instrument or
rubric)
An online rubric was developed that showed a portfolio piece and asked professionals to evaluate
the piece compared to other junior-level portfolio pieces they may have evaluated in the past (see
Attachment E for evaluation rubric).
Two assessments were evaluated.
One was the overall average of scores.
The second was the overall number of "yes" answers when evaluators wee asked if this piece
was of sufficient quality to be a portfolio piece.
3. Expected/satisfactory student results (from assessment plan)
We expected 90% of our student work to receive average or better scores (3.0 or better).
We expected that 50% of the pieces would be considered portfolio quality.
4. Actual results from the past year (attach additional information, if necessary)
The average of all ad evalautions was 3.13. The percentage of ad portfolio pieces deemed to be
of sufficient quality was 66%. The average of all graphic design evaluations was 3.63. The
percentage of graphic design portfolio pieces deemed to be of sufficient quality for a portfolio
was 74%. See Appendix E for a breadown of scores in all categories.
5. Describe and explain available trend data for student performance on this outcome over
the past several years. In other words, describe how the results of this measure have
changed over the past several years.
Since this is the first year, we used this evaluation, these figures will serve as a baseline for
future activities.
6. Proposed action in response to results. (Please note if improvements can be made with
existing department resources. If improvements cannot be made with existing department
resources, consider applying for an Instructional Improvement Grant.)
This program was implemented as a pilot program. It is our goal to develop outside evaluation
processes for other areas of emphasis so that each area has a consistent assessment procedure
using outside evaluators.
The instructor will pay closer attention to copywriting since that score was the one area where
we failed to achieve an average or above score. Also, the comments indicated an overall
weakness in typography. This fact has been shared with the faculty who teach design courses.
We are also exploring the possibility of bringing in an expert in typography to speak to design
students.
Minnesota State University Moorhead
2007-2008 Assessment Reporting Form
(An electronic version of this form can be accessed at http://www.mnstate.edu/assess)
Instructions: Include this form for each student learning outcome assessed during the previous
year. Include Assessment Report Cover Sheet.
Academic Program: Bachelor of Science, Mass Communications
1. Learning outcome assessed (please include the number of the outcome to correspond
with the list on the cover sheet).
7. Demonstrate entry level competence in one's chosen emphasis through successful
completion of an internship or other professional experience.
2. Describe assessment measure used for this learning outcome (attach instrument or
rubric)
A 20-question evaluation rubric is completed by the intern's supervisor at the company where the
intern is serving (see Attachment B).
3. Expected/satisfactory student results (from assessment plan)
The department expects 80% of its student interns to rate at least 4 on a 5-point scale.
4. Actual results from the past year (attach additional information, if necessary)
The overall average for all questions was 4.39 (see Attachement F for breakdown by question).
None of the individual averages for questions fell below the 4.00 (excellent) average.
5. Describe and explain available trend data for student performance on this outcome over
the past several years. In other words, describe how the results of this measure have
changed over the past several years.
Since this is the first year we have officially tracked these numbers, this year will serve as a
baseline for future trend analysis.
6. Proposed action in response to results. (Please note if improvements can be made with
existing department resources. If improvements cannot be made with existing department
resources, consider applying for an Instructional Improvement Grant.)
Since the average exceeded our departmental goal, no further action will be taken at this time
except for a revision of the rubric to better clarify categories.
Appendix D
Teamwork Evaluation
Fall/Spring 2006-07
Rating Scale:
5 = Superior
4 = Excellent
3 = Good
2 = Fair
1 = Poor
Total Students Evaluated: 69
Question
Average
Team Oriented
4.7
Shared in Workload
4.5
Reliable
4.5
Cooperative
4.6
Supportive of Others
4.7
Overall Average
4.6
Appendix E: Results, Professional Documents
MC 490 Ad Portfolio Development
Evaluations
Semester: Spring 2007
Rating Scale: 5 = Significantly Above Average
4 = Slightly Above Average
3 = Average
2 = Slightly Below Average
1 = Significantly Below Average
Ads
Number of Evaluations: 61
Question
Average
1)
Demonstrates Conceptual Thinking (creativity)
3.39
2)
Layout
3.10
3)
Headline/Body Copy
2.87
4)
Overall Impact
3.15
5)
In your opinion, is this piece of sufficient qualityto be included in a portfolio?
Yes 66% No 34%
Overall average for ads:
3.13
Graphic Design
Number of Evaluations: 39
Question
Average
1)
Composition
4.31
2)
Typography
3.26
3)
Demonstrates Conceptual Thinking (creativity)
3.56
4)
Use of Color
3.38
5)
In your opinion, is this piece of sufficient quality to be included in a portfolio?
Yes 74% No 26%
Overall average for graphic design
3.63
Attachment F – Internship Results
Intern Supervisor Evaluations Summer 2006 - Spring 2007Rating Scale: [5
Superior, 4 Excellent, 3 Average, 2 Fair, 1 Poor]
No. Question
Ave Mode Range N/A
1.
Grasps professional situation
4.33 4
3-5
1
2.
Displays technical knowledge
4.25 5
3-5
0
3.
Demonstrates an ability to organize
4.31 4
3-5
0
4.
Takes responsibility for tasks
4.42 5
2-5
0
5.
Requires minimal supervision once the task is explained
4.38 5
3-5
1
6.
Completes tasks in a professional manner
4.46 4
3-5
0
7.
Meets deadlines
4.32 5
3-5
0
8.
Demonstrates initiative
4.15 4
3-5
0
9.
Accepts constructive criticism
4.39 5
2-5
0
10.
Takes pride in assigned work
4.54 5
3-5
0
11.
Uses good judgment
4.29 4
2-5
0
12.
Exhibits adaptability
4.46 5
3-5
0
13.
Demonstrates an ability to work as a member of a team
4.67 5
2-5
1
14.
Exhibits maturity
4.31 4
3-5
0
15.
Exhibits good listening skills
4.37 4
3-5
0
16.
Exhibits good written communication skills
4.16 4
2-5
3
17.
Exhibits good oral communication skills
4.31 4
2-5
0
18.
Exhibits professional demeanor with clients
4.47 5
3-5
2
19.
Respects workplace colleagues
4.68 5
4-5
0
20.
Exhibits positive personal decorum
4.58 5
3-5
0
Total:
4.39 5
2-5
8
Assessment Committee Meeting
Agenda
September 21, 2007
2:15 p.m.
1. Discuss revisions 2006-2007 plan.
2. Discuss report for 2006-2007
3. Discuss 2007-2008 Plan
a. Expand internship evaluation
b. Writing pre-post test
c. Evaluation day
d. Exit survey
Mass Communications
Departmental Meeting
October 5, 2007
Present: Mark Strand, chairperson: Martin Grindeland, Wayne Gudmundson, Shelton
Gunaratne, Bill Hall, CT Hanson, Dan Johnson, Jody Mattern, Aaron Quanbeck, Regene
Radniecki, Camilla Wilson
1)
Jim Anderson, Assistant Director Admissions and Shawn Soderberg, Assistant Registrar
joined our meeting to review MSUM’s Dragon Core requirement and eservices tools.
2)
Assessment reports are due at the end of the month. Jody is coordinating our
departmental assessment plan and also represents the College of Business and Industry on
the university’s assessment committee. She distributed to the faculty the attached
assessment materials and noted that for 2006-07 the department has met its goals for the
following student learning outcomes:
Demonstrate ability to work collaboratively.
Demonstrate the ability to create a professional document that meets industry standards.
Demonstrate entry-level competence in one’s chosen emphasis through successful
completion of an internship or other professional experience.
Following review and discussion of the attached 2007-2008 assessment plan, Mark
moved to submit the plan to the university assessment committee. Martin seconded the
motion. Motion carried.
3)
Announcements:
a)
Bob Mikkelson will be in town and meeting with classes on October 8th and 9th.
b)
Faculty development and Dille grant applications are due October 31st.
c)
The Red River Interscholastic Press Association (RRIPA) fall workshop is
scheduled for Thursday, Nov. 1st in the CMU Ballroom.
d)
Jody noted that she is reading Rebekah Nathan’s book “My Freshmen Year:
What a Professor Learned” which focuses on actual experiences of students.
Anyone wishing to read the book can borrow it from her.
e)
Recycled computer requests are due to the Student Technology Fee Committee by
October 17th for first consideration, however, there are no macs available at this
time. C.T. indicated that he would like to request a pc for use with the perception
analyzer.
Respectfully submitted,
Deb Hval