General
What do we hope to accomplish in assessing student learning at Kalamazoo College?
- We approach this work with the confident stance that outstanding student learning is happening in every unit and department at K College. Assessment gives a structure for telling that story of learning success to all our constituencies.
Is this something new at K?
- No. We’ve been talking intentionally about assessment for at least 20 years — an Assessment Committee in some form has been around since 1999.
- At a more fundamental level, all of us at K engage in assessment of student learning every day: Whenever we notice something in students’ work or behavior, use that observation to form a judgment or make a decision about something we want to change to improve student learning or performance, and then continue to pay attention to see if the change had any effect on the original observation.
- By “assessment”, we have in mind that this natural process of observation and action can be planned, documented, and shared in processes that fit existing workflows.
- We approach this work with the confident stance that outstanding student learning is happening in every unit and department at K College. Assessment gives a structure for telling that story of learning success to all our constituencies.
What is an assessment plan?
- An assessment plan consists of four parts, each of which is described in more detail later in this FAQ:
- a set of learning outcomes that reflect the values of your program.
- the methods you’ll use to assess student learning relative to those outcomes
- a timetable for the use of those methods
- plan for how the information you gain from those methods will be compiled, distributed, discussed and used to make improvements in your program.
- A helpful way to think about an assessment plan is a script that your unit writes for its own use in the coming years. As one year follows the next the people and roles in your unit may change. The assessment plan spells out what is needed without having to re-invent the process each time.
What is the Institutional Assessment Committee?
- Learn more about the Institutional Assessment Committee
Learning Outcomes
What are learning outcomes?
- Assessment begins with stated learning outcomes. A place to start with learning outcomes is to ask, “what behaviors, traits, skills, and aptitudes do we want to see in our students as they finish their time in a program/department/unit?”
- K College has four Institutional Learning outcomes (ILOs): Communicate Effectively, Address Complex Problems, Collaborate Successfully, Demonstrate Intercultural Competence. Your program, department or unit will have additional learning outcomes specific to your area.
- As with the ILOs, the learning outcomes in your area will likely have areas of overlap between them — as an example, communication is an important part of collaboration.
- As you think about the learning outcomes specific to your area, we ask you to make explicit the connections between them and the four ILOs.
- In the Assessment Academy Team, we’ve found it extremely useful take a developmental approach to the learning outcomes—as students progress from first year to senior, we expect that their levels of success in the ILO areas will increase.
Assessment Methods
What are assessment methods?
- Assessment methods provide examples, evidence and actionable information about how students are making progress in the stated learning outcomes. That information could be in the form of student work samples chosen to match one or more learning outcomes, student survey responses to questions or prompts that intentionally address stated learning outcomes, scores on items from an exam chosen to align with stated learning outcomes.
- Some important practical considerations for workable assessment methods
- Ideally the methods can be built into existing annual structures in your program: student work from selected required course(s) in a major, a survey or other activity at the time of major declaration (perhaps as part of a required major course), a senior survey or exit interview, senior seminar, comprehensive exam
- It will be most helpful when you think about an assessment method to also think of specific success indicators that you’ll look for as part of each method.
What are success indicators?
- Success indicators are simply the things you observe in students and their work products that indicate the degree to which they have demonstrated the desired behaviors, traits, skills and aptitudes in your list of learning outcomes.
- In the Assessment Academy Team, we have found it helpful to take a development approach to thinking about success indicators. A new major in a program would be expected to exhibit some degree of success in a particular learning outcome, and a greater degree of success at the time of graduation.
- Success indicators toward a learning outcome are not the same thing as a course grade, or even a grade on a given assignment. We don’t hold first year students to the same standard as graduating seniors.
- To illustrate that idea, a student might receive an A in a first-year course by demonstrating success indicators appropriate to first-year developmental level.
What is an assessment rubric?
- Success indicators for a learning outcome, arranged progressively according to development throughout a student’s time at K.
- The Assessment Academy Team has created assessment rubrics for the four ILOs to use as they are or as a guide to constructing discipline-specific rubrics. Links to all four rubrics can be found on the Institutional Learning Outcomes page.
Does an assessment method need to involve the design of a novel, validated instrument with a sample of student work that assures statistical significance? That sounds like a PhD dissertation!
- No!
- It’s helpful to think about assessment methods simply as ways of collecting examples of student work or feedback that we can use to tell the story of how students are making progress toward learning outcomes we’ve stated.
What are some examples of workable assessment methods?
- An example of an assessment method that is directly related to a course would be to collect representative student work samples or survey responses from a particular question or activity that addresses a learning outcome and read that work with some agreed-upon success indicators in mind.
- A senior survey: As part of a required senior-level course in your program, you might administer a survey with questions that are linked directly to your learning outcomes, such as “The Mathematics program has a goal that all its students will be able to read, understand, and construct logically rigorous proofs. Please write in the space below how your experiences in the mathematics program have helped you progress toward that goal.” This assessment method has the additional benefit of developing students’ meta-cognitive awareness of our programs’ learning goals and their progress in those areas.
- An example: For a given learning outcome your program will create a collection of success indicators —what are you looking for to indicate student success? An assessment method could be as simple as collecting short representative work samples from an assignment in a course or collection of courses that capture most students as they progress in the program. Your success indicators (perhaps arranged in a rubric) will guide you in drawing conclusions about how these work samples illustrate students’ success or progress. Perhaps you will make comparisons for individual students at those two stages, or you could make general comparisons about the earlier group and the later group.
- Asking a fundamental question at several stages of students’ progress in a program: Coordinate across the department to collect short representative work samples from an assignment in a course early in the major program and again in a senior level course. This sample of student work can be read with agreed-upon success indicators in mind to answer the question “Are students in the program progressing toward stated learning outcomes?”
- Class activities and components are often designed and included with specific learning goals in mind. A natural assessment method would be one that focuses on the connection between the activity and the learning outcome, either by asking students to reflect on that connection or by assigning students’ work in that activity with an aspect of the that can be read with success indicators in mind to answer the question “to what degree did this activity develop students toward the stated learning outcomes?”. For example, a group project or presentation could be assigned and graded with a rubric that makes clear the learning outcomes and success indicators.
- The Senior Integrative Project (SIP) is often named by programs as a key assessment method. The assessment aspect comes in reading the students’ work (or viewing students’ presentations) with success indicators in mind to answer the question “To what degree does the work product of the SIP reflect attainment of or progress toward stated learning outcomes?”
Are classes in my program considered to be assessment methods?
- No, not on their own. For example, it might be tempting for a program to say, “our students demonstrate success in the learning outcome ‘master introductory concepts’ by passing our introductory courses.” That just shifts the question from the program to the courses within the program and doesn’t help answer the question about the degree to which students in the program attain the learning outcomes, and it doesn’t provide helpful examples for making changes and improvements. What’s more, course grades often reflect compliance with class rules such as attendance, on-time submissions, and the like, which are not the subject of the stated learning outcomes.
- An example of an assessment method that is directly related to a course would be to collect representative student work samples or survey responses from a particular question or activity that addresses a learning outcome and read that work with some agreed-upon success indicators in mind to answer the question “to what degree are students attaining or progressing toward our stated learning outcomes?”
How could we use a rubric to show growth if our program is small and our numbers will never be “statistically significant”?
- There is no need to report rubric numbers even when rubrics are used. The rubric numbers are arbitrary and helpful only in so far as they help you understand student work qualitatively.
- Numerical results with small numbers of students will naturally vary widely from year to year — we urge caution in using numerical changes over a short span of years as an indicator of trends in student attainment of learning outcomes.
How can we measure growth on an Institutional Learning Outcome when students may only take one or a few of our courses?
- You measure growth on your program’s or course’s learning outcomes, and you identify the alignment of those outcomes with ILOs.
- It is likely that the growth students exhibit is the result of many factors and experiences campus wide. If there isn’t a natural way to tease out the portion of that growth attributable to any one program, the observed growth still contributes to the important task of assessment at the institutional level.