Assessment

It’s April 1^st. I admit, I’m not much for April Fool’s jokes, but that got me thinking about assessment failures (and potentially looking like an assessment fool). I thought it might put people at ease to hear about some of the mistakes I/we have made over the years. Fortunately, we are quick learners and only make big mistakes one time (knock on wood).

 

One of the most memorable was the time we accidently sent an electronic survey to students that appeared to come from the university president. A setting had been changed at the university level in the software system we were using, and no one caught it until the survey came out. In the big scheme of things, the survey content was really benign but not something that the president would really send out. As soon as we found out about it, we were on the phone to the Vice President’s Office so they could communicate up. Not surprisingly, we had a fairly high response rate (so it’s true that your email invitations should come from a recognizable person)!

 

We are all about helping people try new assessment methods. One department had an event outside, so it would be virtually impossible for people to card swipe or sign in. We had the technology to create a QR code that participants could use to basically use their phone to scan the code that would open up the assessment. The department posted the QR codes around the event. Unlike the example above where we had a high response rate, this time we had a 0% response rate! Not one person scanned the QR code (we checked to make sure the technology was working correctly). Lesson learned: students don’t use QR codes. The department was disappointed and embarrassed, so we talked about how they could get feedback from the staff who worked the event as a means of assessment. I gave them lots of kudos for trying something new.

 

Technology is a great thing and has allowed us to personalize survey invitations. At the same time, Excel is not always the best way to keep accurate data. In this case, our client deleted a cell in the spreadsheet, moving all the other cells in that column up one. We did not catch onto that until it was too late. The result is that some of our “personalized” emails went to the wrong person (e.g., Eric’s email said “Howdy Susan!”). That also interfered with the demographic data we had preloaded into the database. Assessment is only as good as the data that gets fed into it.

 

I hope this makes you feel better about any assessment mistakes you have made or will make. The moral of the story is to fix the problem as best you can as soon as you identify it, make note of how to prevent it from happening again, and chuckle at yourself when it’s over.

February is known for Valentine’s Day and celebrating relationships. I hope that you will celebrate your relationship with assessment. Some of you are thinking, “I certainly don’t have a loving relationship with assessment. I’m not sending chocolate or even a greeting card.” Thinking beyond the nuts and bolts of assessment toward the end result of assessment, you might actually have a positive relationship.

Think about the purpose of assessment. I emphasize that assessment is about improvement and doing the best for the students (or staff, clients, users, etc.) we work with. We are doing assessment to provide the best possible experiences for the people we care about. We build the best facilities so students can study, live, exercise, build relationships, and more. We provide programs, services, and activities where students can learn, grow, and connect with others. We coordinate leadership and employment opportunities so students can learn and apply skills and get feedback from peers, supervisors, and advisors.

What do those things have in common? It’s our desire to give students the best experience possible, to encourage their learning, satisfaction, and connection. If we didn’t do assessment, how would be know that we were not living up to that standard? We wouldn’t. We might keep doing the same thing, not knowing that it was not beneficial to the people we care about. It’s all about open communication.

Assessment is like having that “relationship check in” talk. You might be nervous and not know where to start. You might be afraid of what people are going to tell you (you want to be the perfect partner, don’t you?), you think you might not be able to meet their expectations….That’s totally reasonable. But, wouldn’t you rather know how to improve rather than continuing to do something annoying or unproductive?

Even when you “break up” (students graduate, staff move to other positions, etc.), you want people to say positive things about you. You want them to recommend you to others. You want students to be better prepared for their next relationship: employment, grad school, next leadership position, etc. Others will be able to see the positive influence you had when they start a relationship with the student. You want to know how you can be better in the next relationship.

Feeling a little more positive about assessment? I hope so. If not, Student Life Studies can be your assessment relationship mediator. We can help you ask useful questions and interpret what others say, so you can be the best you can be. Feel free to contact us at [email protected] or (979) 862-5624. We are always here to help.

I hear a lot of staff and graduate students talk about “proving” the worth of a program or “proving” student learning took place or having to “prove” that student affairs has value on a college campus. I appreciate the effort and sentiment behind the word, but I also caution folks about making claims they cannot always support. Word choice is important.

Student affairs assessment can be messy. Working with people, mostly college students who are still developing and growing, is messy (literally and figuratively!). Students do not arrive on campus as a clean slate—they bring their own experiences, cultures, beliefs, skills, and motivations. They have various experiences while they are here, so no two students have the exact same college experience. Even within the Division of Student Affairs, students may have similar opportunities, but not the same outcomes. Students may or may not do the following: live on campus, be treated by Student Health Services, join and/or lead a student organization, face discipline for making poor choices, attend a program, work out at the Rec Center….The possibilities and combinations are endless.

What does that have to do with proving things through assessment? We have to be careful about attributing student learning/success to any one intervention or program. Yes, in general, students who live on campus their first year are more academically successful. But, does that mean that every student who lives on campus will be academically successful? Unfortunately, no. In a sense, a few students will disprove the idea that residence life causes academic success.

Another example would be asking students to rate themselves on communication skills after attending a two-hour program. How sure are you that any increase in their scores can be attributed solely to attending the program? What were their communication skills before they attended the program? And, for something like communication skills, were students given any opportunity to practice or demonstrate those skills to be observed and rated by someone who has expertise in that area? So, does attending a communication skills program prove that students improved? Not necessarily. If that topic is important to us, should we keep offering it? Probably. Can the assessment of the program tell us about improvements we can make? Yes.

If you don’t use the word “prove,” what should you use? I like “provide evidence of” or “supports” or “indicates.” While they are not as forceful, they provide a foundation and a demonstration of something in a narrow context. Words are important to communicate correctly to a variety of audiences.

P. S. In my opinion, we need to stop worrying about proving our worth in higher education. Accept that we have value, and take opportunities to share assessment results that show that.

If you need any assessment assistance, please contact Student Life Studies at (979) 862-5624 or [email protected].