My first solo data sort, and a foray into #saass
I did it. I doubted myself, I doubted my abilities, and my brain to be able to complete it. But I did it. And I only broke out into nervous hives once over it.
Ready for it? I completed my first solo qualitative data sort. I know this may seem elementary for a lot of people, but its a pretty big deal for me. I have participated before, but never done it on my own. I used a method that involved a lot of index cards, but I am very happy with my results. If you have not fallen asleep from boredom, allow me to elaborate. But I must first start with some background.
I am currently serving as the Director of the North Engagement Center at Michigan State. In it, I am a part of the Neighborhoods initiative (check out some background information here), which is focused on increasing Retention, Persistence, and Academic Success for our student population. One of our major roles is to work with first year students that end up on academic probation after their first semester (receive less than a 2.0 overall grade point average). My role is to coordinate our outreach and intervention strategy, and work with our campus partners to reach students.
Still with me? Anyone falling asleep yet?
Our first step was to make sure students on probation were seeing their academic advisor prior to the deadline to complete a success plan. In order to do that, we mobilized our live-in student staff to reach out to students and make sure they were aware of their status, and then try to determine what factors caused them to end up on academic probation in the first place. We could not have done this process without our student staff members, and they got us some great information.
What we ended up with was a spreadsheet with lots of information on it, and I wasn’t really sure what to do with it from there. Then I remembered a data sort method I used at Texas A&M involving ideas on index cards, and then grouping them by common themes to develop categories. I figured this was as good a method as any I could think of right now, and started organizing myself. We ended up having 66 respondents, so I took their reasons, and put them on cards. Some students had more than one reason they attributed their academic performance to, so I split them up and each unique idea got a separate card. When the cards settled, I ended up with 98 reasons this cohort of students ended up on probation.
At this point, I think it is important to know that my statistics class in graduate school was based mostly around how to manipulate SPSS, and involved a professor who wore cargo shorts and shiny shirts. The shorts and shirt are the most that I remember from that class, so even getting to this step was a major step for me.
From there, I read each reason (again), and started grouping them together by theme. After reading each one, I ended up with this:
18 different stacks of cards. Some of them were light, some were heavy. I decided to take a second look, and ended up combining some, and sorting a few more out. At the end, I finished with 14 different themes. From there, I labeled each one, and counted the frequency of each response. From the back of my mind, I seemed to remember something from my statistics class involving frequency, and showing it, and then the term histogram popped into my head.
“YES, I NEED A HISTOGRAM!! I WILL HISTOGRAM THE CRAP OUT OF THIS DATA!!!”
So that is what I did. I found some great information, and I will speculate about what it means, and what it can tell us in a later post, but wanted to share the process before the analysis afterwards today while it was still fresh in my mind. At the end, was it super involved, not really, but I do have to admit, I am pretty proud of getting this far. Now, the big step is what we do with it.
What methods are effective for you in sorting qualitative data? What would this data suggest to you?
PS: @lmendersby, I thought about writing the post, and then I did 
Let me just say that “I will histogram the crap out of that data!” will now be heard being shouted, by me, at various intervals throughout the next few months. Amazing.
When I approach a qualitative analysis, sometimes it’s an interesting exercise to go back to your desired goal(s) and/or outcomes to look for potential categories to sort your raw data into. For example, if one of your outcomes was to have students develop a particular skill, you could create a category around words that denote that skill and/or how the skill is reported on/expressed. This is more a priori coding where you decide on categories ahead of your sort, which helps logistically but may also ‘blind’ you to more emerging categories (emergent/emerging coding is the opposite of a priori coding). Granted, having defined outcomes also helps focus your coding to help you find data that can show you whether or not your outcomes were achieved.
I love that you’ve taken such a great step into the assessment world. While we can use a lot of ‘fancy’ terms in the assessment world, good assessment really starts with being open to new and different ideas, and being able to think outside of and beyond our own lens (difficult, yet necessary!). I can’t wait to read more (hint, hint).