Some Formative User Research Methods

During the 2013 UXPA Annual Conference in Washington DC, I took a great all-day workshop on research methods taught by Susan Mercer and Dan Berlin. In this workshop, they covered a multitude of activities which could be used to gather information about a product’s end users – from their behaviors and motivations, to their attitudes, and experiences (including pain points).

Susan and Dan divided these activities into two phases: formative (to be conducted at the beginning of or during the development of a project) and summative (to be conducted once a product is released). There terms were familiar to me from my graduate studies in instructional design in which formative and summative assessment are key to the design process.

Pulling from Susan and Dan’s presentation, I wanted to review a few of the formative methods they spoke about. I won’t go into more detail on individual methods in future posts, but this list will at least give you an idea of some techniques you can try.

One-on-One Interviews

You can conduct on-on-one interviews with both end users and stakeholders at the beginning of the discovery or requirements gathering process. Interviews are great for getting very detailed information, to explore open-ended topics and for learning about users’ experiences, motivations, desires, attitudes, etc. The down side of one-on-one interviews is that it can take a lot of time to both conduct the interviews and analyze the data.

 

Focus Groups

Focus groups have traditionally been the realm of marketing folks and there are definitely very strong opinions within the ux world about the value of focus groups. Those that do tout the value of focus groups, specifically recommend activity based sessions where participants do collaging, collaborative design, card sorting, usability testing, etc. Focus groups are great for exploring attitudes and ideas with a larger number of people. The downside of focus groups lies in the personal dynamics which need to be managed so that no one dominates the conversation and the conversation stays on topic.

People often ask for focus groups about websites, when what they really want are usability tests. I always emphasize – focus groups will tell you opinions and what people say they will do, usability tests will show you what people really do.

 

Collaging

When the UO Libraries worked with AEI’s Wendy Ames to understand the needs of graduate students, she incorporated a collaging exercise into the sessions. Here, students were given an assortment of images to choose from, most of which had nothing to do with libraries or studying, and were asked to choose 3-4 pictures which illustrated their experience using the UO libraries. Once students put their images together, they spoke about how these images illustrated their experience. Collaging will give you a wealth of details about users experiences, values, motivations and frustrations. Analyzing the data can be challenging, but the library came out with some great recommendations on how we could improve our services for graduate students.

 

Surveys

Many of us have conducted a survey to collect data. They can be especially useful to get feedback from a large number of people and when you need quantitative data. Survey questions are notoriously difficult to write and any qualitative feedback (from open-ended questions) are difficult to analyze. Often surveys are used in conjunction with other user research methods to understand the larger picture and uncover where more detailed information needs to be obtained. If you are thinking about conducting an online survey, check out the UO’s Qualtrics tool.

 

Diary Studies

If you’d like to collect some longitudinal data, diary studies might be something to try. In a diary study, participants track their interaction with a product over time, usually over weeks or months. These can be especially helpful if a single interview will not get you the information you need or when the users’ environment plays a role in their experience. You can use online tools to collect diary study entries, but you’ll need to design a system where you check in with users on a regular basis and give them incentives for not dropping out. I’ve often thought it would be interesting to conduct a diary study to understand the admissions experience at the UO or to follow students’ use of the library website to research papers and projects.

 

Field Studies/Contextual Inquiry and Ethnographic Studies

Ethnographic studies come from the world of academia where researchers observe users in their natural environment with minimal interaction. Field Studies or Contextual Inquiry are a version of that in which the ux researcher observes participants in their natural environment using a product and either asks them questions during or after the observation. The benefits of this method is that you will see how participants truly interact with a product in the real world, including the work arounds they have developed to adapt the product to their needs. The downside of this method is that your interactions with the participant can have an impact upon their behavior. It is also difficult to find participants who can give you the time to shadow them or feel comfortable inviting you into their personal worlds.

 

Card Sorting 

If you need to understand users’ mental models in terms of how they understand and organize information, a card sort can be extremely helpful. I use card sorting to get feedback on terminology, menu structures, and general information architecture.

Often, ux researchers begin with an open card sort. This is where you either write tasks or individual pieces of content onto an index card and ask users to group them in whatever way makes the most sense to them. Once they have created their groups, you can then ask them to give these groups titles or category names. There are great online tools to help you conduct and analyze card sorting data. The downside of using an online tool is that you can’t ask the all important “whys.”  I like to sit with users as they conduct an online card sort so that I have the data, but I can also ask them to explain the thinking behind their organization.

The second type of card sort you can conduct is called a closed card sort. This is often helpful when you have a menu structure or information architecture in mind and want to test it out. With a closed card sort, you give users the high level categories or buckets and ask users to sort the content into those buckets. This will help you see if the organization you were considering matches your end users’ mental models.

 

Desirability Testing

If you’d like to understand a target audience’s emotional reactions to different design options, desirability testing is something to consider. This method was originally created by Microsoft and involves giving end users a list of adjectives (60% positive / 40% negative) and ask them to choose a certain number of these adjectives to describe a specific design. I recently worked with UO Digital Communications and Social Media to run a type of desirability test on stories which appear on the UO homepage. In this study, we gave participants a list of “personality traits” which have been used to describe the UO and then asked how these traits lined up with the stories they were shown. This activity gave us a lot of insight on not only how prospective students and their parents think about the UO, but also how Digital Communications can align their stories to support the interests of these target audiences as well as the University’s business goals.

 

There are many other methods one can use to learn about a products end users, but these are some tried and true ones to help you start.