How are those New Year’s resolutions going? If it’s anything like mine, it’s probably not going quite as well as planned. You were really into your new goal for a couple of days, downloaded a new app to help you along, made some kind of plan, and then life got busy.
The old routine kicked in and a few weeks later, you were repeating the same thing again: “I really need to work out more.”
Our team here at AgilityFeat has been conducting interviews with UX practitioners to learn how they are doing usability testing. We started hearing the same kinds of things — “We should do more usability testing” — in a way reminiscent to the whole “I need to work out more” thing.
We all know we need to do it, but it’s often not our top priority. As we’ve continued to explore this problem with failed usability resolutions through conversations, we’ve noticed some trends:
#1 Everyone values usability testing
Everyone we talk to sees tremendous value in usability testing. Anytime you sit down with customers and show them your software, you learn a ton. People look for things you didn’t expect, try to click in places you didn’t make clickable, and stumble on menu structures you thought seemed really clear.
Most people understand this, which is why UX is becoming a more prominent role. Whenever IT teams do get around to some usability testing, they leave the sessions with the same sort of endorphins you feel after a good workout: “That felt great!” “We should do that more!”
And yet they still don’t … Why?
#2 Usability testing means many different things
Because our company is widely distributed around North and Central America, we’re particularly interested in remote usability testing. When talking with UX practitioners, we’ve been a little surprised to learn how many different approaches to usability testing there are. It’s not just a choice of remote and in-person; there’s a lot of variation on timing, pre-screening, incentives, and more.
Some companies only do usability testing after a release, to see if users like the new features or behave the way they expected. Or they do usability testing as a sort of manual A/B testing of different features. Some companies test new features before they’re deployed, and this could involve “testing” against a mockup or prototype, or sending the user to a private beta URL.
Others conduct usability testing purely in the “think aloud” style, asking users to accomplish a task on the production site as they sit back silently and take notes. Some prefer to do the usability testing in person to better gauge a subject’s emotional reactions. Others prefer remote usability testing with desktop screen sharing tools, aiming to see the test subject perform in their natural work or home environment, rather than a fabricated UX lab setting.
Everyone values usability testing, but there are many different compelling motivations to do it — again, just like exercise. Some of us are looking ahead to summer swimsuits, others are just trying to extend their lifespan.
Every UX practitioner values usability testing and *should be* motivated to do it, so again, why aren’t people doing more of it?
#3 Finding relevant test subjects is difficult, depending on the size and nature of the organization
Many companies we’ve researched use usertesting.com to recruit test participants outside of the current customer base. UserTesting provides a base of millions of potential test subjects, which is wonderful, but are those users truly relevant to your application? Finding good test subjects can be complicated and time consuming. Some companies we talked with have this down to a science, others struggle with the amount of work it takes.
Depending on the type of usability test, some companies go to their own user base or live visitors on their website and try to recruit them. Others take cameras and go out to the homes or businesses of their customers. And one company we talked with particularly values testing their applications with users who are not as internet savvy – so they use traditional recruitment methods like posting flyers in grocery stores.
Once you get a willing participant, getting them to actually show up and participate is another issue. Signing up for that boot camp exercise class is easy, actually waking up early enough in the morning to fit it in… not so easy.
#4 Usability testing has a learning curve
One of the biggest challenges to remote usability testing is the onboarding process. In order to conduct a 30-minute usability session, you often need 15-20 minutes of prep time to:
- Explain the protocol of the session
- Explain how you will reward them (the gift card will arrive right after the test, I promise!)
- Help them install GoToMeeting or some other desktop sharing tool
You’re probably doing this over a phone line and then perhaps switching to the conference call tool once it’s setup. It’s a lot of setup time for a relatively short interview, and probably never repeated with that particular person again.
When you join a new gym for the first time, you often get a free session with a personal trainer to show you around the equipment and instruct you on how to use it safely. That’s a lot of work if you only do a couple workouts and then quit. No wonder gyms require long contracts!
#5 How are we going to see results?
The endorphins after that first workout of the new year are awesome. Then, the next morning, you get on the scale and see nothing has changed … yet. Persistence is the key to turning a resolution into a lifestyle change, and usability testing is similar.
The benefits of regular research are obvious to everyone we meet — there’s no convincing needed — but finding a sustainable way to conduct these sessions takes a level of diligence that’s difficult to achieve in the face of the large overhead often necessary to conduct usability testing.
More than one of the interview subjects admitted to feeling almost embarrassed that they don’t do usability testing as often as they’d like. To do it consistently requires getting a real process in place for user recruitment, session protocol, and the incorporation of the results back into the workflow.
People who do all of this well are seeing great results, and others are holding their software up to the mirror and wishing it would fit into that two-piece bathing suit a little better.
Many thanks to Mariana Lopez and Lindsay Ifill, who co-authored this post along with me and have contributed significant amounts of time and expertise to the interviews we have been conducting. We’re learning a lot and plan to post more on this topic in the future, but contact us via twitter if you have any questions in the meantime.
