Testing
Testing gives designers a better understanding of how users are actually using our web sites. I have a pretty good idea before I ever start designing what it is going to look like and how it will function, but that is only an hypothesis. I need to test the hypothesis, just as you would test a drug or an automobile. Granted, in most cases, nobody will die using your web site. But testing is important because users do all kinds of "crazy" things when navigating and using web sites that designers can't imagine until we see it in action. There are plenty of times I have watched users use my webs and I know that she needs to click the big red arrow pointing to the right to go to the next page, but still she misses it. If we test too late we don't find out about this kind of problem until it's too late. If we never test, we may never find out that our site is completely unusable.
Testing should be done all through the design and building of a web site. As described in Initial Design and Ideation I use Card Sorting as one technique for testing. Card Sorting or having users make collages to influence grouping information is part of early testing.
This page concentrates on performing qualitative testing on a prototype web design.
Why Are You Testing?
Remember the user goals? Define the user goals before you begin testing. Is the purpose of the site to provide information, market a product or service, or perform a specific task?
Who Are You Testing?
Remember your personas? They come into play here. Pick testing subjects who represent the personas that use the site.
What Are You Testing?
Narrow the scope of your testing to focus on the functionality and design based on the user goals.
Be Unbiased
It is very important to retain an impartial attitude to the test subjects when doing usability testing. I am testing how the user uses the web site as if I was not even there. I am there to give him a task to perform and watch as he does it.
Choosing Participants
This depends on the web site audience, how much money can be allocated for gathering participants, and for compensating them. Compensate all of your testing participants. Whether it is pizza, bagels, gift certificates, cash, a simple email thanking them, free usage of a paid service you need to compensate them. There are many methods for gathering participants and the methods used depend on what type of web site you are designing. The idea is to get as many users as possible, interview them either with an online survey form, on the phone or via email.
Screening
I create a screening questionnaire to determine the following based on how they use or will use the web site:
- Determine if they do the task that you are testing for. If you are building a feature that replaces an older method of doing the same thing you want to test the people who are already doing it the old way.
- Determine his fit into one or more user groups.
- Determine the proficiency of the user on the site (if it's a redesign). Too much experience could skew the results of your testing.
- Get a good mix of types of users.
Facilitating
We use Camtasia to record what the user is doing on the site while they talk about what they are doing. One person is in the room with the participant asking them questions. In another room that is far away from the testing room is the observation room. That is where the other stakeholders involved in the project can observe, take notes on what the user is doing. Stakeholders can include designers, developers, managers, project managers, other testers, marketing people, writers, editors, all of the various stakeholder involved in the project. I try to encourage the decision makers to attend these sessions. It is always eye-opening to see first-hand how users behave.
Debriefing
After all the testing is done I take the results and process them. This can take a week or two depending on the amount of testing that is done.
The idea is to generate two lists:
- The most serious problems encountered by the users
- The most important things to fix before the next month's round of testing
We go through the list of problems and prioritize the list. Then we go through from worst first and assign resources until we run out of resources. Then we stop. There are always more things to fix than we have resources. This process makes sure we tackle the big problems first.
As you can see testing is not the end of the process. It is only a new beginning. Design and tweaking of a web site is never done. As new content and features are introduced it is necessary to keep testing your site once a month.
Once redesign tasks have been allocated it may be necessary to go all the way back to wireframe mock-ups. The idea at this stage in the game is to go back the fewest number of stages as possible.