Usability testing is a process to uncover potential usability problems. In our case, usability testing will be accomplished by observing representative users performing defined tasks on a prototype. Several documents are generated during the development, analysis, and reporting of a usability test; for this course you will concentrate on the usability test plan and usability report.

About the Test Plan

The goal of the test plan is to describe the tasks that the participant will perform, how you will observe their interactions with the product, and the questions you will ask or other feedback you will elicit from the participants. Participants perform tasks in the context of scenarios that you have described to them.

A test plan serves as an outline or checklist for the test, so that the testing personnel don’t forget any details.

A customizable test plan template is available here: https://www.usability.gov/how-to-and-tools/resources/templates/usability-test-plan-template.html 

Timing and Purpose of the Test Plan

The usability test plan is produced and the protocols are pilot–tested before the actual tests are conducted. The plan and protocols are referred to during the test. It serves the following purposes:

1. Summarize the highest priority usability issues identified during the development process and heuristic evaluation.
2. Outline how the issues were addressed in the development process to improve the user experience.
3. Detail the test protocol(s) to be used to assess users’ opinions of the improvements.

The Test Protocol

A test protocol consists of one or more test scenarios, which may or may not be identical to the scenarios you produced earlier in the semester. Each scenario may have multiple tasks associated with it. You should produce at least two scenarios and associated tasks, keeping in mind that you may only have time to complete one of these in a given test.

The test scenarios should evaluate unique and critical  aspects of the app design. Most test scenarios will have more than one associated task. You should produce at least two test scenarios, but you might have time only to conduct one of the scenarios during the actual testing.

The scenario descriptions, tasks, and pre- and post-questionnaires should be fully fleshed out in sentences appropriate for reading to the participant. Each test scenario should be about two pages (roughly 500 words) long; that’s the level of detail/planning ahead that is expected. You should have two or even three scenarios prepared, to cover as many of the potential usability problems as possible.

An example outline for a test scenario:

1. Test Scenario Name.
2. Goals for the test scenario (or problem that the test scenario is meant to document).
3. Scenario description (script that will be read to the participant prior to the test, description of the test environment).
4. Tasks outlined in as much detail as possible.
5. Observation descriptions (specific observations that will be conducted and data to be collected).
6. Measurements/qualitative metrics (metrics that you will use to document the observation findings).
7. Bug Report Form (see below; a form used to indicate functional software problems, including the person reporting the problem, the nature of the problem, and instructions for replication).
8. Pre- and post-test interview or questionnaire questions (see below for required questions; you should also include some questions specific to your app and your test).
9. Test set-up details (set-up details specific to the environment, user, or equipment, to ensure that each test participant begins from the same point).

To facilitate comparing test results across teams, we have included two additional requirements: a bug reporting form and common usability questions. Bugs are errors in the program. You should fill out the bug form as you conduct the test, share the results with team mates and include the results in your final usability report.  Each bug form should include

1. Bug number – this is an unique number for each bug observed during test
2. Bug name – a short name given to the bug, that will be used to identify bugs.
3. Bug uniqueness – The first time any of your participants encounter this bug put an asterisk by the bug. When the same participant or another participant encounters the same bug, document it, but do not mark it with an asterisk.
4. Bug location – this indicates which page of of the website has the bug
5. Bug description – a short description of the bug, 1 or 2 sentences.

The CS students should be able to help you identify and record bugs.

THIS DOC NEEDS EDITING–OMIT THE FIRST QUESTION. The common usability questions are in CommonUsabilityTestQuestions.docx document.  There are two sets of questions. One set is given before scenario are tested and the other set after the scenarios have been tested. You are expected to add your own questions.

Besides distributing the usability test plan to the undergraduate design group, you should post it on your website. The test plan should have a cover sheet identifying you and the undergraduate design group. An example of a professional usability test report can be found in the resources.

When you have finished and posted your usability test plan email me (rpastel at mtu.edu) and Karla Kitalong (kitalong at mtu.edu) that you have done so. The subject line of the email should be

hu4628 – Usability Test Assignment 2

Your email should NOT have the document attached. Rather, we will read it on your website.