Usability Testing is a process to uncover usability problems by observing potential users performing tasks on a prototype. Several documents are generated during the development, analysis, and reporting of a usability test, but for this course you will formally post on your website only the usability test plan and usability report.

The usability test plan is produced before the test and referred to during the test. It is also a tool to help develop the test, so this document is formatted more like an outline then a report. The goal of the test plan is to describe the tasks that the participant will perform and the questions that you will ask the participants. Participants perform tasks in the context of a scenario that you have described to them.

A test plan consists of a series of test scenarios. An example outline for a test scenario:

1. Test Scenario Name
2. Test Goals for the scenario
3. Quantitative measurement list (Measurement that will be made during the test or derived during the analysis.)
4. Scenario description (What will be read to the participant prior to a test, describing the environment of the test.) (The scenario should be on a separate sheet of paper.)
5. Task list (what you think the participant will do. These are more like goals for the participant. Do not read them to the participant.)
6. Qualitative measurement list (These are general measurements that are derived by analyzing qualitative observation of the participant and answers for post test questions.)
7. Potential observations of participant (Specific observation that will support the qualitative measurements.)
8. Bug Report Form (A form used to indicate functional software problems, including the person reporting the problem, the nature of the problem and instructions for replication.)
9. Post Scenario interview and questionnaire questions (The questionnaire should be on a separate sheet of paper.) Your usability test should include a verbal interview with the participant so that your usability test can explore issues that you have not foreseen.
10. Test set up details (Unusual details specific to the test, such as environment, user, equipment, and any testing platform or frameworks such as Zoom.)

So that I can compare test results across team, we have included two additional requirements a bug reporting form and common usability questions. Bugs are errors in the program. The bug form should include

1. Bug number – this is an unique number for each bug occurred during test
2. Bug name – this is a short name given to the bug, and use to identify bugs.
3. Bug description – this is a short description of the bug, 1 or 2 sentences.

Every time a user encounters a bug, unique bug number is given and the bug name. When a bug is first encounter put an asterisk by the bug.  If the bug occurs across multiple users or even the same user, you do not put a asterisk and do not have to enter the description or location a second time, but do record the same name. The CS students should be able to help you identify and record bugs.

The common usability questions are in CommonUsabilityTestQuestions.docx document. Note there there are two sets of questions. One set is given before scenario are tested and the other set after the scenarios have been tested. You are expected to add your own questions.

You will need a consent form that the participant signs before the actual usability test. The consent form is found in the resources directory or here. After greeting the participant and introducing yourselves, you should hand the consent form to the participate. You should verbal summarize the form by saying, “The test impose minimal risk, no harm should come to you performing the test, the results of the usability test are anonymized, and if at any time you wish to terminate the testing you may.” In addition if you are video or audio recording then you should ask the participant if you can record them.

The test scenario should evaluate a unique and critical design aspect of the undergraduate design. A test scenario may have more than one task associated with it. You should produce at least two test scenarios, but you might have time only to conduct one of the scenarios during the actual testing. A test plan is an outline, but the text of the scenario description, tasks and questionnaire should be full sentences appropriate for reading to the participant. I expect about several page for each test scenario. Besides distributing the usability test plan to the undergraduate design group, you should post it on your website. The test plan should have a cover sheet identifying you and the undergraduate design group. An example of a professional usability test report can be found in the resources.

I will assign you an undergraduate group via class email list. You are responsible for making contact with the group and arranging meetings. Inspect their website immediately and contact the group by email. You will want to meet with them to learn about their design and how much they have implemented. You should study their design so that you can anticipate usability problems; these suspected usability problems will give your ideas for your test scenario. You will also want to meet with the group to tell them what additional output you need from their program for your usability analysis.

Human Subjects (CITI) Training

You are required to have training for human subject research, meaning to administrate the usability test. The RCR training is not a substitute for the for the CITI training in “Conducting minimal risk research” with human subjects.

The link below leads to a page to login/register for the CITI online training on. You only need to take the module labeled “Students conducting no more than minimal risk research.”  The training should take less than an hour.

http://www.mtu.edu/research/administration/integrity-compliance/review-boards/human-subjects/citi-training.html

There is also a link to download Powerpoint slides for “CITI Step-by-step Instruction.”

http://www.mtu.edu/research/administration/integrity-compliance/docs/citi-instructions.pptx

Be sure to select “”Students conducting no more than minimal risk research”, and if the training is taking longer then an hour, please contact the IRB office, 487-2902.

After you have completed the training, download the “Completion Report” and email it to me with the subject line:

cs5760 Human Subject Training

Email Your Team and Submit on Canvas

When you have finished and posted your usability test plan submit on Canvas for the assignment the URL for your test plan.

Your submit should NOT have the document attached. Rather, I will read it on your website.

Also you should email your team that you posted the evaluation on your website.

Modifications for Spring 2020

I emphasize that your test plan should have index listing any tools and platform the test will use, especially for remote usability testing.

You should use the Remote Usability Consent Form.

Also so that we can all learn about the challenges of remote usability testing, you test plan should a table for enter “Testing Challenges” for any difficulties you had during testing. Testing challenges are difficulty that you or the participant might had with the testing environment not involve the app. For example a testing challenge might be a participant having difficulty sharing their screen using Zoom.

The table should have columns:

1. Challenge number – this is a unique number for each challenge.
2. Challenge name – this a short name given to the challenge and used to tag the challenge.
3. Challenge description – this is a short description of the bug, 1 or 2 sentences.