For this assignment, you will write the usability test plan that you will use during the testing. You will also attend the Human Subjects (CITI) Training and receive the certificate.
Reading List
To prepare for this assignment, you should review:
Assignment Description
Usability Test Plan
Usability Testing is a process to uncover usability problems by observing potential users performing tasks on a prototype. Several documents are generated during the development, analysis, and reporting of a usability test, but for this course you will post on your website only the usability test plan and usability report.
The usability test plan is produced before the test and referred to during the test. It is also a tool to help develop the test, so this document is formatted more like an outline than a report. The goal of the test plan is to describe the tasks that the participant will perform and the questions that you will ask the participants. Participants perform tasks in the context of a scenario that you have described to them.
You are responsible for contacting your development/undergraduate team and arranging meetings to discuss their app design and the test plan. Inspect their website immediately and contact the team by email. You will want to meet with them to learn about their design and how much they have implemented. You should study their design so that you can anticipate usability problems; these suspected usability problems will give your ideas for your test scenario. You will also want to meet with the team to tell them what additional output you need from their program for your usability analysis.
Test Scenarios
A test plan consists of a series of test scenarios. An example outline for a test scenario:
- Test Scenario Name
- Test Goals for the scenario
- Any software or equipment required for the testing. For example a “smartphone” or if you are testing remotely then “Zoom Conferencing Software”.
- Quantitative measurement list (Measurement that will be made during the test or derived during the analysis.)
- Scenario Description (This is short description of what you expect will happen.)
- Scenario Text (This is the “story” that you will read to the participants verbatim. It provides context of the test. Typically this begins with “Imagine that you are … and want to …”.) (The scenario should be on a separate sheet of paper.)
- Qualitative measurement list (These are general measurements that are derived by analyzing qualitative observation of the participant and answers for post test questions.)
- Potential observations of participant (Specific observation that will support the qualitative measurements.)
- Bug Report Form (A form used to indicate functional software problems, including the person reporting the problem, the nature of the problem and instructions for replication.)
- Post Scenario questionnaire ( The questionnaire should be on a separate sheet of paper or google form.)
- Post Scenario interview. (Your usability test should include a verbal interview with the participant so that your usability test can explore issues that you have not foreseen.)
- Test set up details (Unusual details specific to the test, such as environment, user, equipment, and any testing platform or frameworks such as Zoom.)
The test scenario should evaluate a unique and critical design aspect of the undergraduate design. A test scenario generally has more than one task associated with it. You should produce at least two test scenarios, but you may need to have more scenarios to cover all the functionality of the app. A test plan is an outline, but the text of the scenario description, tasks and questionnaire should be full sentences appropriate for reading to the participant. I expect several page for each test scenario. Besides distributing the usability test plan to your development/undergraduate team, you should post it on your website. The test plan should have a cover sheet identifying you and the development/undergraduate team. An example of a professional usability test report can be found in the resources.
Additional Forms
So that I can compare test results across team, I require two additional requirements a bug reporting form and common usability questions. Also the consent form that you will share with the participant should be part of the usability test plan. Finally, so that I and other UX consultants can learn from your testing experience include a Testing Challenges form.
Bug Report Form
Bugs are errors in the program. The bug form should include
- Bug number – this is an unique number for each bug occurred during test
- Bug name – this is a short name given to the bug, and use to identify bugs.
- Bug description – this is a short description of the bug, 1 or 2 sentences.
Every time a user encounters a bug, unique bug number is given and the bug name. When a bug is first encounter put an asterisk by the bug. If the bug occurs across multiple users or even the same user, you do not put a asterisk and do not have to enter the description or location a second time, but do record the same name. The CS students should be able to help you identify and record bugs.
Common Usability Questions
The common usability questions are in CommonUsabilityTestQuestions.docx document. Note there there are two sets of questions. One set is given before scenario are tested and the other set after the scenarios have been tested. You are expected to add your own questions.
Consent Form
You will need a consent form that the participant to read or give verbal consent before the actual usability test. Use the Usability Testing Consent Form (also found in the resources directory) and have the participant give verbal consent.
After greeting the participant and introducing yourselves, you should hand the consent form to the participate. You should verbal summarize the form by saying, “The test impose minimal risk, no harm should come to you performing the test, the results of the usability test are anonymized, and if at any time you wish to terminate the testing you may.” In addition if you are video or audio recording then you should ask the participant if you can record them.
Testing Challenges
So that we can all learn about your challenges during usability testing, your test plan should have a table for “Testing Challenges”. Testing challenges are difficulty that you or the participant might had with the testing environment not involve the app. For example a testing challenge might be a participant having difficulty sharing their screen using Zoom.
The table should have columns:
- Challenge number – this is a unique number for each challenge.
- Challenge name – this a short name given to the challenge and used to tag the challenge.
- Challenge description – this is a short description of the bug, 1 or 2 sentences.
Human Subjects (CITI) Training
You are required to have training for human subject research, meaning to administrate the usability test. The RCR training is NOT a substitute for the for the CITI training in “Conducting minimal risk research” with human subjects.
The link below leads to a page to login/register for the CITI online training on. You only need to take the module labeled “Students conducting no more than minimal risk research.” The training should take less than an hour.
https://www.mtu.edu/research/integrity/human-subjects/citi-training/citi-training.html
There is also a link to download Powerpoint slides for “CITI Step-by-step Instruction.”
Be sure to select “Social/Behavioral Research Course (12 modules)”.
After you have completed the training, download the “Completion Report” and submit it with the URL test plan in Canvas.
Submit on Canvas and Email Your Team
When you have finished and posted your usability test plan submit on Canvas for the assignment the URL for your test plan and attach your CITI Training Certificate.
Do NOT include/attach your Usability Test Plan. Rather, I will read it on your website.
Also you should email your team that you posted the test plan on your website.
Grading Rubric
I will evaluate your Usability Test Plan for timeliness, completeness and correctness. Specifically:
- Are there at least two test scenarios?
- Are the test scenarios written in the language that can be read to the participant?
- Do the test scenario cover the appropriate functions of the app.
- Are there sufficient observations and measurements to determine the results of the test scenarios.
- Is the post scenario survey complete and appropriate?
- Are there post scenario interview questions?
- Are all addition forms/tables included and properly formatted?
I will evaluate conducting the Human Subjects (CITI) Training complete or incomplete.
Prepare for Next Assignment
In the next assignment, you will practice your usability test with your team. To prepare, read: