Usability Testing is a process to uncover usability problems by observing potential users performing tasks on a prototype. Several documents are generated during the development, analysis, and reporting of a usability test, but for this course you will formally post on your website only the usability test plan and usability report.
The usability test plan is produced before the test and referred to during the test. It is also a tool to help develop the test, so this document is formatted more like an outline then a report. The goal of the test plan is to describe the tasks that the participant will perform and the questions that you will ask the participants. Participants perform tasks in the context of a scenario that you have described to them.
A test plan consists of a series of test scenarios. An example outline for a test scenario:
- Test Scenario Name
- Test Goals for the scenario
- Quantitative measurement list (Measurement that will be made during the test or derived during the analysis.)
- Scenario description (What will be read to the participant prior to a test, describing the environment of the test.) (The scenario should be on a separate sheet of paper.)
- Task list (what you think the participant will do. These are more like goals for the participant. Do not read them to the participant.)
- Qualitative measurement list (These are general measurements that are derived by analyzing qualitative observation of the participant and answers for post test questions.)
- Potential observations of participant (Specific observation that will support the qualitative measurements.)
- Bug Report Form (A form used to indicate functional software problems, including the person reporting the problem, the nature of the problem and instructions for replication.)
- Post Scenario interview or questionnaire questions (The questionnaire should be on a separate sheet of paper.)
- Test set up details (Unusual details specific to the test, such as environment, user or equipments.)
So that I can compare test results across team, we have included two additional requirements a bug reporting form and common usability questions. Bugs are errors in the program. The bug form should include
- Bug number – this is an unique number for each bug occurred during test
- Bug name – this is a short name given to the bug, and use to identify bugs.
- Bug uniqueness – The first time any of your participants encounter this bug put an asterisk by the bug. When you the same participant or another participant encounters the same bug do not but an asterisk.
Bug location – this indicates which page of the website that has the bug - Bug description – this is a short description of the bug, 1 or 2 sentence.
Every time a user encounters a bug, unique bug number is given and the bug name. When a bug is first encounter put an asterisk by the bug. If the bug occurs across multiple users or even the same user, you do not put a asterisk and do not have to enter the description or location a second time, but do record the same name. The CS students should be able to help you identify and record bugs.
The common usability questions are in CommonUsabilityTestQuestions.docx document. Note there there are two sets of questions. One set is given before scenario are tested and the other set after the scenarios have been tested. You are expected to add your own questions.
You will need a consent form that the participant signs before the actual usability test. The consent form is found in the resources directory or here. After greeting the participant and introducing yourselves, you should hand the consent form to the participate. You should verbal summarize the form by saying, “The test impose minimal risk, no harm should come to you performing the test, the results of the usability test are anonymized, and if at any time you wish to terminate the testing you may.” In addition if you are video or audio recording then you should ask the participant if you can record them.
The test scenario should evaluate a unique and critical design aspect of the undergraduate design. A test scenario may have more than one task associated with it. You should produce at least two test scenarios, but you might have time only to conduct one of the scenarios during the actual testing. A test plan is an outline, but the text of the scenario description, tasks and questionnaire should be full sentences appropriate for reading to the participant. I expect about several page for each test scenario. Besides distributing the usability test plan to the undergraduate design group, you should post it on your website. The test plan should have a cover sheet identifying you and the undergraduate design group. An example of a professional usability test report can be found in the resources.
I will assign you an undergraduate group via class email list. You are responsible for making contact with the group and arranging meetings. Inspect their website immediately and contact the group by email. You will want to meet with them to learn about their design and how much they have implemented. You should study their design so that you can anticipate usability problems; these suspected usability problems will give your ideas for your test scenario. You will also want to meet with the group to tell them what additional output you need from their program for your usability analysis.
Human Subjects (CITI) Training
You are required to have training for human subject research, meaning to administrate the usability test. The RCR training is not a substitute for the for the CITI training in “Conducting minimal risk research” with human subjects.
The link below leads to a page to login/register for the CITI online training on. You only need to take the module labeled “Students conducting no more than minimal risk research.” The training should take less than an hour.
There is also a link to download Powerpoint slides for “CITI Step-by-step Instruction.”
http://www.mtu.edu/research/administration/integrity-compliance/docs/citi-instructions.pptx
Be sure to select “”Students conducting no more than minimal risk research”, and if the training is taking longer then an hour, please contact the IRB office, 487-2902.
After you have completed the training, download the “Completion Report” and email it to me with the subject line:
cs5760 Human Subject Training
Email Me
When you have finished and posted your usability test plan email me (pastel at mtu.edu) that you have done so. The subject line of the email should be
cs5760 Evaluation Assignment 4
Your email should NOT have the document attached. Rather, I will read it on your website.