About This Assignment: Heuristic Evaluation

The purpose of the heuristic evaluation is to identify general usability concerns. The assignment deliverables differ somewhat for HU4628 students working with CS4760 teammates and those working on cross-functional teams within HU4628. However, this introductory material applies to both types of teams.

A heuristic evaluation (sometimes called an expert review) is performed by evaluating a design against a list of usability principles or heuristics specific to the domain or genre of the user interface design. A heuristic evaluation results in

• a list of potential usability concerns
• a list of critical design concerns

Goals of heuristic evaluations differ for the two types of teams as follows:

• Teams working with CS students will perform heuristic evaluations on their team’s preliminary design, with the goal of identifying usability concerns and critical design concerns that suggest where instructional and other documentation is required.
• Teams working within the HU4628 class will perform heuristic evaluations on the user interfaces they are  tasked with analyzing, with the goal of identifying more general usability concerns and critical design concerns.

From these analyses, a  heuristic evaluation report {need sample} is generated, aimed at your teammates, the scientist with whom you’re working, and the instructor(s) of your class(es). The evaluation information will be presented during the Cognitive Walkthrough.

Assignment Details and Definitions

A UI domain is a general category that the system belongs to. For example a word processor design would belong to the UI domain for text document generation and not the same as drawing software. You should also derive your usability principles by consulting the design’s nominal interaction diagram, HTA and other design documents. An article illustrating a heuristic evaluation is in the CS4760 resources and  the following HU4628 assigned articles are available on usability.gov:

• https://www.usability.gov/get-involved/blog/2005/10/judging-the-severity-of-usability-issues.html
• https://www.usability.gov/how-to-and-tools/methods/heuristic-evaluation.html

Potential Usability Problems, Critical Design Concerns, Usability Scenarios

Potential usability problems are UI usability problems that you discover during the Heuristic Evaluation.  This article outlines good information on the importance of identifying potential usability problems and suggests the importance of  being alert to them during heuristic evaluation. The article also notes that different evaluators tend to find different problems, so it’s a good idea to have individuals perform the HE first, before consolidating findings.

Critical design concerns are potential usability problem that would prevent the user from performing essential tasks. Not all potential usability problems are critical design concerns.

Example of Using a Heuristic

Probably the most commonly used heuristic for user-interface design is Jakob Nielsen’s 10 Heuristics for User Interface Design. Nielsen’s list of heuristics was written for the Web and has been adapted for broader use with UI designs. The following hypothetical list of potential usability problems for a GPS UI was generated using Nielsen’s heuristic:

1. On the opening form the function of the right arrow in the upper left is not clear. This violates the visibility principle.
2. On the confirmation screen the save and delete button are adjacent to each other; what if the user hit the delete button by mistake? This violates the error prevention principle.
3. There is not a cancel button on the form for saving. This violates the user control principle.

Critical design concern: No cancel on the save form is critical because it may cause excess work; for example: After entering the save form, Jane recalls that she wants to enter one more waypoint, but now she can only save. To do so, she must enter a name for the path. After the path is saved she must select that path from the list of save paths and open it in the editor. This is a lot more work compared to just canceling the save and adding the waypoint.

Document Format and Outline

The document should identify and explain UI domain. The document should also include lists of the heuristic usability principles followed by usability concerns. Each usability principle should have at least a sentence description. The usability concerns should be explicit problems that you found in the UI. Along with listing and identifying the usability concerns/problems you should also identify the usability principle that the problem violates.   The critical design concerns should illustrated with a short (a few sentences) story.

The document need not be long, perhaps 2 or 3 pages.   The document may contain tables and bullets, but all bullet lists and tables should be supported with full sentence explanations in the text. Your document is not an outline.  The document must be edited and proofread carefully, formatted for ease of reading and skimmability, and must communicate clearly.

Content should include:

1. Cover sheet identifying the group name, group members’ names, project assignment, and date
2. Short description of the design
3. Identification of the UI domain and short description
4. List of heuristics/principles used, the source for these principles, and a rationale for using them.
5. List of usability problems generated from the heuristic evaluation
6. Identification of critical usability concerns
7. Illustrate the critical usability concerns with a scenario

By the due date, post the document to your group’s website in a format appropriate for all web browsers and link it to your cognitive walkthrough.

Email Me and Your Team

Email Robert Pastel (pastel at mtu.edu) and Karla Kitalong (kitalong at mtu.edu) when your website is up-to-date, so that we may view your Heuristic Evaluations. The subject line of the email should be

hu4628 – Heuristic Evaluation

Your email should NOT attach the documents. Rather, we will read them on your website.

Also you should email your team that you posted the evaluation on your website.