Evaluation Assignment 5 – Heuristic Evaluation

In this assignment, you will perform a heuristic evaluation after your development/undergraduate teams has conducted their cognitive walkthrough.

Reading List

To prepare for this assignment, read:

Assignment Description

Heuristic Evaluation

The purpose of the heuristic evaluation is to identify general usability concerns of the undergraduate designs. Besides listing the potential usability concerns, an important goal of the document is to illustrate critical design concerns with a story.

User Interface (UI) Domain

A heuristic evaluation is performed by evaluating the design against a list of usability principles. The usability principles are specific to the user interface (UI) domain of the design. An UI domain is the category that the UI belongs to. For example a word processor would belong to the UI domain for text document generation and not the same as drawing software.

You need to be careful identifying the UI domain. The UI domain should not be too general or specific. For example identifying the application domain as a “web app” or “user interface” would to general. Identifying the application domain as an “application for recording berry harvest” or “application graphing microbit sensor data” would be too specific. More appropriate domain might be “web app for field observations” or “real time graphing web app.” The purpose of identifying the application domain is too generate the appropriate heuristic usability principles. If the application is too general then the heuristics usability principles that you generate will not be specific or relevant for app, and you will probably end up using the publicized general heuristics such as Norman’s 10 heuristics. If your application domain is too specific then you will not be able to generalize the results of your heuristic evaluations or reference other studies.

Heuristic Usability Principles

After identifying the appropriate UI domain, you should be able to generate a list of heuristic usability principles for your evaluation. You can consult usability principles from the literature, but you should select usability principles that are specific to the UI domain. You can also derive your usability principles by consulting the design’s nominal interaction diagram, HTA and other design documents that you and your development team have generated. An article illustrating a heuristic evaluation is in the resources.

Potential Usability Problems and Critical Usability Concerns

Potential usability problems are UI usability problems that you discover during your Heuristic Evaluation.   Critical usability concerns are potential usability problem that would prevent the user from performing essential tasks. Not all potential usability problems are critical design concerns.

Potential Usability Problems

The usability problems listed in your list of potential usability problems should identify the location of the usability problem, briefly describe the usability problem and state the violated heuristic. For example, a list of potential usability problems for a UI to construct a path on a GPS could be:

  1. On the tracking form, the function of the right arrow in the upper left is not clear. This violates the visibility principle.
  2. On appearance of the confirmation screen, the save and delete button are adjacent to each, what if the user hit the delete button by mistake? This violates the error prevention principle.
  3. On the tracking form, there is not a cancel button on the form for saving. This violates user control principle.
  4. etc.

Critical Usability Concerns

Critical usability concerns are derived list of usability problems. For example in the list the lack of a cancel button is a critical usability concern because if the user had made a mistake opening the tracking form then user can back out of the mistake.

Note that the usability problem of the lack of clarity about the right arrow on the form is not critical usability concern because the user may learn what the arrow means. Also the adjacency of the save and delete button is not critical because the user can perform the functions, but must be careful.

Critical Usability Concern Scenario

You make clear that the potential usability problem is critical by writing a usability scenario, a story that illustrates that the problem is critical. For example assume that you think that problem 3 in the list above is critical you would write:

No cancel on the save form is critical because it may cause excess work, for example: After immediately entering the save form, Jane recalls that she wants to enter one more waypoint, but now she can only save and to do so she must enter a name for the path. After the path is saved she must select that path from the list of save paths and open it in the editor. This is a lot more work than compared to just canceling the save and added the waypoint.

Document Format and Outline

The document should identify and explain the UI domain. The document should also include lists of the heuristic usability principles followed by usability concerns. Each usability principle should have at least a sentence description. The usability concerns should be explicit problems that you found in the UI. Along with listing and identifying the usability concerns/problems you should also identify the usability principle that the problem violates.   The critical usability concerns should illustrated with a short (a few sentences) story.

The document need not be long. (I expect 2 or 3 pages.)  The document may contain tables and bullets, but all bullet lists and tables should be supported with full sentence explanations in the text. Your document is not an outline.  Be advised that I will evaluate the correctness of the document, but also how well it communicates.

An example outline:

  1. Cover sheet identifying you and the undergraduate group
  2. Short description of the undergrad design
  3. Identification of the UI domain and short description
  4. List of heuristic usability principles for the design’s UI domain
  5. List of usability problems generated from the heuristic evaluation
  6. Identification of critical usability concerns
  7. Illustrate the critical usability concerns with a short story

I will assign you an undergraduate group via the course email list. After the assignment of your group, check the group’s website for all the documents that you will need. Email the undergraduate group informing them of any additional documents that you need or questions that you might have. I advise emailing the groups to at least introduce your self and to get the link to their design documents.

You are to post the document in your website in a format appropriate for all web browsers and give the undergraduate group a link to the document. Also your document or your website should have a link to the undergraduate project and their cognitive walkthrough.

Meeting with Sponsoring Scientists

The design heuristics evaluation requires that you understand the app and the context of the app that the team is develop. You will need to attend meetings with the scientist sponsoring the app when the team meets with the scientist. It is not sufficient that you learn about scientists requirements and app context from the team. They may not remember all the details of the meeting or misunderstand.

You should not run the meeting with the scientist. Let the team run the meeting. This is an opportunity for the undergraduate students to learn to gather requirements from a client.  You may ask the scientist questions.

Submit on Canvas

After you have posted the your Heuristic Evaluation on your website, submit the URL of the evaluation on Canvas. Your canvas submission should NOT have the documents. Rather, I will read them on your website.

Also you should email your team that you posted the evaluation on your website.

Grading Rubric

The Heuristic Evaluation will be evaluated on timeliness and completeness. Specifically:

  • Can I find and access the document?
  • Is the document understandable?
  • Is the UI domain appropriate and specific?
  • Are heuristic usability principles appropriate and explained?
  • Usability problems and critical concerns correctly identified and explained?
  • Do the critical concern scenarios clearly illustrate the critical concern?

Prepare Next Assignment

In the coming evaluation assignments, you start planning the usability test. To prepare read: