The Heuristic Evaluation Project

Posted: October 6th, 2009 | Author: Lindsay | Filed under: Design, featured, usability | Tags: , , , , , | No Comments »

The project detailed here was created and designed by me, Lindsay Tabas, in Fall 2008.  It is based on Jakob Nielsen’s Heuristic Evaluation methodology that I learned while taking Marti Hearst’s class titled “User Interface Design & Development” in Spring 2006.

Summary


A Heuristic Evaluation is a usability inspection method performed systematically; it is traditionally part of an iterative UI development process and an alternative to user testing.  For this project, an evaluation was performed with all employees within the company.  We used the heuristics more loosely to guide discussion around the major functional areas of the product.

The goals of a heuristic evaluation are:

  • To find usability problems, both major and minor
  • To judge the software’s compliance with recognized usability principles

The discussion elicited many ideas and visions for how the product can accommodate its vast client needs.  The largest benefit was the consensus building and idea trading taking place between colleagues from different departments within the company.

Why Perform a Heuristic Evaluation of Your Product?

A Heuristic Evaluation project is the first step in evaluating an existing product’s strengths and weaknesses, and will support brainstorming within a company on improvements for the next version. The primary outcome of the project is a compiled design evaluation that includes new feature proposals and high level design principles. The secondary outcomes of this type of project is to unite the different business units of the company, promote creativity and forward thinking, and build a shared sense of team and common goals. A tertiary benefit, which should be part of every company’s knowledge management efforts, includes hard-coding employee domain expertise so that in the event of anyone’s departure, this knowledge remains an asset to the company, and does not walk away with the employee.  We may also include education as a benefit, in that more team members grasp basic usability principles that they can carry with them as they embark on features and bug fixes that don’t have time to be reviewed by a designer or analyst.

Background

In April 2008, I was the first trained “any role”* that sat between the client and the software development team. The product has sailed by for the previous 5 years because it was the only web-based SAAS product on the market in its industry. When you have that type of advantage, little effort is required to retain customers. That means that features were poorly implemented and based on only a few loud clients’ requests, the sales team was allowed to oversell the product, and the customer support team was left to handle dissatisfied users. All of this, after five years, left the majority of development resources stuck fighting fires, rather than building a world-class, scalable application. It was also clear that there was a breadth of domain expertise from all parts of the company, but little time to share that knowledge, nor build up from it.

After the first 6 months at work, I lobbied the CTO and the rest of the management team to allow me to collect our company’s knowledge by running a Heuristic Evaluation on our out-of-the-box product. Not only would I be able to hard-code this knowledge as a company asset, I would be able to learn more from my co-workers in order to build better features and modules, as well as work towards Product 2.0.

Heuristics


The ten usability heuristics are listed here:

  • Visibility of System Status
  • Match Between System and the Real World
  • User Control and Freedom
  • Consistency and Standards
  • Error Prevention
  • Recognition Rather than Recall
  • Flexibility and Efficiency of Use
  • Aesthetic and Minimalist Design
  • Help Users Recognize, Diagnose, and Recover from Errors
  • Help and Documentation

Methodology

Over a two week period, I held 8 team meetings. The company was divided into 4 groups for the first week, and a different 4 groups for the second week. Each group (for each week) was made up of participants that had varied roles and expertise within the organization. For example, one group had a junior and senior programmer, a customer support associate, a trainer, and a sales associate. The idea was to build groups with employees that may not normally trade ideas about the product.
Preview
During each 1 hour meeting, a different functional area was the focus. One group looked at scheduling, while another looked at invoicing, and so on. At the beginning of each meeting, we reviewed Jakob Nielsen’s 10 Usability Heuristics (listed below). We then went through 4-5 major user tasks within the assigned functional area; for example, Create a New Patient. Everyone would have a few minutes to perform the task and take notes, then I would choose one person, usually the one least likely to know how to perform the task, to guide us through the steps they took to complete the task.

As one teammate discussed their frustrations, another would offer tips, while another would chime in with why the feature was implemented in such a way. Teammates from customer support would let us know if this was particularly frustrating for the users that they spoke to on a daily basis, and new programmers would detail how they may have implemented such a feature. After frustrations were aired within the context of the heuristics, I would ask everyone to tell me how the feature would operate in a “perfect world” with “no strings attached” to lead into brainstorming.

Final Report

After the 2 weeks of meeting, I prepared a final report for the management team in two versions – one detailed report and one executive report. Here are some of the components of the detailed report:

  • Most often violated heuristics
  • General Pain Points (ex: button placement, help information, data pages and forms)
  • By Functional Area:
    • Table of Pain Points with Examples and Design Opportunities
    • Specific features with suggested design changes
    • Customer Support Statistics (ex: 30-40% of our support issues deal with reports)
  • Business Goals (ex: Reduce the effort to train current users)
  • Design Goals
  • Design Ideas (high level)
  • Structural Guidelines (ex: maintain a multi-user system)
  • Special Considerations
  • Risks
  • Opportunities
  • Next Steps
* “Any Role” = Product Manager, UI Designer, User Experience Researcher, Requirements Analyst, etc. Any role that participates in translating customer needs to software requirements and functional specifications.