Interaction Design:  Beyond Human Computer Interaction
by Jennifer Preece

Preece, J., Rogers, Y., & Sharp, H.. (2002). Interaction Design: Beyond Human-Computer Interaction.
A typical undergraduate level textbook to introduce you to the field, including both scientific background and usability design methods. One of the few that adequately addresses affective measures. [DS & DN]

Jump To:   1  2  3  4  5  6  7  8  9  10  11  12  13  14  15

Table of Contents:

Ch. 1:  Intro to Interaction Design
Ch. 2:  Conceptual  Models and Interface Metaphors
Ch. 3:  Cognition and Mental Models
Ch. 4:  Conversational, Coordination, Awareness Mechanisms and Collaboration
Ch. 5:  Affective Aspects of Interfaces and User Frustration
Ch. 6:  The Process of Interaction Design and Life-Cycle Models
Ch. 7:  Needs and Requirements (Scenarios, Use Cases, Essential Use Cases, HTA)
Ch. 8:  Prototyping, Conceptual Design and Physical Design
Ch. 9:  Ethnography and Participatory Design
Ch. 10:  Introduction to Evaluation
Ch. 11:  Evaluation Paradigms, DECIDE Framework
Ch. 12:  Observing Users
Ch. 13:  Interviews, Questionnaires, Inspections and Walkthroughs
Ch. 14:  User Testing, GOMS, KLM, Fitt's Law
Ch. 15:  Example Applications of Interaction Design

Preece's Summary of the Chapters:  http://www.id-book.com/chapter_index.htm

Remember, usability is of utmost importance to Interaction Design, while user-experience follows (see graph on p. 19 of book)


usability goals:  at center of Interaction Design
user-experience goals:  outer ring of diagram (secondary to usability goals)

Chapter 1:  What is Interaction Design?

Main Goals of this Chapter:

Good and Poor Design

What to Design

Match goals to users - get them involved

Interaction Design:

Definition:  "Designing interactive products to support people in their everyday and working lives"

Interaction Design comes from a multidisciplinary background, extends and enhances the way people work, communicate and interact

Interaction Design involves four basic activities:

  1. Identifying needs and establishing requirements
  2. Developing alternative designs that meet those requirements
  3. Building interactive versions of the designs so that they can be communicated and assessed
  4. Evaluating what is being built throughout the process

Evaluating what has been built is the heart of Interaction Design

Three characteristics of the Interaction Design Process:

  1. Users involved throughout the development of the project
  2. Specific usability and user experience goals should be identified, documented and agreed upon at the beginning
  3. Iteration through the four activities (above) is inevitable
     

The Goals of Interaction Design:  Usability Goals & User Experience Goals

- Usability Goals:  concerned with meeting a usability criteria (e.g. efficiency)

- User Experience Goals:  User experience is what the interaction with the system feels like to the users (subjectively)

Usability:  Design and Usability Goals (generalized abstractions)

Usability Principles / Heuristics (heuristics are design principles used in practice - more prescriptive usability principles that are used as a basis for evaluating a system / prototype)

There are always tradeoffs with usability - can't over constrain things, because it limits how much info is displayed

(top)

Chapter 2:  Understanding and Conceptualizing Interaction

have a clear understanding of what, why and how you are going to design something before writing any code.

Goals of the chapter:

Understanding the Problem Space

- the problem with solving a problem on the nuts and bolts level is that critical usability goals and user needs can be overlooked

- the design of physical aspects are best done AFTER we understand the nature of the problem space

- to understand the problem space:  clarify usability and user experience goals.  Make explicit your implicit assumptions and claims.

Framework for making your implicit assumptions explicit:

Conceptual Models

A conceptual model is a description of the proposed system in terms of a set of integrated ideas and concepts about what it should do, behave and look like, that will be understandable by the users in the manner intended.

"The most important thing to design is the users conceptual model." (David Liddle, '96)

To Develop a Conceptual Model:

Two types of conceptual models are:

The best type of conceptual model to use depends on the nature of the activity.  Often the best answer is a hybrid (such as shopping on the Internet).  However, mixing conceptual models will raise the complexity of the system.

Interface Metaphors

definition:  a conceptual model that has been developed to be similar in some ways to aspects of a physical entity, but that also has its own behaviors and properties

Interface metaphors combine the familiar with new concepts

Benefits:  a good orientation device
Drawbacks:  often the metaphor looks / feels like the physical entity, when they should just map the familiar with the unfamiliar so that users can learn the new (unfamiliar)

There is a growing opposition to metaphors because they can break the rules of the object they represent, they can be too constraining, can conflict with design principles, can cause misunderstanding of system functionality, can limit the designer's imagination, and can have overly literal translation of existing bad design.  See Metaphors description for more information.

Interaction Paradigms

- moving away from WIMP interface / paradigm

- new paradigms:  ubiquitous computing, pervasive computing, wearable computing, augmented reality, attentive environments

From Conceptual Models to Physical Design

Interaction design is an ITERATIVE PROCESS, involving:

The book describes ways of DOING INTERACTION DESIGN

Issues in testing prototypes:

Physical design decisions come out of conceptual decisions (i.e. what information, how to structure graphical objects, what feedback navigation and mechanisms, what kinds of icons…).

(top)

Chapter 3:  Understanding Users

This chapter focuses on USERS and COGNITION.  Cognitive aspects of Interaction Design include:

Main aims of chapter:

Norman said there are two modes of cognition:  Experiential (real world experiences) and Reflective (thinking, comparing, deciding, etc).  Both are necessary for everyday life.

Cognition has been described in SIX KINDS OF PROCESSES:

  1. Attention - selecting things to concentrate on
  2. Perception / Recognition - how information is acquired from the environment via sense organs and translated into experiences (vision is the most dominant)
  3. Memory - recalling various knowledge.  We filter what knowledge to process / memorize. (most researched area)
  4. Learning - how to do something (like learning to use a program)
  5. Reading / Speaking / Writing - using language
  6. Problem Solving / Planning / Reasoning / Decision Making - involves reflective cognition

Often designers try to emulate the physical world with designs in the digital world.  Sometimes this works well, other times it doesn't.

Conceptual Frameworks for Cognition:

Users' Mental Models

- defined as:  when people are using a system, they develop knowledge of how to use the system and to lesser extent how the system works.

- the mental model is used to help people carry out tasks.  It can also give suggestions on what to do in unpredictable situations

- in cognitive psychology, mental models are defined as some sort of internal construction of the external world that are manipulated enabling predictions and inferences to be made

- w/r/t system design:  ideally, the users' mental models should match the designer's conceptual model

- to increase transparency- might make system image easier to learn (p. 95 example?)

Information Processing

- another approach to conceptualize how the mind works:  through metaphors and analogies

- thinks of the mind as an information processor

- mental representations can be images, mental models, rules, other knowledge forms

the human processor model (Card, et. al 1983) is the best known approach (see p. 96)

- there has been an increase in people studying cognitive activities 'in the wild' - in the context in which they take place (how can things in the environment aid human cognition and lighten the cognitive load?)

Alternative frameworks have been suggested:  External cognition and Distributed Cognition

External Cognition

main idea:  people interact with or create information through using a variety of external representations (books, etc.)

- an impressive array of technology has been created by humans to aid cognition (calculators, pens, etc)

- these tools have combined with external representations to extend and support our ability to carry out cognitive activities.

- some of the main goals of this:

Back to Interaction Design:  PROVIDE EXTERNAL REPRESENTATIONS AT THE INTERFACE TO REDUCE MEMORY LOAD (visualizations, cues, etc).

Informing Design:  From Theory to Practice

Theories, models and frameworks provide abstractions for thinking about phenomena.  They provide generalizations, but can be difficult to digest.  For this reason researchers have tried to make them more practical by providing design principles / concepts, design rules, analytic methods and design / evaluation methods.

This has helped - for instance - the human processor model (Card, 83) which has been simplified into GOMS.

(top)

Chapter 4:  Designing for Collaboration and Communication

humans are SOCIAL beings

Purpose of the chapter:  look at ways interactive systems could be developed to support and extend communication and collaboration between peoples.

Social Mechanism in Communication and Collaboration:

Rules, procedures and etiquette have been established to help people know how to behave in social groups, such as:

Conversational Mechanisms:

How to design collaborative technologies to support conversation:

Synchronous CMC: (p.112 table 4.1)

Asynchronous CMC:

Many new communication technologies combine the above, and try to provide new / novel ways to communicate

Coordination Mechanisms:

Collaborative activities require us to coordinate with each other, so we need to figure out how to work with others to progress through the activities.  Examples include:

How to design collaborative technologies to support coordination:

Shared calendars, schedulers, project management tools, and workflow tools have been developed to support coordination activities.

People tend not to follow conventions, because they are often not socially acceptable.  Failure to make them socially acceptable can cause people to not use the system in the way intended or can cause them to abandon it totally.

Awareness Mechanisms:

These provide others with awareness of who is around, what is happening, who they are talking to.  This requires knowing when is an appropriate time to interact with others and to get / pass information.

How to design collaborative technologies to support awareness:

Function is to make others aware of the others they are collaborating with.  For example:

 

Ethnographic studies of collaboration and communication

A main approach to informing the design of collaborative technologies that takes into account the social concerns is to carry out an ethnographic study.

This can be a home, work, school, public place (setting)

Since Suchman's "Seminal Work" many companies have invested in ethnographic studies to see how work actually gets done in a range of companies (government too)

 

Conceptual Frameworks:  Language / Action Framework and Distributed Cognition

Language / Action Framework:

Distributed Cognition:

 

Summary:

(top)

Chapter 5:  Understanding How Interfaces Affect Users

Goals of this chapter:  AFFECTIVE LEARNING - a way to design systems to elicit positive responses from users (feeling at ease, being comfortable, enjoying the experience)

Affective Aspects of HCI

Expressive Interfaces

User Frustration

Can be caused by:

- application doesn't work / crashes
- system won't do what the user wants it to do
- user's expectations aren't met
- when the system doesn't provide enough information to enable the user to know what to do
- vague error messages
- condemning error messages
- when the interface's appearance is patronizing, gimmicky, noisy
- when the user does a long series of steps to complete a task, only to discover an error was made and they have to start over

Things to avoid:

- Gimmicks:  "under construction"
- Error messages:  "the system has unexpectedly quit" (see Shneiderman's guidelines for error messages:  don't use "FATAL" "INVALID" or "BAD", or long hex codes - try to provide context sensitive help)
- Overburdening the user:  forcing them to upgrade, install plugins, do housekeeping just to use a product
- Unpleasant Appearance:  too much crap on the page, featuritis (too many features), weird sound effects, childish interface, poorly laid out input devices

Dealing with User Frustration:

- people will vent, by beating the hell out of their computers, flaming, etc.
- offer helpful error messages that offer a way to fix the problem, and offer hints, help guides, cartoon agents, etc. that can help point the user in the right direction
- Reeves and Naas (1996) argue that computers should apologize when they mess up

Anthropomorphism in HCI:  How much is enough?

Anthropomorphism- assigning human traits to non-human things (dancing butter, talking soda cans, dogs, cars, etc.)
- used heavily in advertising

People debate how much of this to use in system design.  They can add a human feel to the system, but can also get annoying.

Which is more preferable?
- "Hello Matt!  Welcome back.  It's nice to see you again.  Now, what were we doing last time?  Ah, yes, problem five.  Lets get started again."
- "User 24, commence exercise 5."

Or, when doing something wrong:  "Now Matt, that's not right, you can do better than that.  Try again." vs. "Incorrect, try again."

The answer: 
Pros:  Reeves and Naas (1996) found it is helpful to use praise in educational settings when people do something right.  It increased students willingness to continue working. 
Cons:  However, others argue this can make you feel stupid, anxious, inferior.  People hate when a computer character shakes their finger at them and says "you can do better than that, Matt, try again".  In this case, many prefer the impersonal message "Incorrect, try again".

Virtual Characters

virtual characters are becoming more common.  They can be used on the web, in video games, as learning companions, wizards, newsreaders, etc.  However, they can be misleading (people confide in them), they can be very annoying and frustrating ("Clippy" from MS Office 97, etc).

- categorized by degree of anthropomorphism:  synthetic characters, animated agents, emotional agents, embodied conversational agents

Design Implications:  which one to use?
- believability:  the extent to which users come to believe an agent's intentions and personality
- appearance:  better to use cartoon-like characters, or those resembling humans?  it depends on the situation- often the cartoon character is more believable / acceptable
- behavior:  how does the agent move, gesture, refer to things?  facial expressions can show emotion (remember we want constructive feedback rather than conveying inferiority / stupidity / patronizing effect on users)

Summary:

(top)

Chapter 6:  The Process of Interaction Design

The ultimate goal of design is to develop a product that helps its users achieve their goals.  Developing a product must begin with gaining understanding of what is required of it.

The goals of this chapter are to:

What is Interaction Design?

dictionary:  "design is a plan or scheme conceived in the mind and intended for subsequent execution"

The plan or scheme must be informed with knowledge about its use and the target domain, together with practical constraints such as materials, cost and feasibility.

In Interaction Design, we investigate the artifact's use and target domain by taking a user-centered approach to development.  The users' concerns direct the development rather than technical concerns.

Design is also about trade-offs and about balancing conflicting requirements.  Generating alternatives is a key principle and one that should be encouraged in interaction.  "To get a good idea, get lots of ideas." (Mark Rettig)

Typically there is a group of designers.  Therefore, plans should be captured and expressed in a way that allows for review, such as sketches, descriptions in natural language, a series of diagrams, and building prototypes.

Four Basic Activities:

1.  Identify needs and establish requirements
- Who is the target user?
- What kind of support will the interactive product provide?

2.  Develop alternative designs that meet those requirements
- suggest ideas to meet the requirements
- Conceptual design:  produce the conceptual model for the product
- Physical design:  consider the details of the product (colors, sounds, images, menu design, icons, etc.)  Alternatives are considered at every point.

3.  Build interactive versions (so that they can be communicated and assessed)
- a software version is not required- paper based prototypes are quick and cheap to build
- through role-playing, users can get a real sense of what it is like to interact with the product

4.  Evaluate the designs (measure their acceptability)
- determine the usability of the product or design.  Criteria are:  how appealing is it?  how well does it match the requirements?  Is the product fit for the purpose?
- Evaluation results are fed back into further design (FEEDBACK / ITERATIVE DESIGN PROCESS)

Three Characteristics of Interaction Design:

1.  Focus on the USERS
- involve users in the interactive design process, provide opportunities for evaluation and user feedback

2.  Specific usability and user experience goals
- identify and clearly document these at the beginning of the project.  They help designers to choose between different alternative designs

3.  Iteration
- allows for designs to be refined.  It is always necessary to revise ideas in light of feedback, several times.  Innovation rarely emerges whole and ready to go.  Iteration is inevitable because designers never get the solution right the first time

Practical Issues in Interaction Design:

1.  Who are the users?
- those who directly interact with the system.  However, can be any stakeholder:  purchaser, testers, people receiving products from the system
- primary user:  directly use it
- secondary user:  occasionally use it or use through intermediary
- tertiary user:  affected by the system, or will influence its purchase
- stakeholders:  people or organizations affected by the system who influence the system requirements

2.  What do we mean by "needs"?
- we need to understand the characteristics / capabilities of users, what they are trying to achieve, how they currently achieve it, and whether they would achieve their goals more effectively if they were supported differently
- characteristics that impact a product's design:  users physical characteristics (ergonomics:  size of hands, height, etc), strength of product (so a child can't break it), cultural diversity of intended users
- representative users MUST be consulted!
- users rarely know what is possible.  Therefore, users cannot tell us what they "need" to do achieve their goals.
- We need to examine existing task (Activity Theory?) and what the tasks' context, requirements, collaborative nature, and procedure is.  Then we can envision the task being done in a new way (scenarios, etc.)

3.  How do you generate alternative designs?
- it is easy to stick with something that is "good enough".  Humans stick to what they know works.
- innovations arise from cross-fertilization from different applications- allows us to "break out of the box"
- often browsing a collection of designs will inspire designers to consider alternative perspectives and solutions.  Designers are trained to consider alternatives, software people are not.
- design is a process of balancing constraints and constantly trading off one set of requirements with another, and the constraints may be such that there are few viable alternatives available.
- alternatives come from looking at other, similar designs, and the process of inspiration and creativity can be enhanced by prompting a designer's own experience and by looking at others' ideas and solutions.  ("Flair and creativity" : research and synthesis)

4.  How do you choose among alternative designs?
- there are factors that are externally visible and measurable and those that are hidden from the users' view.  Focus on the external / visible.
- prototypes can be used to evaluate with peers and users
- fundamental user-centered design:  choose between alternative designs by letting users and stakeholders interact with them and by discussing their experiences, preferences and suggestions for improvement.
- technical feasibility:  some are just not possible
- quality thresholds:  usability goals lead to criteria.  This USABILITY CRITERIA need to be set early on and checked frequently.
- usability engineering:  the process of writing down formal, verifiable and measurable usability criteria.  Some suggestions:

Lifecycle Models:  Showing how the activities are related

- lifecycle models:  represent a set of activities and how they are used; management tools; simplified versions of reality

- some models from software engineering:  waterfall, spiral, RAD, etc

- some models from HCI:  Star, usability engineering

- a simple lifecycle model for interaction design:  see p. 186, Fig. 6.7



(note how the above principles apply to this model)

Summary:

There are four basic activities in the interactive design process:
1.  Identify needs and establish requirements
2.  Develop ([re]design) alternative designs that meet those requirements
3.  Build interactive versions of the designs so that they can be communicated and assessed
4.  Evaluate them

These are permeated with three principles:
1.  Involve users early in the design process and evaluation of the artifact
2.  Define quantifiable & measurable usability criteria
3.  Iteration is inevitable

Key characteristics of the interaction design process are explicit incorporation of user involvement, iteration and specific usability criteria.

Before you can begin to establish requirements, you must understand who the users are and what their goals are in using the device.

Looking at others' designs provides useful inspiration and encourages designers to consider alternative design solutions, which is key to effective design.

Usability criteria, technical feasibility, and users' feedback on prototypes can all be used to choose among alternatives.

Prototyping is a useful technique for facilitating user feedback on designs at all stages.

Lifecycle models show how development activities relate to one another.

The interaction design process is complementary to lifecycle models from other fields.

(top)

Chapter 7:  Identifying Needs and Establishing Requirements

This chapter talks about different ways to gather requirements by introducing:

What are we trying to achieve in this design activity?

  1. Understand as much as possible about users, task, and context
  2. Produce a stable set of requirements

How can we do this?

Why bother getting it right?

Requirements need clarification, refinement, completion and re-scoping.

Input:  requirements document (maybe) 
Output:  stable requirements

Why 'establish' requirements?

Types of requirements:

Data Gathering Techniques:  (see p.214 / Table 7.1 for excellent graph)

Which of the above data gathering techniques to use?  The above techniques differ in the amount of time, level of detail and risk associated with the findings, and the knowledge the analyst requires

The choice of technique is also affected by the kind of task to be studied: 

Problems with data-gathering:

Guidelines:

Task Descriptions:  (more on p. 226-231)

Scenario for a shared calendar:

“The user types in all the names of the meeting participants together with some constraints such as the length of the meeting, roughly when the meeting needs to take place, and possibly where it needs to take place. The system then checks against the individuals’ calendars and the central departmental calendar and presents the user with a series of dates on which everyone is free all at the same time. Then the meeting could be confirmed and written into people’s calendars. Some people, though, will want to be asked before the calendar entry is made. Perhaps the system could email them automatically and ask that it be confirmed before it is written in.”

Use case for a shared calendar:

1. The user chooses the option to arrange a meeting.
2. The system prompts user for the names of attendees.
3. The user types in a list of names.
4. The system checks that the list is valid.
5. The system prompts the user for meeting constraints.
6. The user types in meeting constraints.
7. The system searches the calendars for a date that satisfies the constraints.
8. The system displays a list of potential dates.
9. The user chooses one of the dates.
10. The system writes the meeting into the calendar.
11. The system emails all the meeting participants informing them of them appointment

Alternative courses for a shared calendar:

Some alternative courses:
    5. If the list of people is invalid,
        5.1 The system displays an error message.
        5.2 The system returns to step 2.
    8. If no potential dates are found,
        8.1 The system displays a suitable message.
        8.2 The system returns to step 5.

Example Use Case Diagram for a shared calendar:

Example Essential Use Case for a shared calendar:

Task Analysis:  (p.231-234)

Task analysis is an umbrella term that covers techniques for investigating cognitive processes and physical actions, at a high level of abstraction and in minute detail.

Hierarchical Task Analysis - involves breaking down a task into subtasks, then sub-sub-tasks and so on.  These are grouped as plans which specify how the tasks might be performed in practice

Example Hierarchical Task Analysis:  Borrowing a book from the library

0. In order to borrow a book from the library
    1. go to the library
    2. find the required book
        2.1 access library catalogue
        2.2 access the search screen
        2.3 enter search criteria
        2.4 identify required book
        2.5 note location
    3. go to correct shelf and retrieve book
    4. take book to checkout counter

Example HTA (Plans):

Example HTA (Graphical):


Summary:

•

(top)

Chapter 8:  Design, Prototyping and Construction

This chapter will cover:  prototyping and construction (low and high fidelity prototyping, vertical and horizontal compromises); conceptual design (conceptual model, using scenarios and prototypes in conceptual design); and physical design (guidelines and widgets).

Flow of Interaction Design:
Identify Needs / Requirements (Ch. 7) --> Prototype cycles / Design (Ch. 8) --> Construction

What is a Prototype?

Why Prototype?

What gets prototyped:  technical issues, work flow, task design, screen layouts / information displays, and any difficult / controversial / critical areas

Low-fidelity Prototyping:

High-fidelity Prototyping:

Compromises associated with Prototyping:

Construction:  taking a prototype and making it whole by engineering a complete product (focus on quality:  usability, reliability, robustness, maintainability, integrity, portability, efficiency, etc.)

 

Conceptual Design:  From requirements to design

Three Perspectives for a Conceptual Model:

  1. Which interaction mode?

    - how user invokes actions
    - Activity-based:  instructing, conversing, manipulating and navigating, exploring and browsing
    - Object-based:  structured around real-world objects

  2. Which interaction paradigm?

    - desktop paradigm with WIMP interface
    - ubiquitous computing
    - pervasive computing
    - wearable computing
    - mobile devices, etc.

  3. Is there a suitable metaphor?

    - Interface metaphors combine familiar knowledge with new knowledge in a  way that will help the user understand the product
    - 3 steps:  understand functionality, identify potential problem areas, generate metaphors
    - evaluate metaphors:  How much structure does it provide?  How much is relevant to the problem?  Is it easy to represent?  Will the audience understand it?  How extensible is it?

Expanding the Conceptual Model:

Scenarios:  Using them for Conceptual Design

- scenarios can be used to explicate existing work situations, but are more commonly used for expressing proposed or imagined situations to help in conceptual design.

- they can be used through design in various ways:  as scripts for user evaluation of prototypes, as a means of co-operation across professional boundaries

- in extreme cases, plus and minus scenarios can be used, which attempt to capture the most positive and the most negative consequences of a proposed design solution

Prototypes:  Using them for Conceptual Design

- prototypes allow for evaluation of emerging ideas

- low-fidelity prototypes are used early on, while high-fidelity prototypes are used later

Physical Design: Getting Concrete

There is no rigid border between conceptual and physical design... they are all iterative processes.  Often in conceptual design some detailed issues come up in the iterations.  The important part is that in the conceptual design that we don't get tied to physical constraints early as they will inhibit creativity and limit our options.

Summary:

(top)

Chapter 9:  User-Centered Approaches to Interaction Design

The main aims of this chapter are to:

Why involve users at all?

  1. To manage their expectations:  no surprises, communicate their expectations, get a common set of realistic expectations
     
  2. So users feel ownership:  by making users active stakeholders, they are more likely to forgive / accept problems, and are more likely to accept the final product

Degrees of user involvement:

What is a user-centered approach?  User-centered approach is based on:

Understanding Users' Work:  Ethnography

Contextual Design:  developed to handle data collection and analysis from fieldwork for developing a software-based product (used commercially quite widely)  There are seven parts to Contextual Design:

1.  Contextual Inquiry
2.  Work Modeling
3.  Consolidation
4.  Work Re-design
5.  User Environment Design
6.  Mock-up and test with customers
7.  Putting it into Practice

1.  Contextual Inquiry:  an approach to ethnographic study where the user is an expert, and the designer is an apprentice.  It is a form of interviewing, but takes place at the users' workplace / workstation, and is often 2-3 hours long.  Four main principles of contextual inquiry are:

1.  Context:  see workplace and what happens
2.  Partnership:  user and developer collaborate
3.  Interpretation:  observations interpreted by user and developer together
4.  Focus:  project focus to help understand what to look for

2.  Work Modeling:  In interpretation sessions, models are drawn from the observations.  Five models are:

3.  Consolidation:  each contextual inquiry (one for each user / developer pair) results in a set of models, which need to be consolidated into one view of 'the work'

Participatory Design:  (Scandinavian background) emphasizes social and organizational aspects - based on study, model-building and analysis of new and potential future systems.  Aspects to user involvement include:

Benefits of Participatory Design:  “Computer-based systems that are poorly suited to how people actually work impose cost not only on the organization in terms of low productivity but also on the people who work with them. Studies of work in computer-intensive workplaces have pointed to a host of serious problems that can be caused by job design that is insensitive to the nature of the work being performed, or to the needs of human beings in an automated workplace.”  [Kuhn, S. in Bringing Design to Software, 1996]

PICTIVE:  Plastic Interface for Collaborative Technology Initiatives through Video Exploration:  Intended to empower users to act as full participants in design

Materials used are:

Equipment required:

Before a PICTIVE session:

A PICTIVE session has four parts:

CARD:  Collaborative Analysis of Requirements and Design - Similar to PICTIVE but at a higher level of abstraction: explores work flow not detailed screen design

Summary:

Exercise:

This exercise is to be done in pairs.

Consider a website application for booking theatre or cinema tickets online

(a) Think about how you would design such a site, and sketch out some ideas
(b) Run a CARD session with a colleague acting as a ‘user’ to map out the functional flow of the website
(c) Ask your colleague to produce some scenarios of how the system may be used. Meanwhile, prepare some ‘empty’ templates for a PICTIVE session for this system, using paper, sticky notes and pens
(d) Run a PICTIVE session to develop the online booking system collaboratively, using PICTIVE.

(top)

Chapter 10:  Introducing Evaluation

What, Why and When to Evaluate...

Goals of this chapter:

What to evaluate:

iterative design and evaluation is a continuous process that examines:  early ideas for a conceptual model; early prototypes of the new system; later, more complete prototypes

Designers need to check that they understand users' requirements

Why evaluate:

- because user experience can be extremely important for a product's success

- because the cycle of design and testing is the only validated methodology in existence that will consistently produce successful results

Five good reasons for investing in user testing (Tognazzini):
- fixed problems before the product is shipped
- the team can concentrate on real problems
- Engineers code instead of debating
- Reduced time to market
- sell without bothering to release patches

When to evaluate:

- when it's brand new:  develop markups of the product to elicit reactions from the potential users
- upgrades:  compare user performance & attitude w/ previous versions

Two main types of Evaluation:  Formative Evaluation and Summative Evaluation

  1. Formative Evaluation:  done at different stages of development to check that the product meets users needs
  2. Summative Evaluation:  assess the quality of a finished product

it is best to evaluate throughout design- from the first descriptions / sketches, all the way to the final product

Design proceeds through iterative cycles of design / test / re-design - where evaluation is a key ingredient for a successful design

(top)

Chapter 11:  An Evaluation Framework

Goals of this chapter:

User studies focus on how people behave, in their natural environments, or in the laboratory, with old technologies and with new ones.

Evaluation Paradigms:

Evaluation Techniques:

Observing users (ch. 12)

Asking users their opinions (ch. 13)

Testing users' performance (ch. 14)

DECIDE:  A framework to guide evaluation

Determine the goals the evaluation addresses (what, who, why...)

Explore the specific questions to be addressed (break down into subquestions- e.g. consumers' attitudes, security, interface, reputation of system, trust, adequate access)

Choose the evaluation paradigm and techniques to answer the questions (paradigm selected determines the techniques used.  Practical issues, ethical issues, and tradeoffs must be considered)

Identify the practical issues (select users, stay on budget, stay on schedule, find evaluators, select equipment)

Decide how to deal with the ethical issues (informed consent form, privacy / confidentiality, and let the participants know the goals of the study, what will happen to the findings, privacy of information, etc..)

Evaluate, interpret, and present the data (what data to collect, how to analyze and present depends on the paradigm used.  Need to consider reliability, validity, biases, scope and ecological validity)

Pilot Studies:

(top)

Chapter 12:  Observing Users

The goals of this chapter:

Goals and questions:  provide a focus for observation

Paradigms:  guide all evaluation studies

What & When to observe:

observing is useful at any time during product development
- early on in design:  helps designers understand users' needs
- later on:  to examine whether the developing prototype meets users' needs

evaluators can be an onlooker, a participant or an ethnographer

Approaches to Observation:

How to observe:

Data collection:

Indirect Observation:  Tracking users' activities

Techniques include:

Analyzing, Interpreting and Presenting the Data

There are three types of data:

  1. qualitative analysis to tell a story
    1. review data and identify key themes
    2. record themes in a coherent / flexible form
    3. recode the data and time of data analysis session
    4. check your understanding with the people you observe
    5. iterate process until your story represents what you observed
    6. report findings to development team (oral or written report)

    analyzing and reporting ethnographic data (from participant observation, interviews, artifacts:

  1. look for key events
  2. look for patterns of behavior (in various situations and players)
  3. compare sources of data
  4. report findings
  1. qualitative analysis for categorization
    1. look for incident patterns
    2. analyze data into categories (content analysis)
    3. determine the content categories reliability and inter-research reliability ratings
    4. analyze discourse (conversation analysis)
       
  2. quantitative data analysis
    1. video data collected in usability labs
    2. annotated
    3. recording to calculate performance times
    4. analyze statistically

Finally, feed the findings back into design!

Summary

(top)

Chapter 13:  Asking Users and Experts

Interviews and Questionnaires are used in "quick and dirty" evaluation, in usability testing, and in field studies to ask about facts, behavior, beliefs and attitudes.

Interviews

Questionnaires

Heuristic Evaluation

Walkthroughs

Summary

- Interviews can be structured, semi-structured, or unstructured.  Structured and semi-structured are designed to be replicated.  Questions can be open or closed (format)

- Focus groups are a type of group interview.

- Questionnaires are a cheap and easy way to reach large numbers of people.

- typically 5 experts can find 75% of the usability problems.

- heuristic evaluation is cheaper and more flexible than user testing.

- User testing and heuristic evaluation often reveal different usability problems.

- pluralistic and cognitive walkthroughs are focused and good for evaluating a small part of the interface.

(top)

Chapter 14:  Testing and Modeling Users

A central part of Interaction design is user testing.

Usability testing uses a combination of techniques, including user testing and user satisfaction questionnaires.  User testing is of central concern.

The end of this chapter talks about GOMS (Goals, Operators, Methods, Selection rules), KLM (Keystroke Level Model) and Fitt's Law.

User Testing

Using the DECIDE Framework for User Testing

Using Experiments for Usability Evaluation

Design Advantages Disadvantages
Different Participants No order effects Many participants needed. Individual differences between participants is a problem.  Can be offset to some extent by randomly assigning to groups.
Same participants Eliminates individual differences between experimental conditions. Need to counterbalance to avoid ordering effects.
Matched participants Same as different participants, but the effects of individual differences are reduced. Can never be sure that subjects are matched across variables.

Predictive Models

1.  GOMS:  Goals, Operators, Methods, Selection rules (Card, et. al 1983)
- see Carroll Ch. 4 for more information

Goals:  the state the user wants to achieve
Operators:  the cognitive processes and physical actions performed to attain those goals (decide which search engine to use, think up a keyword.  The goal is obtained; an OPERATOR is executed)
Methods:  learned procedures for accomplishing the goals, steps to do so (drag mouse over field, type in keywords, press 'Go' button)
Selection rules:  determine which method to select when there is more than one available

2.  The Keystroke Level Model (KLM)  (Card et. al, 1983)

Pros and Cons of GOMS:

Pros:  allows for comparative analysis for different interfaces or computer systems relatively easily
    outcome:  counter-intuitive, help make decisions about the effectiveness of new products

Cons:  not often used for evaluation purposes because of its highly limited scope
    only good for predicting expert performance, and error is not modeled (average users not predicted)
    many unpredictable factors come into play

A Con of Predictive Models:  they can make predictions about predictable behavior, but it is difficult to use them as a way of evaluating how systems will be used in the real world.  They are only useful for comparing the efficiency of different methods in completing a short, simple task.

3.  Fitt's Law  (Paul Fitts, 1954)

Summary

(top)

Chapter 15:  Design and Evaluation in the Real World:  Communicators and Advisory Systems

Key Issues:

From Requirements to Design:

Case study:  Nokia's mobile communicator

- used ethnographic research and did scenarios and task models to get requirements
- (method) followed participatory design- involved users throughout
- (method) used interface metaphors
- (method) followed with frequent low fidelity prototypes based on alternative designs / immediate evaluation (yielded invaluable insight to designers)
- wrote usage scenarios (high level descriptions of device in use)
- user testing (usability tests):  did summative testing before release (at end, not throughout [formative]) and questionnaires after release

Case study:  redesign of a telephone response information system (TRIS)

- current system was very hard to use- a deep menu system over the phone- users can't remember it without cues
- GOMS / KLM used to show how interface supported users' tasks
- heuristic evaluation used as an alternative method for showing usability problems (expert review of system)
- methods complemented each other and showed benefit of doing a re-design

Key points:

(top)

Jump To:   1  2  3  4  5  6  7  8  9  10  11  12  13  14  15