Card Sorting Methodology

summary | detail | methodology

A card sorting study often supplements an information architecture evaluation, and includes design, user sessions, and analysis.

Card Sorting Design

The design phase of a card sorting project includes identifying and recruiting representative users as participants and determining the card labels to be tested. For a typical project, the team identifies 25–40 labels representing content areas of the website or other information product. Use the following guidelines to design cards for your session:

Make cards for content areas that are most important to target users or mission-critical to your company.
Include cards for any new or revamped content areas or navigation labels.
Include cards with labels that prior experience suggests are misleading or ambiguous. For example, “search history,” “expert search,” or “tools” can be interpreted many ways.
Use cards with multidimensional labels. For example, “request email notification of new articles about nutrition” describes method, mode, and specific content. Using this rather complex label tests preferences for categorizing information by task, format, timeliness, or topic.
Include cards with alternative labels for content or concepts to discourage grouping of cards by word similarity alone.
Cards can include areas for participant notes, such as cross-references to other groups, lists of similar terms, or questions that participants have about the term.
Cards can include a checkbox to indicate that the term is unfamiliar or meaningless.

Card Sorting Sessions

A typical exploratory card sorting session includes the following steps:

A test administrator gives the participant a stack of 25–40 cards that are in no particular order or in an acknowledged default order such as alphabetical order. The participant is then asked to organize the cards into groups that seem logical.
Participants “think out loud” as they sort cards. Researchers record their comments and, if necessary, ask for clarification or elaboration. For example, a researcher might ask why a particular card seemed difficult to classify.
After the participant arranges the cards into groups, the administrator may ask them to write a name for each group on a blank card (while continuing to think out loud).

For a verification card sort, the administrator asks the participant to put each card into one of several groups that are already labeled.

Card Sorting Data Collection and Analysis

To collect qualitative data, you examine comments made by participants, including their reactions to the wording on the cards and their definitions of the labels they assigned to groups of cards. The observations of the test team and responses to any questionnaire or interview questions gathered during the session are also examined.

At a high level, the test team also examines the groups and group labels created by participants, to help answer questions such as:

How did participants group cards? What types of groups did they create (topic, task, audience, document type)?
What category names did they provide?
What cards did they have difficulty categorizing?
What cards did they place in unexpected categories?
What cards did they place in multiple groups?
What groups did they have difficulty naming?
For a verification card sort: What cards did they place in categories other than the expected categories? What predefined group labels did they seem to misinterpret?

The answers to these questions can help inform decisions about website or interface structure and site maps, part or chapter organization, or menu structure, as well as the labels used to identify the organizational elements of each.

To collect quantitative data, researchers record each pairing of two cards with the name that the participant gave to the group in which both appear. With this tabulated data, the team can examine which cards were placed in which groups, and how many participants put a particular card in any of similarly named groups. (Often, similar group labels such as “My Account” and “Member Services” can be further collected into “supergroups” for purposes of quantitative analysis.)

The tabulated data provides a way to look at the data at a low level, from the “bottom up,” and can help answer questions such as:

How many groups did participants create, and how many cards did they place in each group?
What cards were paired most often (regardless of the group label)? Under what group labels did the pairs appear?
What clusters, or sets of interrelated pairings, emerged? What are the strongest clusters—that is, which clusters are most extensive? Which occur most often?
Within a cluster, what cards were most often paired with other cards within the cluster?
What cards were associated with multiple clusters?
How are clusters related, and what clusters are isolated from all other clusters?
What cards were unexpectedly not paired together?

The answers to these questions can help inform decisions about organization, breadth and depth of content, ambiguous card labels, and navigation to related content areas.

Card Sorting Report

Using notes and tabulated card sort data, the usability team summarizes the findings, formulates conclusions, and provides recommendations for content organization, labels for content areas, indices, metadata, advanced search choices, or home page content. When card sorting is a component of an information architecture evaluation, its results are integrated into the overall information architecture report