SUMMARY

Compared to physical media, digital media are extremely plastic: Previews enable one to experiment with a command before actually modifying data, while Undo provides the ability to return to a previous state should an entire series of actions prove undesirable. However, despite this plasticity, an observational study we conducted of artists and graphic designers suggests that current user interfaces make it difficult to quickly develop and explore sets of alternative solutions in parallel. Exploring sets of alternatives is a common practice among expert practitioners since it allows them to better understand the problem and its space of potential solutions. Rather than think in the abstract, sets of alternatives enable one to directly compare and contrast the relative merits of each possibility. While it is not impossible to explore sets of alternatives with current interfaces, it must be manually orchestrated by the user: Time must be spent preparing the user interface for exploration (e.g., by making copies of a solution before deriving a new standalone solution), and few mechanisms exist to ease the tasks of comparing the alternatives and keeping them in sync. Ultimately, these costs can be high enough to discourage users from exploring in both breadth and depth. As a result, current interfaces tend to lead to highly linear problem solving processes whereby a single solution is continually revised until deemed acceptable. We term such interfaces point-based interfaces because of their focus on one solution at a time.

To better support exploration, we introduce the concept of a set-based interface, or an interface that provides explicit support for simultaneously generating, manipulating, evaluating, and managing sets of alternative solutions for the same problem. Such an interface is intended to enable a problem solving process characterized by broad exploration and the simultaneous consideration of multiple solution instances. We present two tools demonstrating the concepts of a set-based interface, Side Views and Parallel Pies. Both are designed for use in image manipulation tasks. Side Views is an enhanced previewing mechanism that simultaneously displays sets of previews for one or more commands and their parameters. Parallel Pies, on the other hand, streamlines the process of “forking” so one can more easily pursue promising alternatives in parallel. In contrast to history-tracking tools that simply record all the states a person has visited, Parallel Pies allows an individual to easily spawn new standalone alternatives: As promising possibilities are discovered while interacting with a command, individuals may use Parallel Pies to duplicate their current document, apply the command to the copy, and insert the new result into the same workspace – all via a single action. Parallel Pies’ visualization tool then allows one to compare and contrast the generated alternatives.

To understand the impact these tools have on the problem solving process, we conducted two controlled, within-subjects laboratory studies and a third think-aloud study. In the first controlled study, subjects color-corrected images to make them match a known (visible) target image. Though it employed a well-defined task, the solution path was ill-defined, making experimentation essential to solving the problem well. In the second controlled study, subjects developed color schemes for watches, offering a view of how these tools perform in much more open-ended tasks. The final study paired individuals to work cooperatively on the same color scheme task as the second study.

This third study design helped to externalize individuals’ thought processes as they worked through problems as subjects needed to communicate with one another to solve each problem. The collective results from these studies indicate that Parallel Pies leads to a discernibly different problem solving process that subjects perceive to be more desirable. In particular, Parallel Pies leads to broader exploration and the creation of more solution alternatives in the same amount of time. The studies further indicate that this problem solving process can result in more optimal solutions when solving tightly constrained tasks, where optimality is measured by the number of operations required to derive a solution state from a given start state.

While Parallel Pies can lead to broader exploration, we found that too much exploration can actually lower solution quality under certain circumstances. In particular, when solving open-ended problems under tight time constraints, individuals can initially overuse the capability to broadly explore and not spend enough time maturing a single solution. However, there are indications that this issue would disappear after users have fully acclimated to this capability. Parallel Pies was designed to enable broad exploration by increasing the ease with which one can spawn a new, separate solution instance. One unexpected side-effect of this capability is that it can increase the likelihood of backtracking. Specifically, as individuals work, they sometimes use Parallel Pies to create back-up copies of different stages of their work, thereby increasing the frequency with which they return to a previous state to pursue alternative paths. Though the tool was not expressly designed to support this practice, this use suggests the need for history mechanisms that more completely track all states a user visits. In contrast to Parallel Pies, our studies show that the multiple previews of Side Views have a far more subtle effect on the problem solving process. While we find no evidence that multiple sets of previews can affect one’s performance, we did find that they can lead to individuals serendipitously discovering viable solution alternatives. That is, while interacting with a command, the multiple previews can suggest ways of solving a problem that the user had not originally considered. Side Views’ multiple sets of previews per parameter can also reduce the need to scan for settings of interest. In our studies, subjects moved commands’ sliders about half as much when Side Views were present, suggesting that users can find values of interest with less effort by using this tool.

Finally, our studies indicate that Side Views’ multiple previews can help ground communication between individuals as they cooperatively solve problems. Rather than abstractly describe how they wish to proceed, users can simply point to a Side View that is close to the desired direction. From these three studies, we conclude that user interface mechanisms that support the rapid generation and automatic management of new standalone alternatives are highly valued and useful facilities for solving complex, ill-defined problems. By making it easy to create sets of potential solutions that are simultaneously active, these capabilities streamline the process of exploring the design space and help users to perform concrete, side-by-side comparisons of their options.

We also conclude that current interface designs lead to highly linear problem solving processes due to their lack of mechanisms to support broad exploration. When such mechanisms are available, they are eagerly adopted and lead to much wider explorations of the problem space, espite them being completely optional components to the problem solving process. Thus, this work argues for the need to more fully investigate how process support tools (such as history mechanisms and previewing tools) impact the problem solving process.

While this research has documented how Side Views and Parallel Pies can assist in experimentation and exploration, this work also makes significant contributions in the methodology and metrics used to perform the evaluations. For over 20 years, mechanisms such as branching histories and enhanced versions of Undo have been proposed to facilitate experimentation. However, very few evaluations have been performed on these types of tools. The dearth of evaluations can be partially attributed to the lack of analytical methods appropriate to evaluating how these types of mechanisms affect the problem solving process.

Noting this lack of evaluation instruments, this work contributes a suite of metrics that formalize concepts such as backtracking, breadth and depth of exploration, and “dead-ends.” These metrics enable statistical comparisons to be performed across these dimensions, allowing a researcher to investigate whether their user interface mechanisms lead to significantly different problem solving strategies. A visualization called a process diagram aids in conveying results by rendering an individual’s problem solving process through a set of integrated directed graphs.