From cooking recipes to home improvement videos to software tutorials, people are increasingly turning to online instructions as guides for learning new skills or accomplishing unfamiliar tasks in everyday lives.

However, the number of available instructional materials even for a single task is easily in the magnitude of thousands, the diversity and the scale of the instructions introduce new user challenges in currently used software interfaces for authoring, sharing and consuming these naturally crowdsourced instructions.

It is difficult to find the contextually useful information because the interfaces are unfortunately not designed for effectively navigating and following the instructions, and do not support comparison and analysis for sensemaking of the various instructions.

For example, for cooking professionals like chefs, cooking journalists, and culinary students, it is important to understand not only the culinary characteristics of the ingredients but also the diverse cooking processes. For example, categorizing different approaches to cooking a dish or identifying usage patterns of particular ingredients and cooking methods are crucial tasks for them

RecipeScape is an analytics dashboard for analyzing and mining hundreds of instructions for a single dish. The interface was designed to solve a critical problem for cooking professionals, the need to understand not only the composition of ingredients but more importantly the diverse cooking processes. RecipeScape is powered by a novel computational pipeline that collects, annotates, computes structural and semantic similarities, and visualizes hundreds of recipes for a single dish. Cooking professionals and culinary students found that RecipeScape 1) empowers users with stronger analytical capabilities by enabling at-scale exploration of instructions, 2) supports user-centered queries like “what are recipes with more decorations?”, and 3) supports creativity by allowing comparative analysis like “where would my recipe stand against the rest?”. Moreover, the three visualization components allow users to reason and provide their own interpretations and explanations of how the recipes are grouped together in human language, suggesting user with appropriate tools can interpret clustering algorithms.

We believe our computational pipeline and interactive visualization techniques can be extended to other media like tutorial videos as well as are highly applicable to different types of sequential tasks, such as software workflows, manufacturing, and customer service manuals.

This work was presented at CHI 2018 in Montreal as “RecipeScape: An Interactive Tool for Analyzing Cooking Instructions at Scale”. For more detail, please visit our project website, https://recipescape.kixlab.org/