Image browsing, processing, and clustering for participatory sensing: lessons from a DietSense prototype

Abstract

Imagers are an increasingly significant source of sensory observations about human activity and the urban environment. ImageScape is a software tool for processing, clustering, and browsing large sets of images. Implemented as a set of web services with an Adobe Flash-based user interface, it supports clustering by both image features and context tags, as well as re-tagging of images in the user interface. Though expected to be useful in many applications, ImageScape was designed as an analysis component of DietSense, a software system under development at UCLA to support (1) the use of mobile devices for automatic multimedia documentation of dietary choices with just-in-time annotation, (2) efficient post facto review of captured media by participants and researchers, and (3) easy authoring and dissemination of the automatic data collection protocols. A pilot study, in which participants ran software that enabled their phones to autonomously capture images of their plates during mealtime, was conducted using an early prototype of the DietSense system, and the resulting image set used in the creation of ImageScape. ImageScape will support two kinds of users within the DietSense application: The participants in dietary studies will have the ability to easily audit their images, while the recipients of the images, health care professionals managing studies and performing analysis, will be able to rapidly browse and annotate large sets of images.