Semantic Interaction Project

The goal of this project is to enable the creation of new human-centered computing tools that will help people effectively analyze large collections of textual documents by providing powerful statistical analysis functionality in a usable and intuitive form. To accomplish that, this project investigates “semantic interaction” in visual analytics as a method to combine the large-data computationally-intensive foraging abilities of formal statistical mining algorithms with the intuitive cognitively-intensive sensemaking abilities of human analysts. Semantic interaction enables users to inject their domain expertise into the algorithms by interacting directly with the data. For example, analysts synthesize hypotheses about a set of documents by simply re-organizing them within a spatial visualization, highlighting important sentences, or annotating in the margins. Meanwhile, the underlying statistical models learn from these actions and interactively respond to help spatially organize additional relevant information according to the user’s feedback.

Intellectual merit: Semantic interaction offers a new approach to interactive visual analytics that emphasizes usability. This research will (1) contribute new user interaction and visual feedback techniques for naturally controlling algorithms via the interactive sensemaking process; (2) contribute a flexible visual analytics framework that seamlessly integrates mathematical models with interactive visualization; and (3) evaluate the effectiveness of semantic interaction, which provides a quantitative mechanism to investigate the complex interplay between human intuition and formal statistical methods.

Broader impacts: This research will support a broad range of applications in visual text analytics, including intelligence analysis, funding portfolio management, and literature research. Participating agencies will test the software in ecologically valid settings. The software framework will be distributed to enable others to integrate additional models and to provide a usable platform for analysts. The project also proposes an educational and outreach agenda called “CS for CSI” that exploits the popularity of crime investigation stories to attract young students to technology research.

This project is supported by NSF HCC grant #1218346 entitled "Semantic Interaction for Visual Text Analytics", PI Chris North.

Links to project products:

InfoVis Taxomony: