%0 Conference Paper %B IEEE VIS Short Papers %D 2020 %T CrowdTrace: Visualizing Provenance in Distributed Sensemaking %A Li, Tianyi %A Belghith, Yasmine %A North, Chris %A Luther, Kurt %B IEEE VIS Short Papers %P 5 pages %8 10/2020 %0 Journal Article %J Proc. ACM Hum.-Comput. Interact. %D 2019 %T Dropping the Baton?: Understanding Errors and Bottlenecks in a Crowdsourced Sensemaking Pipeline %A Li, Tianyi %A Manns, Chandler J. %A North, Chris %A Luther, Kurt %K crowdsourcing %K Intelligence Analysis %K investigations %K mysteries %K sensemaking %K text analytics %X Crowdsourced sensemaking has shown great potential for enabling scalable analysis of complex data sets, from planning trips, to designing products, to solving crimes. Yet, most crowd sensemaking approaches still require expert intervention because of worker errors and bottlenecks that would otherwise harm the output quality. Mitigating these errors and bottlenecks would significantly reduce the burden on experts, yet little is known about the types of mistakes crowds make with sensemaking micro-tasks and how they propagate in the sensemaking loop. In this paper, we conduct a series of studies with 325 crowd workers using a crowd sensemaking pipeline to solve a fictional terrorist plot, focusing on understanding why errors and bottlenecks happen and how they propagate. We classify types of crowd errors and show how the amount and quality of input data influence worker performance. We conclude by suggesting design recommendations for integrated crowdsourcing systems and speculating how a complementary top-down path of the pipeline could refine crowd analyses. %B Proc. ACM Hum.-Comput. Interact. %I ACM %C New York, NY, USA %V 3 %P 136:1–136:26 %8 11/2019 %U http://doi.acm.org/10.1145/3359238 %R 10.1145/3359238 %0 Journal Article %J IEEE Transactions on Visualization and Computer Graphics %D 2019 %T The Effect of Edge Bundling and Seriation on Sensemaking of Biclusters in Bipartite Graphs %A Sun, Maoyuan %A Zhao, Jian %A Hao Wu %A Luther, Kurt %A North, Chris %A Ramakrishnan, Naren %K Bicluster %K bicluster visualizations %K bicluster-based seriation %K Bioinformatics %K Bipartite graph %K bipartite graph based visualizations %K data analysis %K data visualisation %K edge bundles %K edge bundling %K edge crossings %K exploratory data analysis %K graph theory %K Image edge detection %K Layout %K pattern clustering %K product bundles %K seriation %K Visual Analytics %X Exploring coordinated relationships (e.g., shared relationships between two sets of entities) is an important analytics task in a variety of real-world applications, such as discovering similarly behaved genes in bioinformatics, detecting malware collusions in cyber security, and identifying products bundles in marketing analysis. Coordinated relationships can be formalized as biclusters. In order to support visual exploration of biclusters, bipartite graphs based visualizations have been proposed, and edge bundling is used to show biclusters. However, it suffers from edge crossings due to possible overlaps of biclusters, and lacks in-depth understanding of its impact on user exploring biclusters in bipartite graphs. To address these, we propose a novel bicluster-based seriation technique that can reduce edge crossings in bipartite graphs drawing and conducted a user experiment to study the effect of edge bundling and this proposed technique on visualizing biclusters in bipartite graphs. We found that they both had impact on reducing entity visits for users exploring biclusters, and edge bundles helped them find more justified answers. Moreover, we identified four key trade-offs that inform the design of future bicluster visualizations. The study results suggest that edge bundling is critical for exploring biclusters in bipartite graphs, which helps to reduce low-level perceptual problems and support high-level inferences. %B IEEE Transactions on Visualization and Computer Graphics %V 25 %P 2983-2998 %8 07/2019 %N 10 %R 10.1109/TVCG.2018.2861397 %0 Journal Article %J Proc. ACM Hum.-Comput. Interact. %D 2018 %T CrowdIA: Solving Mysteries with Crowdsourced Sensemaking %A Li, Tianyi %A Luther, Kurt %A North, Chris %K crowdsourcing %K Intelligence Analysis %K investigation %K mysteries %K sensemaking %K text analytics %B Proc. ACM Hum.-Comput. Interact. %I Association for Computing Machinery %C New York, NY, USA %V 2 %U https://doi.org/10.1145/3274374 %R 10.1145/3274374 %0 Conference Paper %B CHI 2018 Workshop on Sensemaking in a Senseless World %D 2018 %T Crowdsourcing Intelligence Analysis with Context Slices %A Li, Tianyi %A Asmita Shah %A Luther, Kurt %A North, Chris %B CHI 2018 Workshop on Sensemaking in a Senseless World %8 04/2018