Sangho Suh

selected publications [full list]

1. NIPS

   Re-VACNN: Steering convolutional neural network via real-time visual analytics

   Chung, Sunghyo, Park, Cheonbok, Suh, Sangho, Kang, Kyeongpil, Choo, Jaegul, and Kwon, Bum Chul

   In Future of interactive learning machines workshop at the 30th annual conference on neural information processing systems (NIPS) 2016

   Abs PDF Slides Demo Presentation

   Recently, deep learning has become exceptionally popular due to its outstanding performances in various machine learning and artificial intelligence applications. Convolutional neural network (CNN), a representative model of deep learning, has been successfully applied to solve computationally burdening tasks in various applications like computer vision. Despite its outstanding capability, training a CNN model properly is time-consuming and prone to overfitting and/or bad local minima. To address these issues, this study aims at improving the interpretation of the training process and using it for subsequent human intervention, specifically steering the training process of CNN model in the real time. In this paper, we present ReVACNN, a real-time visual analytic system for CNN, where (1) the overall training process (e.g., the amount of activations and that of changes each filter/layer has at a particular iteration/epoch) is visualized in network view and (2) the 2D embedding of trained filters within layers is visualized to show the relationships between filters as well as layers. In particular, ReVACNN allows users to perform several interactions in real time: (1) skipping the gradient descent update on the sub-part of a CNN model to reduce the subsequent training time and (2) steering filters interactively in the 2D embedding view to avoid bad local minima. At the end, we present several use cases that demonstrate the benefits users can gain from ReVACNN.

2. ICDM Best Student
   Paper Award

   L-ensnmf: Boosted local topic discovery via ensemble of nonnegative matrix factorization

   Suh, Sangho, Choo, Jaegul, Lee, Joonseok, and Reddy, Chandan K

   In 2016 IEEE 16th International Conference on Data Mining (ICDM) 2016

   Abs PDF Code Slides Presentation

   Nonnegative matrix factorization (NMF) has been widely applied in many domains. In document analysis, it has been increasingly used in topic modeling applications, where a set of underlying topics are revealed by a low-rank factor matrix from NMF. However, it is often the case that the resulting topics give only general topic information in the data, which tends not to convey much information. To tackle this problem, we propose a novel ensemble model of nonnegative matrix factorization for discovering high-quality local topics. Our method leverages the idea of an ensemble model, which has been successful in supervised learning, into an unsupervised topic modeling context. That is, our model successively performs NMF given a residual matrix obtained from previous stages and generates a sequence of topic sets. Our algorithm for updating the input matrix has novelty in two aspects. The first lies in utilizing the residual matrix inspired by a state-of-the-art gradient boosting model, and the second stems from applying a sophisticated local weighting scheme on the given matrix to enhance the locality of topics, which in turn delivers high-quality, focused topics of interest to users. We evaluate our proposed method by comparing it against other topic modeling methods, such as a few variants of NMF and latent Dirichlet allocation, in terms of various evaluation measures representing topic coherence, diversity, coverage, computing time, and so on. We also present qualitative evaluation on the topics discovered by our method using several real-world data sets.

3. CHI

   Curiosity Notebook: A Platform for Learning by Teaching Conversational Agents

   Law, Edith, Baghaei Ravari, Parastoo, Chhibber, Nalin, Kulic, Dana, Lin, Stephanie, Pantasdo, Kevin D, Ceha, Jessy, Suh, Sangho, and Dillen, Nicole

   In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems 2020

   Abs PDF

   Learning by teaching is an established pedagogical technique; however, the exact process through which learning happens remains difficult to assess, in part due to the variability in the tutor-tutee pairing and interaction. Prior research proposed the use of teachable agents acting as students, in order to facilitate more controlled studies of the learning by teaching phenomenon. In this work, we introduce a learning by teaching platform, Curiosity Notebook, which allows students to work individually or in groups to teach a conversational agent a classification task in a variety of subject topics. We conducted a 4-week exploratory study with 12 fourth and fifth grade elementary school children, who taught a conversational robot how to classify animals, rocks/minerals and paintings. This paper outlines the architecture of our system, describes the lessons learned from the study, and contributes design considerations on how to design conversational agents and applications for learning by teaching scenarios.

4. IDC

   How do we design for concreteness fading? survey, general framework, and design dimensions

   Suh, Sangho, Lee, Martinet, and Law, Edith

   In Proceedings of the Interaction Design and Children Conference (IDC) 2020

   Abs PDF Code Slides

   Over the years, concreteness fading has been used to design learning materials and educational tools for children. Unfortunately, it remains an underspecified technique without a clear guideline on how to design it, resulting in varying forms of concreteness fading and conflicting results due to the design inconsistencies. To our knowledge, no research has analyzed the existing designs of concreteness fading implemented across different settings, formulated a generic framework, or explained the design dimensions of the technique. This poses several problems for future research, such as lack of a shared vocabulary for reference and comparison, as well as barriers to researchers interested in learning and using this technique. Thus, to inform and support future research, we conducted a systematic literature review and contribute: (1) an overview of the technique, (2) a discussion of various design dimensions and challenges, and (3) a synthesis of key findings about each dimension. We open source our dataset to invite other researchers to contribute to the corpus, supporting future research and discussion on concreteness fading

5. VL/HCC

   Coding Strip: A Pedagogical Tool for Teaching and Learning Programming Concepts through Comics

   Suh, Sangho, Lee, Martinet, Xia, Gracie, and Law, Edith

   In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) 2020

   Abs PDF Supp Article Slides Website

   The abstract nature of programming makes learning to code a daunting undertaking for many novice learners. In this work, we advocate the use of comics—a medium capable of presenting abstract ideas in a concrete, familiar way—for introducing programming concepts. Particularly, we propose a design process and related tools to help students and teachers create coding strips, a form of comic strips that are associated with a piece of code. We conducted two design workshops with students and high school computer science teachers to evaluate our design process and tools. We find that our design process and tools are effective at supporting the design of coding strips and that both students and teachers are excited about using coding strip as a tool for learning and teaching programming concepts.