Seongmin Lee

CS PhD student at Georgia Tech

My research bridges machine learning and human-computer interaction to advance Responsible AI. I design and develop novel visual explanations for complex machine learning models, and scalable algorithms that promote accountable use of AI.
I have collaborated with researchers and developers at JP Morgan Chase & Co., Cisco Systems, IBM Research, and AVAST Software.

Education

Aug. 2021 — Present
Ph.D. in Computational Science & Engineering
Georgia Institute of Technology, Atlanta, GA
Advisor: Duen Horng (Polo) Chau
Mar 2016 — Feb. 2021
B.S. in Electrical and Computer Engineering
Seoul National University, Seoul, South Korea
Graduated with Summa Cum Laude, Overall GPA: 3.91/4.00, Major GPA: 3.94/4.00
Mar 2013 — Feb. 2016
Math and Science Specialized High School for Gifted
Korea Science Academy of KAIST, Busan, South Korea
One year early entrance, Graduated with distinction in GPA

Research Experience

May 2024 — Aug. 2024
JP Morgan Chase & Co., New York, NY
Summer Associate, Research Teechnologies Program
Mentor: Hsiang Hsu, Chun-Fu (Richard) Chen
May 2022 — Jul. 2022
Cisco Systems, Inc., San Jose, CA
Ph.D. Intern
Mentor: Ali Payani
Led model interpretation and visualization development for an open-source project for Responsible AI
Aug. 2021 — Present
Georgia Institute of Technology, Atlanta, GA
Graduate Research Assistant, School of Computational Science and Engineering
Advisor: Duen Horng (Polo) Chau
Jan. 2020 — Feb. 2021
Seoul National University, Seoul, South Korea
Undergraduate Research Intern, Data Mining Lab
Advisor: U Kang
Sep. 2019 — Apr. 2020
Seoul National University, Seoul, South Korea
Undergraduate Research Intern, Communication and Machine Learning Laboratory
Advisor: Jungwoo Lee
Jul. 2018 — Aug. 2018
The University of Tokyo, Tokyo, Japan
Undergraduate Research Intern, Biomedical Precision Laboratory
Advisor: Keiichi Nakagawa, Ayumu Ishijima
Constructed a microscopic motion-picture laser camera with sub-nano spatial/temporal resolution, Investigated interaction between human cells and acoustic waves in collaboration with students majoring in biology and physics

Honors and Awards

2024
NeurIPS SoLaR Workshop Travel Fund Award
Selected as a recipient of the travel fund award of $1,800 for the NeurIPS SoLaR Workshop.
2024
Best Poster Award
Transformer Explainer receives Best Poster Award at VIS 2024.
2023
Next-generation Science and Technology Leader NET
Research grant of $1,500 funded by KOFST. Conducted interdisciplinary research on 'Wearable Devices and AI to Enhance Sustainability and Reliability of Korean Healthcare Data' with 5 other students.
2021
CS7001 Best Final Project Award Honorable Mention
Awarded for the project 'VisCUIT: Visual Auditor for Bias in CNN Image Classifier' in CS7001, a course taken by all first-year CS Ph.D. students at Georgia Tech. 2nd place among 60+ presentations.
2016 - 2019
The Presidential Science Scholarship
Full tuition and stipend of $5,000 per year. Around 150 students are selected nationally.
2018 - 2019
SNU Engineering Honor Society (STEM)
Used to be vice president, currently honorary member of STEM.
2017, 2018
Seoul National University ECE Voluntary Service Award
Selected based on the evaluation of the students, staffs, and faculty members of department of ECE.
2014 - 2015
Hansung Sonjaehan Scholarship
Stipend of $5,000 per year. Around 150 high school students studying math and science are selected nationally.

Publications

Effective Guidance for Model Attention with Simple Yes-no Annotations
Seongmin Lee, Ali Payani, Duen Horng (Polo) Chau
Oral Paper, IEEE International Conference on Big Data (BigData). Washington DC, USA, 2024.
Project PDF BibTeX
LLM Hallucination Reasoning with Zero-shot Knowledge Test
Seongmin Lee, Hsiang Hsu, Chun-Fu (Richard) Chen
38th Conference on Neural Information Processing Systems (NeurIPS 2024) Socially Responsible Language Modelling Research Workshop (SoLaR). Vancouver, Canada, 2024.
Project
Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusion
Seongmin Lee, Benjamin Hoover, Hendrik Strobelt, Zijie J. Wang, ShengYun Peng, Austin P. Wright, Kevin Li, Haekyu Park, Haoyang Yang, Duen Horng (Polo) Chau
Short Paper, IEEE Visualization Conference (VIS). Tampa, USA, 2024.
Project Demo PDF Recording Poster Code (221) BibTeX
Transformer Explainer: Interactive Learning of Text-Generative Models
Aeree Cho, Grace C Kim, Alexander Karpekov, Alec Helbling, Zijie J. Wang, Seongmin Lee, Benjamin Hoover, Duen Horng (Polo) Chau
Poster, IEEE Visualization Conference (VIS). 2024.
Project Demo PDF Recording Code (3.1k) BibTeX Best Poster
LLM Attributor: Interactive Visual Attribution for LLM Generation
Seongmin Lee, Zijie J. Wang, Aishwarya Chakravarthy, Alec Helbling, ShengYun Peng, Mansi Phute, Duen Horng (Polo) Chau, Minsuk Kahng
Poster, IEEE Visualization Conference (VIS). 2024.
Project PDF Recording Code BibTeX
SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks
Zijie J. Wang, David Munechika, Seongmin Lee, Duen Horng (Polo) Chau
Extended Abstracts on ACM Human Factors in Computing Systems (CHI). 2024.
Project Demo PDF Code (62) BibTeX
UniTable: Towards a Unified Framework for Table Structure Recognition via Self-Supervised Pretraining
ShengYun Peng, Seongmin Lee, Xiaojing Wang, Rajarajeswari Balasubramaniyan, Duen Horng (Polo) Chau
ArXiv. 2024.
Project PDF Code (354) BibTeX
ClickDiffusion: Harnessing LLMs for Interactive Precise Image Editing
Alec Helbling, Seongmin Lee, Duen Horng (Polo) Chau
CVPR 2024 Workshop on AI for Content Creation Workshop (CVPR). 2024.
Project PDF Code (65) BibTeX
Mobile Fitting Room: On-device Virtual Try-on via Diffusion Models
Justin Blalock, David Munechika, Harsha Karanth, Alec Helbling, Pratham Mehta, Seongmin Lee, Duen Horng (Polo) Chau
CVPR 24 Workshop on Virtual Try-On (CVPR). 2024.
Project PDF BibTeX
High-Performance Transformers for Table Structure Recognition Need Early Convolutions
ShengYun Peng, Seongmin Lee, Xiaojing Wang, Rajarajeswari Balasubramaniyan, Duen Horng (Polo) Chau
Oral Paper, Neural Information Processing Systems (NeurIPS) TRL Workshop (TRL). New Orleans, LA, USA, 2023.
Project PDF Code BibTeX
VisGrader: Automatic Grading of D3 Visualizations
Matthew Hull, Vivian Pednekar, Hannah Murray, Nimisha Roy, Emmanuel Tung, Susanta Routray, Connor Guerin, Justin Chen, Zijie J. Wang, Seongmin Lee, Mahdi Roozbahani, Duen Horng (Polo) Chau
Full Paper, IEEE Visualization and Visual Analytics (VIS). Melbourne, Australia, 2023.
Project PDF Code BibTeX
Concept Evolution in Deep Learning Training: A Unified Interpretation Framework and Discoveries
Haekyu Park, Seongmin Lee, Benjamin Hoover, Austin Wright, Omar Shaikh, Rahul Duggal, Nilaksh Das, Judy Hoffman, Duen Horng (Polo) Chau
Full Paper, ACM International Conference on Information and Knowledge Management (CIKM). Birmingham, United Kingdom, 2023.
Project PDF Code BibTeX DOI
Explaining Website Reliability by Visualizing Hyperlink Connectivity
Seongmin Lee, Sadia Afroz, Haekyu Park, Zijie J. Wang, Omar Shaikh, Vibhor Sehgal, Ankit Peshin, Duen Horng (Polo) Chau
Short Paper, IEEE Visualization and Visual Analytics (VIS). Oklahoma City, OK, USA, 2022.
Project Demo PDF BibTeX
NOVA: A Practical Method for Creating Notebook-Ready Visual Analytics
Zijie J. Wang, David Munechika, Seongmin Lee, Duen Horng (Polo) Chau
Poster Paper, IEEE Visualization and Visual Analytics (VIS). Oklahoma City, OK, USA, 2022.
Project Demo PDF Poster Code BibTeX
VisCUIT: Visual Auditor for Bias in CNN Image Classifier
Seongmin Lee, Zijie J. Wang, Judy Hoffman, Duen Horng (Polo) Chau
Demo Paper, Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, USA, 2022.
Project Demo PDF Recording Code BibTeX
MisVis: Explaining Web Misinformation Connections via Visual Summary
Seongmin Lee, Sadia Afroz, Haekyu Park, Zijie J. Wang, Omar Shaikh, Vibhor Sehgal, Ankit Peshin, Duen Horng (Polo) Chau
Extended Abstracts on ACM Human Factors in Computing Systems (CHI). New Orleans, LA, USA, 2022.
Project Demo PDF BibTeX DOI
Multi-EPL: Accurate Multi-source Domain Adaptation
Seongmin Lee, Hyunsik Jeon, U Kang
Public Library of Science (PLOS ONE). 2021.
Project PDF Code BibTeX
Unsupervised multi-source domain adaptation with no observable source data
Hyunsik Jeon, Seongmin Lee, U Kang
Public Library of Science (PLOS ONE). 2021.
Project PDF Code BibTeX
Auber: Automated BERT Regularization
Hyun Dong Lee*, Seongmin Lee*, U Kang
Public Library of Science (PLOS ONE). 2021.
Project PDF Code BibTeX *Authors contributed equally

Patent

Pruning Method for Attention Head in Transformer Neural Networks for Regularization and Apparatus Thereof
Seongmin Lee, Hyun Dong Lee, U Kang
10-2020-0162397, 2020.

Book

I Got to Wish to Go Engineering School
SNU Engineering Honor Society (STEM)
Megastudy Books, 2021.
Wrote Part 3 "Life as a Lab Intern"

Invited Talks

Expanding Your Career Canvas: Pitching Strategies for Diverse Audiences
Aug 2024
Fostering Innovation in Rising Experts (FIRE) at United States-Korea Conference (UKC)
Diffusion Explainer: Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusion
Nov 2023
Georgia Tech, School of CSE Recruiting Event
STEM Global Engineer Seminar
May 2023
Seoul National University, SNU Engineering Honor Society (STEM)
MisVis: Explaining Website Reliability by Visualizing Hyperlink Connectivity
Nov. 2022
Georgia Tech, School of CSE Recruiting Event
Mar. 2022
AVAST Software, Fake News Monthly Sync Up
Multi-Source Domain Adaptation with Pseudo-labels
Jun. 2020
Seoul National University, AIIS Retreat
STEM Vision Mentoring
Jul. 2019
Seoul National University, STEM Vision Mentoring
Organized and gave lectures on engineering for high school students. More than 700 students attended.

Teaching

Spring 2024
Graduate Teaching Assistant
Georgia Institute of Technology, Atlanta, GA
Data and Visual Analytics (CSE6242), Instructor: Duen Horng (Polo) Chau, Mahdi Roozbahani
Online course with 1201 graduate students enrolled.
Fall 2022
Graduate Teaching Assistant
Georgia Institute of Technology, Atlanta, GA
Data and Visual Analytics (CSE6242), Instructor: Duen Horng (Polo) Chau, Mahdi Roozbahani
Online course with 1146 graduate students enrolled.
Jan. 2021
Lab Session Lecturer
LG Chem, Seoul, South Korea
Deep Learning Course
Nov. 2020
Lab Session Lecturer
SK Hynix, Seoul, South Korea
Deep Learning Course
Fall 2020, Fall 2019
Undergraduate Student Tutor
Seoul National University, Seoul, South Korea
Introduction to Algorithm
Selected as tutor for two consecutive years based on interviews and academic performanace
Spring 2017
Undergraduate Student Tutor
Seoul National University, Seoul, South Korea
Introduction to Physics I
Selected as tutor based on academic performance

Mentoring

Fall 2023 — Present
Sri Ranganathan Palaniappan
B.S. in Computer Science, Georgia Institute of Technology
Fall 2024 — Present
Kunal Mohindra
B.S. in Computer Science, Georgia Institute of Technology
Fall 2023 — Fall 2024
Aishwarya Chakravarthy
B.S. in Computer Science, Georgia Institute of Technology
Fall 2023 — Fall 2024
Grace Kim
B.S. in Computer Science, Georgia Institute of Technology
Fall 2023 — Fall 2024
Alexander Karpekov
M.S. in Computer Science, Georgia Institute of Technology
Now: CS Ph.D. at Georgia Institute of Technology
Fall 2023 — Fall 2024
Aeree Cho
M.S. in Computer Science, Georgia Institute of Technology
Now: ML Ph.D. at Georgia Institute of Technology
Fall 2021 — Fall 2023
David Munechika
B.S. in Computer Science, Georgia Institute of Technology
Now: Machine Learning Engineer at Apple
Fall 2021 — Spr. 2023
Kevin Li
B.S. in Computer Science and Biomedical Engineering, Georgia Institute of Technology
Now: ML Ph.D. at Carnegie Mellon University
Fall 2021 — Spr. 2023
Alex Yang
M.S. in Computer Science, Georgia Institute of Technology
Now: CS Ph.D. at Georgia Institute of Technology
Fall 2021 — Spr. 2022
Omar Shaikh
B.S. in Computer Science, Georgia Institute of Technology
Sigma Xi Best Undergraduate Research Award, Georgia Institute of Technology
Now: CS Ph.D. at Stanford University

Grants and Funding

2024
Democratizing Visual AI: Enhancing Efficiency and Usability of Large Vision Models for Fostering Under-Resourced Access
NSF Medium Grant
Co-PIs: Judy Hoffman, Celine Lin, Duen Horng (Polo) Chau
Co-authored winning proposal. Contributed to experiment with on-device implementation of the CLIP model.
2023
Scaling Up Educators' Capacities: Automating Visualization Assessment via Generative AI
Google Award for Inclusion Research
Co-PIs: Duen Horng (Polo) Chau
Co-authored a proposal resulting in $60k resesarch funding from Google.
2022
Cisco Research Funding
Co-PIs: Duen Horng (Polo) Chau
Co-authored a proposal resulting in $200k resesarch funding on responsible AI from Cisco.

Service

Reviewer
ACM CHI conference on Human Factors in Computing Systems (CHI), 2025
IEEE Pacific Visualization Conference (PacificVis), 2025
7th Workshop on Visualization for AI Explainability (VISxAI), 2024
ACM Conference On Computer-Supported Cooperative Work and Social Computing (CSCW), 2023
Institutional
2021 – Present
Georgia Tech Korean Student Association
Have served as treasury, web administrator, and recruitment officer. Organize events for Korean graduate students at Georgia Tech.
2022 – 2024
Georgia Tech CSE Website Committee
Improve external and internal communication in collaboration with a communication officer and faculty members
2020
SNU Buddy
Selected as the best buddy out of over 60 buddy students based on the votes from international exchange students.
2017 – 2018
Seoul National University ECE Student Vice President
Elected from student vote.
2017 – 2021
Seoul National University ECE Student Reporter
Published a magazine about research, people, and events of Department of ECE every semester.
2018, 2019
Dream Camp Mentoring
Conducted mentoring program about "how to materialize one's dream" for high school students in countryside
2016, 2017
KOSAF Talent Sharing Volunteer Camp
Conducted self-planned mentoring for elementary school students in underprivileged areas in Korea
Member
2021 - Present
Association for Computing Machinery (ACM)
2021 - Present
Institute of Electrical and Electronics Engineers (IEEE)
2018 — 2019
SNU Engineering Honor Society (STEM)

References

Dr. Polo Chau, Associate Professor
School of Computational Science and Engineering
Georgia Institute of Technology