About Me
Luo
Computer Science PhD ∈ Northwestern University
I am a third-year Ph.D. candidate in the Computer Science Department at Northwestern University, advised by Professor Yan Chen and Han Liu in the MAGICS Lab.
My primary research interest lies in the intersection of Machine Learning, Health Care, and Financial Data.
Topics I am currently working on (Machine/Deep Learning):
- Large Foundation Models:
- Comprehensible Foundation Models, integrating modern Hopfield networks. [ICML ‘24]
- Responsible Foundation Models, encompassing jailbreaks, adversarial attacks, and risks associated with scientific foundation models. [arXiv] [arXiv]
- Accessible Foundation Models, incorporating PEFT, quantization, and outlier removal for efficient training and enhanced quantization robustness. [ICML ‘24] [ES-FoMo@ICML ‘24]
- Actionable Foundation Models, featuring Chain of Thought and Chain of Action methodologies. [ICLR ‘25] [arXiv]
- Applications of Large Foundation Models, including Genomic Foundation Models and Human Mobility Foundation Models. [arXiv] [HuMob@SIGSPATIAL ‘24]
- Memory retrieval, memory-enhanced models, and memory editing techniques. [arXiv]
Topics I worked on in the past (before my Ph.D. studies):
- Multimodality, including Video Question Answering and Visual Question Answering. [arXiv]
- Human-Robot Interaction. [ROMAN ‘23]
- Reinforcement Learning. [ICMLA ‘22]
- Natural Language Understanding.
- Question Answering. [Advances in Information and Communication]
Besides schoolwork and research, I have developed interests in many activities over time, including photography, hiking, and traveling.
News
2025.1: Our paper Chain-of-Action: Faithful and Multimodal Question Answering through Large Language Models has been accepted by ICLR 2025.
2025.1: I am serving as a TA for CS368.
2025.1: I received $1,000 in research funding from the OpenAI Research Access Program.
2024.9: I am serving as a Research Assistant in Dashun's Lab.
2024.10: Our workshop paper, ST-MoE-BERT: A Spatial-Temporal Mixture-of-Experts Framework for Long-Term Cross-City Mobility Prediction, has been accepted by the 2nd ACM SIGSPATIAL International Workshop on the Human Mobility Prediction Challenge, where we achieved the top 1 DTW score.
2024.7: Our workshop papers Outlier-Efficient Hopfield Layers for Large Transformer-Based Models and Fast Adaptation and Robust Quantization of Multi-Modal Foundation Models from Associative Memory: A Case Study in SpeechLM have been accepted by ICML ES-FoMo 2024.
2024.5: Our paper OutEffHop: A Principled Outlier-Efficient Attention Layer from Dense Associative Memory Models has been accepted by ICML 2024.
2024.4: I received $5,000 in research funding from the OpenAI Research Access Program.
2023.6: I served as a Program Committee (PC) member for the industry track of EMNLP 2023.
2023.4: Our conference paper Open-Ended Multi-Modal Relational Reasoning for Video Question Answering was accepted by RO-MAN 2023.
2022.10: Our conference paper IGN: Implicit Generative Networks was accepted by ICMLA 2022.
2022.9: I started my Ph.D. studies at Northwestern University!
Education
Northwestern University - MAGICS Lab
Computer Science PhD @ Northwestern University
Sept. 2022Northwestern University
GPA 3.92/4.0, MSIT @ Northwestern University (Transfer to CS PhD program)
Sept. 2021Georgia Institute of Technology
GPA 3.7/4.0, VQA-related research @ Machine Learning Lab
Augest. 2020 - May 2021MS in Computer Science
Georgia Institute of Technology
Graduate with High Hornor, GPA 3.53/4.0, NLP-related research @ Machine Learning Lab, Dean List
Jan. 2018 - May 2020BS in Computer Science
Michigan State University
Transfer to Georgia Institute of Technology with GPA 3.65/4.0
Sept. 2016 - Dec 2017BS in Computer Engineering
My Research
Northwestern University
Integrated Modern Hopfield Networks into a knowledge-augmented language model to address limitations in self-attention architectures and foundation models, such as no-op outliers during training. Explored the interpretability of jailbreak strategies in safety-aligned LLMs by employing in-context learning and outlier analysis to enhance alignment and safety in superhuman AI systems. Applied LLMs to multivariate time-series challenges, including in-context time series forecasting, Limit Order Book dynamics prediction, and cross-sectional regression time-series analysis. Developed efficient hyperparameter optimization (HPO) methods for time-series prediction using the Tree-structured Parzen Estimator (TPE) and Genetic Algorithm.
Dec. 2021 - PresentGraduate Research Assistant
Northwestern University
Worked on distributed deep reinforcement learning by implementing the IQN model with the Wasserstein GAN algorithm. Designed a policy evaluation experiment using the Atari Game Database with fixed policy input and compared results with a baseline fixed policy. Developed policy optimization for the IQN model using the Wasserstein GAN algorithm.
Sept. 2021 - Nov. 2021Graduate Research Assistant
Northwestern University
Worked on model-free reinforcement learning to achieve state-of-the-art performance in a physical simulator. Implemented PPO and SAC algorithms to compute and visualize reinforcement learning rewards. Contributed to a paper in progress, RobLAX-A Differentiable Robotics Framework for Physics-Augmented Reinforcement Learning.
May. 2021 - Sept. 2021Graduate Research Assistant
Georgia Institute of Technology
Developed an NLP labeling pipeline for the brat annotation tool. Researched polysemy problems and applied the BERT model for target word and phrase labeling. Studied the Operational Transformation Algorithm to enable collaborative real-time editing in the cloud.
Sept. 2020 - May 2021Graduate Research Assistant
Georgia Institute of Technology
Worked on Continuous Neuro-Symbolic Visual Question Answering (VQA), particularly researching soft logic functions. Designed and implemented a reasoning model combining BERT, bidirectional LSTM, and Stack-NMN, and evaluated its performance against other VQA models. Researched Video Question Answering and developed a model using R(2+1)D to detect object actions and movements. Contributed to a paper in progress, “Differentiable End-to-End Program Executor for Sample and Computationally Efficient VQA.”
Aug. 2020 - Oct 2020Graduate Research Assistant
Georgia Institute of Technology
Researched the Visual Question Answering (VQA) problem using the NS-VQA algorithm to classify object relationships in images and answer GQA questions. Investigated reasoning mechanisms and improved performance using bidirectional LSTMs. Explored the functionality and latest advancements in VQA technology.
Aug. 2019 - May 2020Undergraduate Research Assistant
Georgia Institute of Technology
Developed a computational pipeline for analyzing adsorption energy in chemical reactions using Python and the ASE API. Implemented a machine learning model to analyze chemical molecular structures.
Jan. 2019 - May 2020DFT Model Analyze Adsorption Energies Team
Georgia Institute of Technology
Developed a data pipeline that collects and analyzes stadium network data to provide precise insights for stadium staff. Built a website tool for setting up a remote database for the LoPT Database Team.
Jan. 2018 - Dec 2018VIP Georgia Tech – LoPt Database Team
Publication
A Spatial-Temporal Mixture-of-Experts Framework for Long-Term Cross-City Mobility Prediction
2nd ACM SIGSPATIAL International Workshop on the Human Mobility Prediction Challenge
Oct. 2024Haoyu He, Haozheng Luo, Qi R. Wang
Decoupled Alignment for Robust Plug-and-Play Adaptation
Review by USENIX Security'2025
May. 2024Haozheng Luo*, Jiahao Yu*, Wenxin Zhang*, Jialong Li, Jerry Yao-Chieh Hu, Xingyu Xin, Han Liu
Enhancing Jailbreak Attack Against Large Language Models through Silent Tokens
Review by USENIX Security'2025
May. 2024Jiahao Yu*, Haozheng Luo*, Jerry Yao-Chieh Hu, Wenbo Guo, Han Liu, Xinyu Xing
Conv-CoA- Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Action
Review by ICML'2025
May. 2024Zhenyu Pan*, Haozheng Luo*, Manling Li, Han Liu
Fast Adaptation and Robust Quantization of Multi-Modal Foundation Models from Associative Memory - A Case Study in SpeechLM Authors
Accepted by Efficient Systems for Foundation Models II@ ICML2024
June. 2024Shang Wu*, Yen-Ju Lu*, Haozheng Luo*, Jerry Yao-Chieh Hu, Jiayi Wang, Najim Dehak, Jesus Villalba, Han Liu
OutEffHop- A Principled Outlier-Efficient Attention Layer from Dense Associative Memory Models
Accepted by Efficient Systems for Foundation Models II@ ICML2024
June. 2024Haozheng Luo*, Jerry Yao-Chieh Hu*, Pei-Hsuan Chang*, Hong-Yu Chen, Weijian Li, Wei-Po Wang, Han Liu
Outlier Efficient Modern Hopfield Model for Large Transformer-Based Models
Accepted by ICML'2024
Jan. 2024Jerry Yao-Chieh Hu*, Pei-Hsuan Chang*, Haozheng Luo*, Hong-Yu Chen, Weijian Li, Wei-Po Wang, Han Liu
Chain-of-action- Faithful and Multimodal Question Answering through Large Language Models
Accepted by ICLR'2025
Jan. 2024Zhenyu Pan, Haozheng Luo, Manling Li, Han Liu
SWGA - A Distributed Hyperparameter Search Method for Time Series Prediction Models
Review by ICML'2025
Aug. 2023Weijian Li*, Haozheng Luo*, Chenwei Xu, Han Liu
SMUTF - Schema Matching Using Generative Tags and Hybrid Features
Review by Information Systems
Aug. 2023Yu Zhang*, Di Mei*, Haozheng Luo*, Chenwei Xu, Richard Tzong-Han Tsai
IGN - Implicit Generative Networks
ICMLA'2022 - 2022 21st IEEE International Conference on Machine Learning and Applications
March. 2022Haozheng Luo, Tianyi Wu, Feiyu Han, Zhijun Yan
RobLAX-A Differentiable Robotics Framework for Physics Augmented Reinforcement Learning
Sept. 2021
Guo Ye, Qinjie Lin, Tim Tsz-Kit Lau, Wanxin Jin, Cheng Zhou, Haozheng Luo, Zhuoran Yang, Zhaoran Wang, Han Liu
SciAnnotate- A Tool for Integrating Weak Labeling Sources for Sequence Labeling
Under Reviewed by ACL-SDProc 2024
Jan. 2021Mengyang Liu *, Haozheng Luo*,Leonard Thong*, Yinhao Li, Chao Zhang, Le Song
Open-ended multi-modal relational reason for video question answering
RO-MAN'2023 - 2023 32nd IEEE International Conference on Robot and Human Interactive Communication
Dec. 2020Haozheng Luo, Ruiyang Qin
HOW TO DESIGN SAMPLE AND COMPUTATIONALLY EFFICIENT VQA MODELS
Oct 2020
Karan Samel*, Zelin Zhao*, Kuan Wang*, Haozheng Luo, Binghong Chen, Song Le
Question ClassiNication with Deep Contextualized Transformer
FICC'2021 - Proceedings of the 2021 Future of Information and Communication Conference
Oct 2019Haozheng Luo, Ningwei Liu, Charles Feng
My Career
Georgia Institute of Technology
I grade students’ projects and exams and provide feedback to facilitate improvement; Have a solid understanding of algorithms in KMM, Gradient Model and Neural-Networks; Provide students with the means to succeed by holding ofkice hours and responding to questions online
Augest. 2020 - May 2021Graduate Teaching Assistant
Splunk
I automated the process of sending security alerts through Slack and Email by developing an app that would be deployed on all windows machines at Splunk; Developed a pipeline that detects sensitive kiles used by customer support and implements the required retention policies.
June. 2020 - Augest 2020Backend Software Engineering Intern
Splunk
I designed and developed a customer support tool that automatically suggests answers to customer questions, by using Python/React.js/ Flask/Electron/TensorFlow and NLTK; Created a pipeline that updates the customer support knowledge base with customer support feedback from the previous tool, it potentially saved 35% of the original time.
May 2019 - Augest 2019Backend Software Engineering Intern
Michigan Republican Party
I delivered a pipeline capable of predicting Twitter usernames from local residents' real names; Implemented a visualization tool that describes the analysis with several kinds of statistic chart
Augest 2018 - September 2018Data Team Developer
R2.ai Inc.
I contributed work to the construction of a website tool which provides customers cloud-computing server; Designed a pipeline that tests machine learning functions which are inside web servers and compares results with common machine learning algorithms using Python and TensorFlow.
June 2018 - Augest 2018Machine Learning Summer Intern
DBAPP Security
I developed a pipeline capable of scrawling data from the Common Vulnerability and Exposures website using Python and Selenium; Delivered the scrawled data into PostgreSQL
May 2017 - July 2017Programmer Analyst Intern
Talks
- Job Talk at Ryan Wang Lab
- Funding Presentation with Ant Group
- Oral Presentation at IEEE RO-MAN 2023
- Group Reading Presentation on the Autoformer paper
- Group Reading Presentation on the Gradient Boosting Algorithm
- Oral Presentation at FICC 2021
Service
- Teaching:
- [Winter 2025] TA, Programming Massively Parallel Processors with CUDA (COMP_SCI 368), Northwestern University.
- [Spring 2024] TA, Fundamentals of Computer Programming 1.5 (COMP_SCI 150), Northwestern University.
- [Winter 2024] TA, Introduction to Artificial Intelligence (COMP_SCI 348), Northwestern University.
- [Fall 2023] TA, Generative Methods (COMP_SCI 327), Northwestern University.
- [Spring 2021] TA, Modeling and Simulation (CSE 6730), Georgia Institute of Technology.
- [Fall 2020] TA, Computational Data Analysis (CSE 6740), Georgia Institute of Technology.
- Reviewer: WWW 2020, NAACL 2024/2025, ACL 2020/2023/2024/2025, EMNLP 2023/2024, MLIS 2023, AIM 2024, NeurIPS 2024, ICLR 2025, ICML 2025, ACL-ARR (DOA).
- Program Committee: EMNLP Industry Track 2023.
Collaboration and Mentoring
- Chenghao Qiu, Tianjin University, BSCS '25 → CS Ph.D. study at TAMU (Fall '25)
- Zhenyu Pan, MSECE '24 at the University of Rochester → CS Ph.D. study at NU (Fall '24)
- Mingyu Jin, MSCS '24 at NU → CS Ph.D. study at Rutgers University (Fall '24)
- Hong-Yu Chen, NTU, Physics MS '24 → CS Ph.D. study at NU (Fall '24)
- Outlier-Efficient Hopfield Layers for Large Transformer-Based Models [ICML '24]
- Yihong Yu, University of California, Irvine, BSCS '25
- Jingyu Elaine Wu, Hong Kong University of Science and Technology, BS '23 → Northwestern University, MSCS '25
- Shaopeng Frank Gu, Northwestern University, CS + Statistics, BS '25
My Skills
My Projects
Reward & Patent
Patents
Movable infusion stand (201120560457X)
Robin Luo
December 2013 in Hangzhou
Other
High Hornor of Graduateions
May 2020 in Atlanta
Miscellaneous
30-30 Project
Hopefully, before I turn 30 years old, I can:
Visit 30 countries/regions (7/30)
🇨🇳 China, 🇭🇰 Hong Kong, 🇯🇵 Japan, 🇺🇸 United States, 🇨🇦 Canada, 🇻🇳 Vietnam, 🇸🇬 Singapore.
Visit 30 provinces in China (13/30)
Beijing, Zhejiang, Fujian, Hainan, Guangdong, Jiangsu, Heilongjiang, Tianjin, Hong Kong, Shandong, Anhui, Shanxi, Shanghai.Visit 30 states in the US (30/30)
❄️ AK, 🌉 CA, 🏂 CO, 📃 CT, 🐼 DC, 🍊 FL, 🍑 GA, 🌋 HI, 💨 IL, 🏁 IN, 🚜 IA, 🏇 KY, 🔮 MA, 🐢 MD, 🚘 MI, ♈ MO, 🌟 MN, ✈ NC, 🐍 NH, 💡 NJ, 🏜️ NV, 🗽 NY, 🏈 OH, 🌹 OR, 🌊 RI, 🌴 SC, 🎸 TN, 🗼 TX, 🍺 WI, ☔ WA.Visit 30 National Parks in the US (51/30)
Indiana Dunes NP, Yosemite NP, Mount Rainier NP, North Cascades NP, Olympic NP, Great Smoky Mountains NP, Gateway Arch NP, Rocky Mountain NP, Mammoth Cave NP, Congaree NP, Boston NHP, Manhattan Project NHP, Minute Man NHP, Lewis & Clark NHP, San Francisco Maritime NHP, Golden Gate NRA, Ross Lake NRA, Lake Chelan NRA, Big South Fork NRRA, Herbert Hoover NHS, Lincoln Home NHS, Ulysses S. Grant NHS, Boston African American NHS, Fort Point NHS, Saugus Iron Works NHS, Salem Maritime NHS, Statue of Liberty NM, Fort Pulaski NM, Muir Woods NM, Fort McHenry NM, Florissant Fossil Beds NM, Lewis & Clark NHT, Star-Spangled Banner NHT, Juan Bautista de Anza NHT, Sleeping Bear Dunes NL, Ice Age NST, Point Reyes NS, Obed WSR, Korean War Veterans Memorial, Lincoln Memorial, Pullman Memorial, Pearl Harbor Memorial, Vietnam Veterans Memorial, White House, Alcatraz Island, Presidio of San Francisco, Washington Monument, World War II Memorial, Wing Luke Museum Affiliated Area, Blue Ridge Parkway, Baltimore-Washington Parkway.Visit 30 Airports (40/30)
ZSYT, KORD, KJFK, ZSHC, ZSPD, KATL, KDTW, ZYHB, ZBTJ, ZBAA, ZJSY, ZLXY, KLAX, KSFO, KLEX, KEWR, KLGA, VHHH, KMIA, KSJC, KMCO, KMSP, KLAN, KDFW, KDEN, ZSAM, KIAH, KAUS, KBWI, KDCA, KSEA, KCID, KSLC, KLAS, KHNL, KIAD, KBRL, ZGGG, ZGSZ, VVTS.Fly with 30 Airlines (18/30)
HU, CA, DL, AA, UA, CZ, MU, JD, CX, MF, KA, F9, NK, WN, AS, AC, 9K, NK.Accumulate 300,000 kilometers of air travel (238,593/300,000)
Link
© 2025 Robin(Haozheng) Luo