Pillar 3 · Trustworthy self-improvement
Alignment
Turn feedback into reliable behavior. We design reward models, online learning procedures, and decoding algorithms that make alignment more robust, equitable, and efficient.
Research question
How can models improve from diverse preferences and their own discovered failures?
We design reward models, online learning procedures, and decoding algorithms that make alignment more robust, equitable, and efficient.
- Reward-model self-correction
- Online alignment
- Preference diversity
- Reward-guided generation
Representative projects
Turn feedback into reliable behavior.
Each project connects its central idea with papers, code, datasets, demonstrations, and public explanations.
GenARM
Autoregressive rewards steer generation directly at test time, providing fine-grained control without retraining the generator.
MaxMin-RLHF
A max-min objective aligns language models without letting majority preferences erase systematically under-served groups.
SAIL
A self-improving online alignment loop learns efficiently from model-generated responses and continually updated feedback.
Selected publications
The papers behind the projects.
Browse the full publication database for the broader body of work in trustworthy ai.
Teach a Reward Model to Correct Itself: Reward Guided Adversarial Failure Discovery for Robust Reward Modeling
Main Conference, The 64th Annual Meeting of the Association for Computational Linguistics (ACL), Oral, 2026
BibTeX
@inproceedings{pathmanathan2026teach8fee,
title = {Teach a Reward Model to Correct Itself: Reward Guided Adversarial Failure Discovery for Robust Reward Modeling},
author = {Pankayaraj Pathmanathan and Furong Huang},
booktitle = {Main Conference, The 64th Annual Meeting of the Association for Computational Linguistics (ACL), Oral, 2026},
year = {2026},
eprint = {2507.06419},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2507.06419},
}GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-Time Alignment
The Thirteenth International Conference on Learning Representations (ICLR), 2025
BibTeX
@inproceedings{xu2025genarmceb1,
title = {GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-Time Alignment},
author = {Yuancheng Xu and Udari Madhushani Sehwag and Alec Koppel and Sicheng Zhu and Bang An and Furong Huang and Sumitra Ganesh},
booktitle = {The Thirteenth International Conference on Learning Representations (ICLR), 2025},
year = {2025},
eprint = {2410.08193},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2410.08193},
}MaxMin-RLHF: Alignment with Diverse Human Preferences
Proceedings of the 41st International Conference on Machine Learning (ICML), 2024
BibTeX
@inproceedings{chakraborty2024maxmin06de,
title = {MaxMin-RLHF: Alignment with Diverse Human Preferences},
author = {Souradip Chakraborty and Jiahao Qiu and Hui Yuan and Alec Koppel and Furong Huang and Dinesh Manocha and Amrit Bedi and Mengdi Wang},
booktitle = {Proceedings of the 41st International Conference on Machine Learning (ICML), 2024},
year = {2024},
eprint = {2402.08925},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2402.08925},
}SAIL: Self-improving Efficient Online Alignment of Large Language Models
ICML 2024 Workshop on Theoretical Foundations of Foundation Models, ICML 2024
BibTeX
@inproceedings{ding2024sail1436,
title = {SAIL: Self-improving Efficient Online Alignment of Large Language Models},
author = {Mucong Ding and Souradip Chakraborty and Vibhu Agrawal and Zora Che and Alec Koppel and Mengdi Wang and Amrit Bedi and Furong Huang},
booktitle = {ICML 2024 Workshop on Theoretical Foundations of Foundation Models, ICML 2024},
year = {2024},
url = {https://x.com/furongh/status/1806151592783093776?s=20},
}