Posts by Collection

Incorporating world knowledge to document clustering via heterogeneous information networks

Chenguang Wang, Yangqiu Song, Ahmed El-Kishky, Dan Roth, Ming Zhang, and Jiawei Han.

In Proc. 2015 ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD 2015).

We provide three ways to specify the world knowledge to domains by resolving the ambiguity of the entities and their types, and represent the data with world knowledge as a heterogeneous information network.

Text classification with heterogeneous information network kernels

Chenguang Wang, Yangqiu Song, Haoran Li, Ming Zhang and Jiawei Han.

In Proc. 2016 AAAI Conf. on Artificial Intelligence (AAAI 2016).

This paper presents a novel text as network classification framework, which introduces a structured and typed heterogeneous information networks (HINs) representation of texts, and a meta-path based approach to link texts.

Crowd-in-the-loop: A hybrid approach for annotating semantic roles

Chenguang Wang, Alan Akbik, Laura Chiticariu, Yunyao Li, Fei Xia, and Anbang Xu.

In Proc. 2017 Conf. on Empirical Methods on Natural Language Processing (EMNLP 2017).

Our experimental evaluation shows that the proposed approach reduces the workload for experts by over two-thirds, and thus significantly reduces the cost of producing SRL annotation at little loss in quality.

Language models with Transformers

Chenguang Wang, et al.

In arXiv preprint arXiv:1904.09408 (arXiv 2019).

Gets more than 4.4k blog views and more than 320 Likes and Retweets on Twitter. Experimental results on the PTB, WikiText-2, and WikiText-103 show that proposed method achieves perplexities between 20.42 and 34.11 on all problems, i.e. on average an improvement of 12.0 perplexity units compared to state-of-the-art LSTMs.

Language Models are Open Knowledge Graphs

Chenguang Wang, Xiao Liu, and Dawn Song.

In arXiv preprint arXiv:2010.11967 (arXiv 2020).

What's the relationship between deep language models (e.g., BERT, GPT-2, GPT-3) and knowledge graphs? Can we use the pre-trained deep language models to construct knowledge graphs? We find that we can construct knowledge graphs from the pre-trained language models. The generated knowledge graphs not only cover the knowledge already in existing knowledge graphs, such as Wikidata, but also feature open factual knowledge that is new.

ENGtube: An integrated subtitle environment for ESL

Chi-Ho Li, Shujie Liu, Chenguang Wang, and Ming Zhou.

In MT Summit XIII: the Thirteenth Machine Translation Summit (MTSummit 2011).

Paraphrasing adaptation for web search ranking

Chenguang Wang, Nan Duan, Ming Zhou, and Ming Zhang.

In Proc. 2013 Annual Meeting of the Association for Computational Linguistics (ACL 2013).

Measuring domain influence in heterogeneous networks

Quan Liu, Chenguang Wang, and Ming Zhang.

In Proc. 2014 ACM Int. Conf. on Web Search and Data Mining Workshop on Diffusion Networks and Cascade Analytics (WSDM 2014 Workshop).

Spectral label refinement for noisy and missing text labels

Yangqiu Song, Chenguang Wang, Ming Zhang, Hailong Sun, and Qiang Yang.

In Proc. 2015 AAAI Conf. on Artificial Intelligence (AAAI 2015).

Incorporating world knowledge to document clustering via heterogeneous information networks

Chenguang Wang, Yangqiu Song, Ahmed El-Kishky, Dan Roth, Ming Zhang, and Jiawei Han.

In Proc. 2015 ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD 2015).

Constrained information-theoretic tripartite graph clustering to identify semantically similar relations

Chenguang Wang, Yangqiu Song, Dan Roth, Chi Wang, Jiawei Han, Heng Ji, and Ming Zhang.

In Proc. 2015 Int. Joint Conf. on Artificial Intelligence (IJCAI 2015).

KnowSim: A document similarity measure on structured heterogeneous information networks

Chenguang Wang, Yangqiu Song, Haoran Li, Ming Zhang, and Jiawei Han.

In Proc. of 2015 IEEE Int. Conf. on Data Mining (ICDM 2015).

Text classification with heterogeneous information network kernels

Chenguang Wang, Yangqiu Song, Haoran Li, Ming Zhang, and Jiawei Han.

In Proc. 2016 AAAI Conf. on Artificial Intelligence (AAAI 2016).

RelSim: Relation similarity search in schema-rich heterogeneous information networks

Chenguang Wang, Yizhou Sun, Yanglei Song, Jiawei Han, Yangqiu Song, Lidan Wang, and Ming Zhang.

In Proc. 2016 SIAM Int. Conf. on Data Mining (SDM 2016)".

World knowledge as indirect supervision for document clustering

Chenguang Wang, Yangqiu Song, Ahmed El-Kishky, Dan Roth, Ming Zhang, and Jiawei Han.

In ACM Transactions on Knowledge Discovery from Data (TKDD 2016).

HINE: Heterogeneous information network embedding

Yuxin Chen, and Chenguang Wang.

In Proc. 2017 Int. Conf. on Database Systems for Advanced Applications (DASFAA 2017).

Towards re-defining relation understanding in financial domain

Chenguang Wang, Doug Burdick, Laura Chiticariu, Rajasekar krishnamurthy, Yunyao Li, and Huaiyu Zhu.

In Proc. of 2017 ACM SIGMOD Int. Conf. on Management of Data Workshop (SIGMOD 2017 Workshop).

Semi-supervised learning over heterogeneous information networks by ensemble of meta-graph guided random walks

He Jiang, Yangqiu Song, Chenguang Wang, Ming Zhang, and Yizhou Sun.

In Proc. 2017 Int. Joint Conf. on Artificial Intelligence (IJCAI 2017).

Active learning for black-box semantic role labeling with neural factors

Chenguang Wang, Laura Chiticariu, and Yunyao Li.

In Proc. 2017 Int. Joint Conf. on Artificial Intelligence (IJCAI 2017).

Crowd-in-the-loop: A hybrid approach for annotating semantic roles

Chenguang Wang, Alan Akbik, Laura Chiticariu, Yunyao Li, Fei Xia, and Anbang Xu.

In Proc. 2017 Conf. on Empirical Methods on Natural Language Processing (EMNLP 2017).

Distant meta-path similarities for text-based heterogeneous information networks

Chenguang Wang, Yangqiu Song, Haoran Li, Yizhou Sun, Ming Zhang, and Jiawei Han.

In Proc. 2017 ACM Int. Conf. on Information and Knowledge Management (CIKM 2017).

Unsupervised meta-path selection for similarity measure on heterogeneous information networks

Chenguang Wang, Yangqiu Song, Haoran Li, Ming Zhang, and Jiawei Han.

In Proc. 2018 Data Mining and Knowledge Discovery (DMKD 2018).

Co-occurrent features in semantic segmentation

Hang Zhang, Han Zhang, Chenguang Wang, and Junyuan Xie.

In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2019).

From shallow to deep language representations: Pre-training, fine-tuning, and beyond

Aston Zhang, Haibin Lin, Chenguang Wang, Mu Li, and Alexander Smola.

In Proc. 2019 ACM SIGKDD Int. Conf.on Knowledge Discovery and Data Mining (KDD 2019).

Language models with Transformers

Chenguang Wang, Mu Li, and Alexander Smola.

In arXiv preprint arXiv:1904.09408 (arXiv 2019).

Transformer on a diet

Chenguang Wang, Zihao Ye, Aston Zhang, Zheng Zhang, and Alexander Smola.

In arXiv preprint arXiv:2002.06170 (arXiv 2020).

PoD: Positional dependency-based word embedding for aspect term extraction

Yichun Yin, Chenguang Wang, and Ming Zhang.

In Proc. 2020 Int. Conf. on Computational Linguistics (COLING 2020).

GluonCV and GluonNLP: Deep learning in computer vision and natural language processing

Jian Guo, He He, Tong He, Leonard Lausen, Mu Li, Haibin Lin, Xingjian Shi, Chenguang Wang, Junyuan Xie, Aston Zhang, Hang Zhang, Zhi Zhang, Zhongyue Zhang, and Shuai Zheng.

In Journal of Machine Learning Research (JMLR 2020).

Language models are open knowledge graphs

Chenguang Wang, Xiao Liu, and Dawn Song.

In arXiv preprint arXiv:2010.11967 (arXiv 2020).

Zero-shot information extraction as a unified text-to-triple translation

Chenguang Wang, Xiao Liu, Zui Chen, Haoyun Hong, Jie Tang, and Dawn Song.

In Proc. 2021 Conf. on Empirical Methods on Natural Language Processing (EMNLP 2021).

Improving representation of the AOD to PM2.5 relationship with a convolutional neural network

Siyuan Shen, Aaron van Donkelaar, Randall V. Martin, Nathan Jacobs, and Chenguang Wang.

In Proc. 2022 Advancing Earth and Space Science (AGU 2022).

Protecting intellectual property of language generation APIs with lexical watermark

Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang.

In Proc. 2022 AAAI Conf. on Artificial Intelligence (AAAI 2022).

Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models

Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang and Xu Sun.

In Proc. 2022 Conf. on Empirical Methods on Natural Language Processing (EMNLP 2022).

Benchmarking Language Models for Code Syntax Understanding

Da Shen, Xinyun Chen*, Chenguang Wang*, Koushik Sen and Dawn Song.

In Proc. 2022 Conf. on Empirical Methods on Natural Language Processing (EMNLP 2022).

PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion

Jianhao Shen, Chenguang Wang*, Ye Yuan, Jiawei Han, Heng Ji, Koushik Sen, Ming Zhang* and Dawn Song*.

In Proc. 2022 Conf. on Empirical Methods on Natural Language Processing (EMNLP 2022).

IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models

Chenguang Wang, Xiao Liu and Dawn Song.

In Proc. 2022 Conf. on Empirical Methods on Natural Language Processing (EMNLP 2022).

Joint language semantic and structure embedding for knowledge graph completion

Jianhao Shen, Chenguang Wang*, Linyuan Gong, and Dawn Song.

In Proc. 2022 Int. Conf. on Computational Linguistics (COLING 2022).

DeepStruct: Pretraining of language models for structure prediction

Chenguang Wang*, Xiao Liu*, Zui Chen*, Haoyun Hong, Jie Tang, and Dawn Song.

In Proc. 2022 Annual Meeting of the Association for Computational Linguistics (ACL 2022).

Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study

Myeongseob Ko, Ming Jin, Chenguang Wang, Ruoxi Jia.

In International Conf. on Computer Vision (ICCV 2023).

CodeIPPrompt: Intellectual Property Infringement Assessment of Code Language Models

Zhiyuan Yu, Yuhao Wu, Ning Zhang, Chenguang Wang, Yevgeniy Vorobeychik, Chaowei Xiao.

In Proc. of the 40th International Conf. on Machine Learning (ICML 2023).

Enhancing Global Estimation of Fine Particulate Matter Concentrations by Including Geophysical a Priori Information in Deep Learning

Siyuan Shen, Chi Li, Aaron van Donkelaar, Nathan Jacobs, Chenguang Wang, Randall V. Martin.

In ACS ES&T Air (ACS ES&T Air 2024).

Evaluating Large Language Models in an Emerging Domain: A Pilot Study in Decentralized Finance

Joshua Carter Pearlson, Xiaoyuan Liu, Kripa Ann George, Dawn Song, and Chenguang Wang.

In The Twelfth International Conf. on Learning Representations DPFM Workshop (ICLR 2024 DPFM Workshop).

Benchmarking Zero-Shot Robustness of Multimodal Foundation Models: A Pilot Study

Chenguang Wang, Ruoxi Jia, Xin Liu, and Dawn Song.

In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2024 Workshop of Adversarial Machine Learning on Computer Vision).

Measuring Social Norms of Large Language Models

Ye Yuan, Kexin Tang, Jianhao Shen, Ming Zhang, and Chenguang Wang.

In 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2024).

Measuring Vision-Language STEM Skills of Neural Models

Jianhao Shen, Ye Yuan, Srbuhi Mirzoyan, Ming Zhang, and Chenguang Wang.

In The Twelfth International Conf. on Learning Representations (ICLR 2024).

Agent Instructs Large Language Models to be General Zero-Shot Reasoners

Nicholas Crispino, Kyle Montgomery, Fankun Zeng, Dawn Song, and Chenguang Wang.

In The Forty-first International Conference on Machine Learning (ICML 2024).

Re-Tuning: Overcoming the Compositionality Limits of Large Language Models with Recursive Tuning

Eric Pasewark*, Kyle Montgomery*, Kefei Duan, Dawn Song, and Chenguang Wang.

In The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024).

Preference Poisoning Attacks on Reward Model Learning

Junlin Wu, Jiongxiao Wang, Chaowei Xiao, Chenguang Wang, Ning Zhang, Yevgeniy Vorobeychik.

In IEEE Symposium on Security and Privacy (IEEE S&P 2025).

JudgeBench: A Benchmark for Evaluating LLM-based Judges

Sijun Tan*, Siyuan Zhuang*, Kyle Montgomery*, William Y. Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, Ion Stoica.

In The International Conf. on Learning Representations (ICLR 2025).

MLAN: Language-Based Instruction Tuning Improves Zero-Shot Generalization of Multimodal Large Language Models

Jianhong Tu*, Zhuohao Ni*, Nicholas Crispino, Zihao Yu, Michael Bendersky, Beliz Gunel, Ruoxi Jia, Xin Liu, Lingjuan Lyu, Dawn Song, Chenguang Wang.

In The Annual Meeting of the Association for Computational Linguistics (ACL 2025 Workshop on Towards Knowledgeable Foundation Models).

Predicting Task Performance with Context-aware Scaling Laws

Kyle Montgomery, David Park, Jianhong Tu, Michael Bendersky, Beliz Gunel, Dawn Song, Chenguang Wang.

In The Annual Meeting of the Association for Computational Linguistics (ACL 2025 Workshop on Towards Knowledgeable Foundation Models).

COSMIC: Generalized Refusal Direction Identification in LLM Activations

Vincent Siu, Nicholas Crispino, Zihao Yu, Sam Pan, Zhun Wang, Yang Liu, Dawn Song, Chenguang Wang.

In The Annual Meeting of the Association for Computational Linguistics (ACL 2025).

AGENTVIGIL: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents

Zhun Wang, Vincent Siu, Zhe Ye, Tianneng Shi, Yuzhou Nie, Xuandong Zhao, Chenguang Wang, Wenbo Guo, Dawn Song.

In Proc. 2025 Conf. on Empirical Methods on Natural Language Processing (EMNLP 2025).

Towards an AI biomedical scientist: Accelerating discoveries in neurodegenerative disease

Kaleigh F. Roberts, Eric C. Landsness, Justin Reese, Donald Elbert, Gabrielle Strobel, Elizabeth Wu, Yixin Chen, Albert Lai, Zachary B. Abrams, Mingfang Zhu, Justin Melendez, Srinivas Koutarapu, Sihui Song, Yun Chen, Robert Lazar, Payam Barnaghi, John F. Crary, Sergio Pablo Sardi, Marc D. Voss, Rajaraman Krishnan, Joel W. Schwartz, Ron Mallon, Gustavo A. Jimenez-Maggiora, Chenguang Wang, Thomas Sandmann, Niranjan Bose, Mukta Phatak, Gayle Wittenberg, Yannis G. Kevrekidis, Cassie S. Mitchell, Ludovico Mitchener, Towfique Raj, Luca Foschini, Gregory J. Moore, Randall J. Bateman.

In The Journal of Prevention of Alzheimer's Disease (JPAD 2026).

LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess

Sai Kolasani, Maxim Saplin, Nicholas Crispino, Kyle Montgomery, Jared Quincy Davis, Matei Zaharia, Chi Wang, Chenguang Wang.

In The Annual Conference on Neural Information Processing Systems (NeurIPS 2025 Workshop on Foundations of Reasoning).

Budget-aware Test-time Scaling via Discriminative Verification

Kyle Montgomery, Sijun Tan, Yuqi Chen, Siyuan Zhuang, Tianjun Zhang, Raluca Ada Popa, Chenguang Wang.

In The Annual Conference on Neural Information Processing Systems (NeurIPS 2025 Workshop on Efficient Reasoning).

VMDT: Decoding the Trustworthiness of Video Foundation Models

Yujin Potter, Zhun Wang, Nicholas Crispino, Kyle Montgomery, Alexander Xiong, Ethan Y. Chang, Francesco Pinto, Yuqi Chen, Rahul Gupta, Morteza Ziyadi, Christos Christodoulopoulos, Bo Li, Chenguang Wang, Dawn Song.

In The Annual Conference on Neural Information Processing Systems (NeurIPS 2025).

LLM Agentic System Safety Requires Hybrid Alignment

Vincent Siu, Kyle Montgomery, Yujin Potter, Zhun Wang, Dawn Song, Chenguang Wang.

In The International Conf. on Learning Representations (ICLR 2026 AIWILD Workshop).

RepIt: Steering Language Models with Concept-Specific Refusal Vectors

Vincent Siu, Nathan W Henry, Nicholas Crispino, Yang Liu, Dawn Song, Chenguang Wang.

In The International Conf. on Learning Representations (ICLR 2026).

SkillOptimizer: Agent Skill Optimization Through Subskills Without Task Supervision

Nicholas Crispino, Shubham Gaur, Xuefang Yang, Angela Yu, Berat Ercevik, Clara Sapugay, Yujin Potter, Dawn Song, Chenguang Wang.

In The International Conference on Machine Learning (ICML 2026 AIWILD Workshop).

FaultLoc: Evaluating Coding Agents For Fault Localization

Jianhong Tu, Shubham Gaur, Rathik Murtinty, Zhun Wang, Tianneng Shi, Dawn Song, Chenguang Wang.

In The International Conference on Machine Learning (ICML 2026 AIWILD Workshop).

MIRAGE: A Polarity-Flipping Encoding Subspace in LLM Agents

Pratibha Revankar, Kargi Chauhan, Jihye Kim, Sadiba Nusrat Nur, Vincent Siu, Chenguang Wang.

In The International Conference on Machine Learning (ICML 2026 AIWILD Workshop).

SafeClawBench: An Operating-System Perspective on Evaluating the Security of Claw-like Agent Systems

Peizhi Niu, Shangding Gu, Wenjie Qu, Tianneng Shi, Yuankai Li, Ahmad Tawaha, Hend Alzahrani, Vincent Siu, Boyi Li, Chenguang Wang, Jiaheng Zhang, Basel Alomair, Ming Jin, Muhao Chen, Chi Wang, Costas Spanos, Dawn Song.

In The International Conference on Machine Learning (ICML 2026 AIWILD Workshop).

Controlling Tool Use with Heading-Specific Activation Steering

Yuqi Chen, Vincent Siu, Yang Liu, Dawn Song, Chenguang Wang.

In The International Conference on Machine Learning (ICML 2026 AIWILD Workshop).

SudoBench: A Contextual Authorization Benchmark for LLM Agents

Vincent Siu, Tianneng Shi, Shangding Gu, Zhun Wang, Dawn Song, Chenguang Wang.

In The International Conference on Machine Learning (ICML 2026 AIWILD Workshop).

FICO: Evaluating Vision-Language Models under Visual Fidelity and Compression at Scale

Jianhong Tu, Nicholas Crispino, Kyle Montgomery, Chenguang Wang, Dawn Song.

In The Annual Meeting of the Association for Computational Linguistics (ACL 2026).

ProActor: Timing-Aware Reinforcement Learning for Proactive Task Scheduling Agents

Lei Ding, Bin He, Chenguang Wang, Yang Liu.

In The Annual Meeting of the Association for Computational Linguistics (ACL 2026).

A benchmark of expert-level academic questions to assess AI capabilities

Long Phan, Wenjin Zhang, Nick Crispino, Chenguang Wang, Daofeng Li, Jiawei Shen, Kyle Montgomery, Hannah Szlyk, Ting Wang, Summer Yue, Alexandr Wang, Dan Hendrycks, many others.

In Nature 2026.

Position: Agent Security Needs Redefinition through a Holistic Framework

Vincent Siu, Jingxuan He, Kyle Montgomery, Zhun Wang, Chenguang Wang, Dawn Song.

In The International Conference on Machine Learning (ICML 2026).

CyberGym-E2E: Scalable Real-World Benchmark for AI Agents’ End-to-End Cybersecurity Capabilities

Tianneng Shi, Robin Rheem, Dongwei Jiang, Mona Wang, Francisco De La Riega, Zhun Wang, Jingzhi Jiang, Alexander Cheung, Sean Tai, Jonah Cha, Jianhong Tu, Gabriel Han, Chenguang Wang, Wenbo Guo, Jingxuan He, Dawn Song.

In The International Conference on Machine Learning (ICML 2026).

Peer-Preservation in Frontier Models

Yujin Potter, Nicholas Crispino, Vincent Siu, Chenguang Wang, Dawn Song.

In The International Conference on Machine Learning (ICML 2026).

SteeringSafety: A Systematic Safety Evaluation Framework of Representation Steering in LLMs

Vincent Siu, Nicholas Crispino, David Park, Nathan W Henry, Zhun Wang, Yang Liu, Dawn Song, Chenguang Wang.

In The International Conference on Machine Learning (ICML 2026).

Component and Dimension Sparsity in Transformer Refusal Mechanisms

Vincent Siu, Glenn Grant-Richards, Vlad Pavlovich, Yizhou Sun, Dawn Song, Chenguang Wang.

In The Conference on Language Modeling (COLM 2026).

Humanity’s Last Exam

Long Phan, Wenjin Zhang, Nick Crispino, Chenguang Wang, Daofeng Li, Jiawei Shen, Kyle Montgomery, Hannah Szlyk, Ting Wang, Summer Yue, Alexandr Wang, Dan Hendrycks, many others.

In arXiv preprint 2025.

rLLM: A Framework for Post-Training Language Agents

Sijun Tan, Michael Luo, Colin Cai, Tarun Venkat, Kyle Montgomery, Aaron Hao, Tianhao Wu, Arnav Balyan, Manan Roongta, Chenguang Wang, Li Erran Li, Raluca Ada Popa, Ion Stoica.

In Notion 2025.

rLLM v0.2: RL Training over General Agentic Programs

Sijun Tan, Kyle Montgomery, the rLLM Team.

In rLLM blog 2025.

Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey

Wei-Chieh Huang, Weizhi Zhang, Yueqing Liang, Yuanchen Bei, Yankai Chen, Tao Feng, Xinyu Pan, Zhen Tan, Yu Wang, Tianxin Wei, Shanglin Wu, Ruiyao Xu, Liangwei Yang, Rui Yang, Wooseong Yang, Chin-Yuan Yeh, Hanrong Zhang, Haozhen Zhang, Siqi Zhu, Henry Peng Zou, Wanjia Zhao, Song Wang, Wujiang Xu, Zixuan Ke, Zheng Hui, Dawei Li, Yaozu Wu, Langzhou He, Chen Wang, Xiongxiao Xu, Baixiang Huang, Juntao Tan, Shelby Heinecke, Huan Wang, Caiming Xiong, Ahmed A. Metwally, Jun Yan, Chen-Yu Lee, Hanqing Zeng, Yinglong Xia, Xiaokai Wei, Ali Payani, Yu Wang, Haitong Ma, Wenya Wang, Chenguang Wang, Yu Zhang, Xin Wang, Yongfeng Zhang, Jiaxuan You, Hanghang Tong, Xiao Luo, Xue Liu, Yizhou Sun, Wei Wang, Julian McAuley, James Zou, Jiawei Han, Philip S. Yu, Kai Shu.

In arXiv preprint 2026.

rLLM On-Policy Distillation: Training Smaller Students from Stronger Teachers

Brian Chen, Kyle Montgomery, the rLLM Team.

In rLLM blog 2026.

A Framework for Formalizing LLM Agent Security

Vincent Siu, Jingxuan He, Kyle Montgomery, Zhun Wang, Neil Gong, Chenguang Wang, Dawn Song.

In arXiv preprint 2026.

AgentBeats: Agentifying Agent Assessment for Openness, Standardization, and Reproducibility

Xiaoyuan Liu, Jianhong Tu, Yuqi Chen, Siyuan Xie, Sihan Ren, Tianneng Shi, Gal Gantar, Evan Sandoval, Donghyun Lee, Daniel Miao, Peter J. Gilbert, Nick Hynes, Mauro Staver, Warren He, David Marn, Andrew Low, Xi Zhang, Elron Bandel, Michal Shmueli-Scheuer, Siva Reddy, Alexandre Drouin, Alexandre Lacoste, Ramayya Krishnan, Elham Tabassi, Yu Su, Victor Barres, Chenguang Wang, Wenbo Guo, Dawn Song.

In arXiv preprint 2026.

SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks

Xiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, Zonglin Di, Yifeng He, Shenghan Zheng, Kyoung Whan Choe, Jiankai Sun, Shuyi Wang, Chujun Tao, Binxu Li, Xuandong Zhao, Hejia Geng, Xiaojun Wu, Junwei Zhou, Xiaokun Chen, Hanwen Xing, Yubo Li, Qunhong Zeng, Di Wang, Yuanli Wang, Roey Ben Chaim, Penghao Jiang, Haotian Shen, Luyang Kong, Xinyi Liu, Runhui Wang, Xuanqing Liu, Jiachen Li, Xin Lan, Yueqian Lin, Wengao Ye, Junwei He, Songlin Li, Yue Zhang, Yipeng Gao, Yijiang Li, Ze Ma, Liqiang Jing, Tianyu Wang, Kaixin Li, Yiqi Xue, Haoran Lyu, Yizhuo He, Yuchen Tian, Shutong Wu, Bowei Wang, Yixuan Gao, Bo Chen, Litong Liu, Sikai Cheng, Jiajun Bao, Shuaicheng Tong, Shuwen Xu, Terry Yue Zhuo, Tinghan Ye, Qi Qi, Miao Li, Longtai Liao, Zelin Tan, Chang Shi, Xilin Tang, Srinath Tankasala, Boqin Yuan, Yaoyao Qian, Jianhong Tu, Chenguang Wang, Yizhou Sun, Wei Wang, Aaron Taylor, Ziyue Yang, Changkun Guan, Zhikang Dong, Xinyu Zhang, Steven Dillmann, Han-chung Lee, Dawn Song.

In arXiv preprint 2026.

Agents’ Last Exam

Yiyou Sun, Xinyang Han, Jianhong Tu, Vincent Siu, Patricia Cavazos-Rehg, Daofeng Li, Kyle Montgomery, Hannah Szlyk, Chenguang Wang, Ting Wang, Dawn Song, many others.

In arXiv preprint 2026.