可靠机器学习的熵正则化 | AIRS in the AIR 预告

11月28日 16:00-17:00

第 61 期

Entropic Regularization for Reliable Machine Learning

第61期AIRS in the AIR邀请佐治亚理工学院工业工程系博士生王捷分享可靠机器学习与机器学习模型鲁棒性的相关研究。王捷曾在运筹与管理科学领域顶级期刊Operations Research发表论文，他目前在佐治亚理工学院工业工程系攻读博士，主要的研究方向是不确定性下的决策，曾获2022 ISyE Robert Goodell Brown Research Excellence award、Winner in 2022 INFORMS Poster Competition等奖项。

01

主持人

庞旭芳

AIRS特种机器人中心副研究员

02

报告嘉宾

王捷

佐治亚理工学院工业工程系博士生

Jie Wang is a 4-th year Ph.D. student in Industrial Engineering at the H. Milton Stewart School of Industrial and Systems Engineering. He received BS degree in Pure of Mathematics Major from The Chinese University of Hong Kong, Shenzhen. His main research studies decision-making under uncertainty. His research has been published on several journals and conferences including Operations Research, Information and Inference a Journal of the IMA, NeurIPS, AISTATS, and ISIT. He has received several awards, such as 2022 ISyE Robert Goodell Brown Research Excellence award, Winner in 2022 INFORMS Poster Competition, and Winner for Best Theoretical Paper in 2023 INFORMS Workshop on DMDA.

03

报告介绍

报告主题：Entropic Regularization for Reliable Machine Learning

报告嘉宾：王捷

Despite the growing prevalence of artificial neural networks in real-world applications, their vulnerability to adversarial attacks remains to be a significant concern, which motivates us to investigate the robustness of machine learning models. While various heuristics aim to optimize the distributionally robust risk using the Wasserstein metric, such a notion of robustness frequently encounters computation intractability. To tackle the computational challenge, we develop a novel approach to adversarial training that integrates entropic regularization into the distributionally robust risk function. This regularization brings a notable improvement in computation compared with the original formulation. We develop stochastic gradient methods with near-optimal sample complexity to solve this problem efficiently. Moreover, we establish the regularization effects and demonstrate this formulation is asymptotic equivalence to a regularized empirical risk minimization (ERM) framework, by considering various scaling regimes of the entropic regularization $\eta$ and robustness level $\rho$. These regimes yield gradient norm regularization, variance regularization, or a smoothed gradient norm regularization that interpolates between these extremes. We numerically validate our proposed method in supervised learning and reinforcement learning applications and showcase its state-of-the-art performance against various adversarial attacks.

活动时间

2023年11月28日 16:00 - 17:00

参与方式

扫码报名，与嘉宾实时互动

AIRS in the AIR 为 AIRS 重磅推出的系列活动，与您一起探索人工智能与机器人领域的前沿技术、产业应用、发展趋势。2022年举办至今，已邀请百余位国内外嘉宾，吸引了超40万人次参与。

AIRS in the AIR 往期视频回顾请前往哔哩哔哩观看：https://www.bilibili.com/video/BV1yr4y1z7zr/?spm_id_from=333.788

（如有感兴趣的主题或合作意向，欢迎在 AIRS 公众号后台留言。）

相关阅读

AIRS in the AIR | 非策略评估问题的稳健型算法和提升分布式随机梯度算法的暂态时间

AIRS in the AIR | NeurlPS 2022交流分享会

AIRS in the AIR | “特种机器人”系列讲座