Approximate Exploitability: Learning a Best Response
Finbarr Timbers, Nolan Bard, Edward Lockhart et al. · International Joint Conference on Artificial Intelligence · 2022
Researchers have shown that neural networks are vulnerable to adversarial examples and subtle environment changes. The resulting errors can look like blunders to humans, eroding trust in these agents. In prior games research, agent evaluation often focused on the in-practice game outcomes. Such evaluation typically fails to evaluate robustness to worst-case outcomes. Computer poker research has examined how to assess such worst-case performance. Unfortunately, exact computation is infeasible with larger domains, and existing approximations are poker-specific. We introduce ISMCTS-BR, a scalable search-based deep reinforcement learning algorithm for learning a best response to an agent, approximating worst-case performance. We demonstrate the technique in several games against a variety of agents, including several AlphaZero-based agents. Supplementary material is available at https://arxiv.org/abs/2004.09677.
Cited by 11 works. doi:10.24963/ijcai.2022/484 Open-access PDF
Authors
- Finbarr Timbers
- Nolan Bard
- Edward Lockhart
- Marc Lanctot
- Martin S. Schmid
- NEIL R. BURCH
- Julian Schrittwieser
- Thomas Hubert
- Michael Bowling
References
- Human-level control through deep reinforcement learning
- A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
- Dataset Shift in Machine Learning
- Agnostic Learning with Unknown Utilities
- DeepStack: Expert-level artificial intelligence in heads-up no-limit poker
- Searching for solutions in games and artificial intelligence
- Synthesizing Robust Adversarial Examples
- Regret Minimization in Games with Incomplete Information
- Superhuman AI for heads-up no-limit poker: Libratus beats top professionals
- Chinook: The World Man-Machine Checkers Champion