When was MuZero released and what games did it master?
DeepMind published the MuZero preprint on the 19th of November 2019. The program mastered chess, Go, shogi, and all 57 games in the Arcade Learning Environment through self-play alone, with no access to the rules of any game.
How does MuZero differ from AlphaZero?
MuZero replaces AlphaZero's explicit game-rules simulator with three separate learned neural networks: one for representing board states, one for predicting how actions change those states, and one for estimating policy and value. AlphaZero was restricted to two-player, zero-sum games, while MuZero also works in single-agent environments with continuous rewards of arbitrary magnitude.
What hardware did MuZero use to train on board games and Atari games?
Board-game training used 16 third-generation TPUs for learning and 1000 TPUs for selfplay, running 800 simulations per step. Atari training used 8 TPUs for training and 32 for selfplay, at 50 simulations per step.
How did MuZero perform against R2D2 on the Atari benchmark?
MuZero surpassed R2D2, the Recurrent Replay Distributed DQN, in both mean and median performance across the 57-game Arcade Learning Environment. It never performed well on 6 of those 57 games.
What is EfficientZero and how does it improve on MuZero?
EfficientZero is a variant of MuZero proposed in late 2021. It achieved 194.3 percent mean human performance and 109.0 percent median performance on the Atari 100k benchmark using only two hours of real-time game experience.
What is Stochastic MuZero and how does it differ from MuZero?
Stochastic MuZero is a variant proposed in early 2022 that extends MuZero to games involving randomness, such as 2048 and backgammon. It uses afterstate dynamics and chance codes to account for unpredictable elements when training the dynamics network.