AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients

Zhuang, Juntang, Tang, Tommy, Ding, Yifan · arXiv · 2020