Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning
Open accessAbstract
In recent years, neural networks have enjoyed a renaissance as function approximators in reinforcement learning. Two decades after Tesauro's TD-Gammon achieved near top-level human performance in backgammon, the deep reinforcement learning algorithm DQN achieved human-level performance in many Atari 2600 games. The purpose of this study is twofold. First, we propose two activation functions for neural network function approximation in reinforcement learning: the sigmoid-weighted linear unit (SiLU) and its derivative function (dSiLU). The activation of the SiLU is computed by the sigmoid function multiplied by its input. Second, we suggest that the more traditional approach of using on-policy learning with eligibility traces, instead of experience replay, and softmax action selection can be competitive with DQN, without the need for a separate target network. We validate our proposed approach by, first, achieving new state-of-the-art results in both stochastic SZ-Tetris and Tetris with a small 10 × 10 board, using TD(λ) learning and shallow dSiLU network agents, and, then, by outperforming DQN in the Atari 2600 domain by using a deep Sarsa(λ) agent with SiLU and dSiLU hidden units.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
SiLU provides a smooth non-linearity that improves model performance, particularly in deeper models.
“SiLU combines the sigmoid function with its input, providing a smooth non-linearity that improves model performance, particularly in deeper models.”
From Tiny Language Models for Automation and Control: Overview, Potential Applications, and Future Research Directions (2025) · cites this paper as [84]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · see citing works
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148 · see citing works
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · see citing works
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300 · see citing works
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161 · see citing works
Information processing in dynamical systems: foundations of harmony theory
Smolensky, P. (1986) · The MIT Press eBooks · Cited by 1,673 · see citing works
Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit
Hahnloser, R. H. R., Sarpeshkar, R., Mahowald, M. A., Douglas, R. J., & Seung, H. S. (2000) · Nature · doi:10.1038/35016072 · Cited by 1,293 · see citing works
Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding
Sutton, R. S. (1995) · Cited by 1,051 · see citing works
TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play
Tesauro, G. J. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.215 · Cited by 803 · see citing works
Rumelhart, D. E., & McClelland, J. L. (1986) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · Cited by 553 · see citing works
Issues in Using Function Approximation for Reinforcement Learning
Thrun, S., & Schwartz, A. (1999) · Cited by 315 · see citing works
Unsupervised learning of distributions on binary vectors using two layer networks
Freund, Y., & Haussler, D. H. (1991) · Cited by 269 · see citing works
Approximate modified policy iteration and its application to the game of Tetris
Scherrer, B., Ghavamzadeh, M., Gabillon, V., Lesner, B., & Geist, M. (2015) · HAL · Cited by 71 · see citing works
Burgiel, H. (1997) · The Mathematical Gazette · doi:10.2307/3619195 · Cited by 68 · see citing works
Neural Network Ensembles in Reinforcement Learning
Faußer, S., & Schwenker, F. (2013) · Neural Processing Letters · doi:10.1007/s11063-013-9334-5 · Cited by 51 · see citing works
Approximate Dynamic Programming Finally Performs Well in the Game of Tetris
Gabillon, V., Ghavamzadeh, M., Scherrer, B., & Hal, H. I. (2013) · Cited by 45 · see citing works
Improvements on Learning Tetris with Cross Entropy
Thiéry, C., & Scherrer, B. (2009) · ICGA Journal · doi:10.3233/icg-2009-32104 · Cited by 38 · see citing works
Expected energy-based restricted Boltzmann machine for classification
Elfwing, S., Uchibe, E., & Doya, K. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.09.006 · Cited by 33 · see citing works
Self-citationElfwing, S., Uchibe, E., & Doya, K. (2016) · Neural Networks · doi:10.1016/j.neunet.2016.07.013 · Cited by 17 · see citing works
Self-citationHigh-Dimensional Function Approximation for Knowledge-Free Reinforcement Learning
Jaśkowski, W., Szubert, M., Liskowski, P., et al. (2015) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2739480.2754783 · Cited by 11 · see citing works
Flagged first, then by citations
Research works printed but not linked: 9
- Elfwing, S., Uchibe, E., & Doya, K. (2017). Sigmoid-weighted linear units for neu- ral network function approximation in reinforcement learning. arXiv:1702. 03118 [cs.LG].
- Morgan Kaufmann.
- Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., & Maria, A. D. et al., (2015). Massively parallel methods for deep reinforcement learning. arXiv: 1507.04296 [cs.LG]. · arXiv
- Ramachandran, P., Zoph, B., & Le, Q. V. (2017). Searching for activation functions. arXiv:1710.05941 [cs.NE]. · arXiv
- Rummery, G. A., & Niranjan, M. (1994). On-line Q-learning using connectionist systems. Tech. Rep. CUED/F-INFENG/TR 166, Cambridge University Engineering Department.
- Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2016). Prioritized experience replay. In International conference on learning representations, Puerto Rico.
- Szita, I., & Szepesvári, C. (2010). SZ-Tetris as a benchmark for studying key problems of reinforcement learning. In ICML 2010 workshop on machine learning and games.
- van Hasselt, H. (2010). Double q-learning. In Proceedings of advances in neural information processing systems (pp. 2613-2621).
- van Hasselt, H., Guez, A., & Silver, D. (2015). Deep Reinforcement learning with double q-learning. arXiv:1509.06461 [cs.LG]. · arXiv
Cited by
Sorted by citations
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · see citing works
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708 · see citing works
YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications
Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., Nie, W., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.02976 · Cited by 1,792 · see citing works
“Among these activation functions, SiLU is the most used.”
BackgroundInfluentialActivation functions in deep learning: a comprehensive survey and benchmark
Dubey, S. R., Singh, S. K., & Chaudhuri, B. B. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.06.111 · Cited by 1,259 · see citing works
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection
Li, Y., Yu, A. W., Meng, T., Caine, B., Ngiam, J., Peng, D., Shen, J., Lu, Y., Zhou, D., Le, Q. V., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01667 · Cited by 563 · see citing works
EfficientDet: Scalable and Efficient Object Detection
Tan, M., Pang, R., & Le, Q. V. (2019) · arXiv · Cited by 9,841 · see citing works
Metabolomic profiles predict individual multidisease outcomes
Buergel, T., Steinfeldt, J., Ruyoga, G., Pietzner, M., Bizzarri, D., Vojinovic, D., Upmeier zu Belzen, J., Loock, L., Kittner, P., Christmann, L., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01980-3 · Cited by 406 · see citing works
Predicting cellular responses to perturbation across diverse contexts with State
Adduri, A. K., Gautam, D., Bevilacqua, B., Naghipourfar, M., Imran, A., Shah, R., Teyssier, N., Verma, R., Carpenter, C., Eraslan, B., et al. (2026) · Cell · doi:10.1016/j.cell.2026.07.052 · Cited by 100 · see citing works
Fdftnet: facing off fake images using fake detection fine-tuning network
Jeon, H., Bang, Y., & Woo, S. S. (2020) · IFIP Advances in Information and Communication Technology · doi:10.1007/978-3-030-58201-2_28 · Cited by 65 · see citing works
Lamaakal, I., Maleh, Y., El Makkaoui, K., Ouahbi, I., Pławiak, P., Alfarraj, O., Almousa, M., & Abd El-Latif, A. A. (2025) · Sensors · doi:10.3390/s25051318 · Cited by 60 · see citing works
A Generalization Performance Study Using Deep Learning Networks in Embedded Systems
Gorospe, J., Mulero, R., Arbelaitz, O., Muguerza, J., & Antón, M. Á. (2021) · Sensors · doi:10.3390/s21041031 · Cited by 32 · see citing works
Lai, D. K.-H., Cheng, E. S.-W., So, B. P.-H., Mao, Y.-J., Cheung, S. M.-Y., Cheung, D. S. K., Wong, D. W.-C., & Cheung, J. C.-W. (2023) · Mathematics · doi:10.3390/math11143081 · Cited by 18 · see citing works
Titans: Learning to Memorize at Test Time
Behrouz, A., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2501.00663 · Cited by 24 · see citing works
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
Cheng, X., Zeng, W., Dai, D., Chen, Q., Wang, B., Xie, Z., Huang, K., Yu, X., Hao, Z., Zhang, H., et al. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.226 · Cited by 3 · see citing works
- Authors
- Elfwing, S., Uchibe, E., & Doya, K.
- Affiliations
- Advanced Telecommunications Research Institute International (JP)
- Okinawa Institute of Science and Technology Graduate University (JP)
- Corresponding author
- Elfwing, Stefan
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2018-11
- Volume
- 107
- Pages
- 3-11
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2962834855
- Cited by
- 2,119
- References
- 26
- Citations by year
- 2026: 408 · 2025: 584 · 2024: 428 · 2023: 319 · 2022: 162 · 2021: 109 · 2020: 49 · 2019: 41 · 2018: 17 · 2017: 2
- FWCI
- 30.43
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Neural Networks (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Institutional Repositories DataBase (IRDB) (submittedVersion, cc-by-nc-nd)
- Okinawa Institute of Science and Technology Graduate University (Okinawa Institute of Science and Technology Graduate University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Funders
- New Energy and Industrial Technology Development Organization (NEDO)
- MEXT KAKENHI (16H06563, 17H06042)
- Okinawa Institute of Science and Technology Graduate University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- neural network function approximation, reinforcement learning, deep Q-network, on-policy learning, eligibility traces, Sarsa(lambda), Tetris, Atari 2600, target network
- MeSH terms
- Deep Learning, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
