CoChat
Sign Up
Cited by 187Computer Graphics and Visualization TechniquesAdvanced Database Systems and QueriesScientific Computing and Data Management

World Models

Open access
Ha, David, Schmidhuber, Jürgen·2018·Zenodo
DOI 10.5281/zenodo.1207631
Open this paper in CoChat — free accountOpen PDF

Abstract

We explore building generative neural network models of popular reinforcement learning environments. Our world model can be trained quickly in an unsupervised manner to learn a compressed spatial and temporal representation of the environment. By using features extracted from the world model as inputs to an agent, we can train a very compact and simple policy that can solve the required task. We can even train our agent entirely inside of its own hallucinated dream generated by its world model, and transfer this policy back into the actual environment. An interactive version of this article is available at worldmodels.github.io.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.5281/zenodo.1207631

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Early work by Ha and Schmidhuber focused on abstracting the external world to gain a deep understanding of its underlying mechanisms.

    “early work by Ha and Schmidhuber [66] focused on abstracting the external world to gain a deep understanding of its underlying mechanisms.”

    From Understanding World or Predicting Future? A Comprehensive Survey of World Models (2026) · cites this paper as [66]

    Section 3.1 (p. 4) says the world model learns a low-dimensional representation of observations, supporting the abstraction part of the claim. It describes using that representation to train a controller, however, not gaining a deep understanding of the external world's underlying mechanisms. The Introduction (p. 2) likewise frames the work around world models and agent training. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Ha and Schmidhuber's agent model receives feedback from the real-world environment, transforms it into inputs that train the model, and simulates potential outcomes following specific actions, creating a mental simulation of potential future world evolutions with decisions based on predicted outcomes.

    “In this pioneering model, the agent receives feedback from the real-world environment, which is then transformed into a series of inputs that train the model.This model is adept at simulating potential outcomes following specific actions within the external environment.Essentially, it creates a mental simulation of potential future world evolutions, with decisions made based on the predicted”

    From Understanding World or Predicting Future? A Comprehensive Survey of World Models (2026) · cites this paper as [66]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Ha et al. significantly revived and popularized the term 'world model' in 2018 by proposing neural-network-based implicit models for learning latent representations.

    “Ha et al. [65,66] significantly revived and popularized the term”

    From Understanding World or Predicting Future? A Comprehensive Survey of World Models (2026) · cites this paper as [66]

    The Abstract (p. 1) describes a neural-network world model that learns compressed spatial and temporal representations, supporting the model-description part of the claim. The Introduction (p. 2) discusses earlier RNN-based world models, but our copy does not establish that this 2018 paper significantly revived or popularized the term. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Traditional world models primarily focused on predicting discrete or static future states.

    “Traditional world models primarily focused on predicting discrete or static future states [66,109].”

    From Understanding World or Predicting Future? A Comprehensive Survey of World Models (2026) · cites this paper as [66]

    The Related Work section (p. 11) says some earlier feed-forward models predicted the next video frame, which supports the future-state prediction part of the claim. It also describes earlier RNN models generating sequences of future frames, so it does not establish that traditional world models primarily predicted discrete or static states. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Knowing P(Z) would be useful for many kinds of predictions; this would constitute a world model over the events that cause our observations.

    “In this idealized world, knowing P(Z) would be useful for many kinds of predictions; this would constitute a world model over the events that cause our observations (Werbos, 1987;Ha & Schmidhuber, 2018;Richens & Everitt, 2024).”

    From The Platonic Representation Hypothesis (2024) · cites this paper as Ha & Schmidhuber, 2018

    Section 2.2 (p. 3) describes a probabilistic world model that predicts future latent representations, and the Full World Model section (p. 5) says those predictions help guide decisions. But its distribution is conditional on actions and prior states, P(zₜ₊₁ | aₜ, zₜ, hₜ); the paper does not identify this with an unconditional P(Z) over events that cause observations. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 87

  1. Al-Shedivat, M., Bansal, T., Burda, Y., Sutskever, I., Mordatch, I., and Abbeel, P. Continuous adaptation via meta-learning in nonstationary and competitive environments. ArXiv preprint , October 2017. URL https: //arxiv.org/abs/1710.03641 .
  2. Alvernaz, S. and Togelius, J. Autoencoder-augmented neuroevolution for visual doom playing. ArXiv preprint , July 2017. URL https://arxiv.org/abs/1707. 03902 .
  3. Arulkumaran, K., Deisenroth, M. P., Brundage, M., and Bharath, A. A. Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine , 34(6):26-38, Nov 2017. ISSN 1053-5888. doi: 10.1109/MSP.2017. 2743240. URL https://arxiv.org/abs/1708. 05866 . · DOI
  4. Bansal, T., Pachocki, J., Sidor, S., Sutskever, I., and Mordatch, I. Emergent complexity via multi-agent competition. ArXiv preprint , October 2017. URL https: //arxiv.org/abs/1710.03748 .
  5. Bartol, Thomas M, Jr, Bromer, Cailey, Kinney, Justin, Chirillo, Michael A, Bourne, Jennifer N, Harris, Kristen M, and Sejnowski, Terrence J. Nanoconnectomic upper bound on the variability of synaptic plasticity. eLife Sciences Publications, Ltd , 2015. doi: 10.7554/ eLife.10778. URL https://doi.org/10.7554/ eLife.10778 .
  6. Bousmalis, K., Irpan, A., Wohlhart, P., Bai, Y., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K., Levine, S., and Vanhoucke, V. Using simulation and domain adaptation to improve efficiency of deep robotic grasping. ArXiv e-prints , September 2017. URL https://arxiv.org/abs/1709.07857 . · arXiv
  7. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. Openai gym. ArXiv preprint , June 2016. URL https://arxiv. org/abs/1606.01540 .
  8. Cheang, L. and Tsao, D. The code for facial identity in the primate brain. Cell , 2017. doi: 10.1016/j.cell. 2017.05.011. URL http://www.cell.com/cell/ fulltext/S0092-8674%2817%2930538-X . · DOI
  9. Chiappa, S., Racaniere, S., Wierstra, D., and Mohamed, S. Recurrent environment simulators. ArXiv preprint , April 2017. URL https://arxiv.org/abs/ 1704.02254 .
  10. Denton, E. and Birodkar, V. Unsupervised learning of disentangled representations from video. ArXiv preprint , May 2017. URL https://arxiv.org/abs/1705. 10915 .
  11. Depeweg, S., Hernandez-Lobato, J, Doshi-Velez, F., and Udluft, S. Learning and policy search in stochastic dynamical systems with bayesian neural networks. ArXiv preprint , May 2016. URL https://arxiv.org/ abs/1605.07127 . · arXiv
  12. Dosovitskiy, A. and Koltun, V. Learning to act by predicting the future. ArXiv preprint , November 2016. URL https: //arxiv.org/abs/1611.01779 .
  13. Fernando, C., Banarse, D., Blundell, C., Zwols, Y., Ha, D., Rusu, A., Pritzel, A., and Wierstra, D. Pathnet: Evolution channels gradient descent in super neural networks. ArXiv preprint , January 2017. URL https://arxiv.org/ abs/1701.08734 . · arXiv
  14. Finn, C., Tan, X., Duan, Y., Darrell, T., Levine, S., and Abbeel, P. Deep spatial autoencoders for visuomotor learning. ArXiv preprint , September 2015. URL https: //arxiv.org/abs/1509.06113 .
  15. Foster, David J. Replay comes of age. Annual Review of Neuroscience , 40(1):581-602, 2017. doi: 10.1146/annurev-neuro-072116-031538. URL https://doi.org/10.1146/ annurev-neuro-072116-031538 . · DOI · Paper page
  16. French, Robert M. Catastrophic interference in connectionist networks: Can it be predicted, can it be prevented? In Cowan, J. D., Tesauro, G., and Alspector, J. (eds.), Advances in Neural Information Processing Systems 6 , pp. 1176-1177. Morgan-Kaufmann, 1994. URL https://goo.gl/jwpLsk .
  17. Gauci, Jason and Stanley, Kenneth O. Autonomous evolution of topographic regularities in artificial neural networks. Neural Computation , 22(7):1860-1898, July 2010. ISSN 0899-7667. doi: 10.1162/neco.2010. 06-09-1042. URL http://eplex.cs.ucf.edu/ papers/gauci\_nc10.pdf . · DOI
  18. Gemici, M., Hung, C., Santoro, A., Wayne, G., Mohamed, S., Rezende, D., Amos, D., and Lillicrap, T. Generative temporal models with memory. ArXiv preprint , February 2017. URL https://arxiv.org/abs/1702. 04649 .
  19. Gerrit, M., Fischer, J., and Whitney, D. Motion-dependent representation of space in area mt+. Neuron , 2013. doi: 10.1016/j.neuron.2013.03.010. URL http://dx.doi. org/10.1016/j.neuron.2013.03.010 . · DOI
  20. Gers, F., Schmidhuber, J., and Cummins, F. Learning to forget: Continual prediction with lstm. Neural Computation , 12(10):2451-2471, October 2000. ISSN 0899-7667. doi: 10.1162/089976600300015015. URL ftp://ftp. idsia.ch/pub/juergen/FgGates-NC.pdf . · DOI
  21. Gomez, F. and Schmidhuber, J. Co-evolving recurrent neurons learn deep memory pomdps. Proceedings of the 7th Annual Conference on Genetic and Evolutionary Computation , pp. 491-498, 2005. doi: 10.1145/ 1068009.1068092. URL ftp://ftp.idsia.ch/ pub/juergen/gecco05gomez.pdf .
  22. Gomez, F., Schmidhuber, J., and Miikkulainen, R. Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research , 9: 937-965, June 2008. ISSN 1532-4435. URL http:// people.idsia.ch/˜juergen/gomez08a.pdf .
  23. Gottlieb, J., Oudeyer, P., Lopes, M., and Baranes, A. Information-seeking, curiosity, and attention: computational and neural mechanisms. Cell , September 2013. doi: 10.1016/j.tics.2013.09.001. URL http://www. pyoudeyer.com/TICSCuriosity2013.pdf . · DOI · Paper page
  24. Graves, Alex. Generating sequences with recurrent neural networks. ArXiv preprint , 2013. URL https: //arxiv.org/abs/1308.0850 .
  25. Ha, D. and Eck, D. A neural representation of sketch drawings. ArXiv preprint , April 2017. URL https://magenta.tensorflow.org/ sketch-rnn-demo .
  26. Ha, D., Dai, A., and Le, Q. Hypernetworks. ArXiv preprint , September 2016. URL https://arxiv.org/abs/ 1609.09106 .
  27. Hansen, N. The cma evolution strategy: A tutorial. ArXiv preprint , 2016. URL https://arxiv.org/abs/ 1604.00772 .
  28. Hausknecht, M., Lehman, J., Miikkulainen, R., and Stone, P. A neuroevolution approach to general atari game playing. IEEE Transactions on Computational Intelligence and AI in Games , 2013. URL http://www.cs.utexas. edu/˜ai-lab/?atari .
  29. Hein, D., Depeweg, S., Tokic, M., Udluft, S., Hentschel, A., Runkler, T., and Sterzing, V. A benchmark environment motivated by industrial control problems. ArXiv preprint , September 2017. URL https://arxiv.org/abs/ 1709.09480 .
  30. Higgins, I., Pal, A., Rusu, A., Matthey, L., Burgess, C., Pritzel, A., Botvinick, M., Blundell, C., and Lerchner, A. Darla: Improving zero-shot transfer in reinforcement learning. ArXiv e-prints , July 2017. URL https:// arxiv.org/abs/1707.08475 . · arXiv
  31. Keller, GeorgB., Bonhoeffer, Tobias, and Hbener, Mark. Sensorimotor mismatch signals in primary visual cortex of the behaving mouse. Neuron , 74(5):809 -815, 2012. ISSN 0896-6273. doi: https://doi.org/10.1016/j.neuron.2012.03.040. URL http://www.sciencedirect.com/science/ article/pii/S0896627312003844 . · DOI · Paper page
  32. Kelley, H. J. Gradient theory of optimal flight paths. ARS Journal , 30(10):947-954, 1960.
  33. Kempka, Michael, Wydmuch, Marek, Runc, Grzegorz, Toczek, Jakub, and Jaskowski, Wojciech. Vizdoom: A doom-based ai research platform for visual reinforcement learning. In IEEE Conference on Computational Intelligence and Games , pp. 341-348, Santorini, Greece, Sep 2016. IEEE. URL http://arxiv.org/abs/1605. 02097 . The best paper award.
  34. Kingma, D. and Welling, M. Auto-encoding variational bayes. ArXiv preprint , 2013. URL https://arxiv. org/abs/1312.6114 .
  35. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.and Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., and Hadsell, R. Overcoming catastrophic forgetting in neural networks. ArXiv preprint , December 2016. URL https://arxiv.org/abs/1612. 00796 .
  36. Koutnik, J., Cuccu, G., Schmidhuber, J., and Gomez, F. Evolving large-scale neural networks for vision-based reinforcement learning. Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation , pp. 1061-1068, 2013. doi: 10.1145/2463372. 2463509. URL http://people.idsia.ch/ ˜juergen/compressednetworksearch.html . · DOI
  37. Lehman, Joel and Stanley, Kenneth. Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation , 19(2):189-223, 2011. ISSN 1063-6560. URL http://eplex.cs.ucf.edu/ noveltysearch/userspage/ .
  38. Leinweber, Marcus, Ward, Daniel R., Sobczak, Jan M., Attinger, Alexander, and Keller, Georg B. A sensorimotor circuit in mouse cortex for visual flow predictions. Neuron , 95(6):1420 - 1432.e5, 2017. ISSN 08966273. doi: https://doi.org/10.1016/j.neuron.2017.08. 036. URL http://www.sciencedirect.com/ science/article/pii/S0896627317307791 . · DOI
  39. Linnainmaa, S. The representation of the cumulative rounding error of an algorithm as a taylor expansion of the local rounding errors. Master's thesis, Univ. Helsinki, 1970.
  40. Matthew Guzdial, Boyang Li, Mark O. Riedl. Game engine learning from video. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17 , pp. 3707-3713, 2017. doi: 10.24963/ijcai.2017/518. URL https://doi.org/ 10.24963/ijcai.2017/518 . · DOI
  41. McAllister, R. and Rasmussen, C. Data-efficient reinforcement learning in continuous-state pomdps. ArXiv preprint , February 2016. URL https://arxiv.org/abs/ 1602.02523 .
  42. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. Playing atari with deep reinforcement learning. ArXiv preprint , December 2013. URL https://arxiv.org/abs/ 1312.5602 .
  43. Mobbs, Dean, Hagan, Cindy C., Dalgleish, Tim, Silston, Brian, and Prvost, Charlotte. The ecology of human fear: survival optimization and the nervous system., 2015. URL https://www.frontiersin.org/ article/10.3389/fnins.2015.00055 . · DOI · Paper page
  44. Munro, P. W. A dual back-propagation scheme for scalar reinforcement learning. Proceedings of the Ninth Annual Conference of the Cognitive Science Society, Seattle, WA , pp. 165-176, 1987.
  45. Nagabandi, A., Kahn, G., Fearing, R., and Levine, S. Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning. ArXiv preprint , August 2017. URL https://arxiv.org/abs/1708. 02596 .
  46. Nguyen, N. and Widrow, B. The truck backer-upper: An example of self learning in neural networks. In Proceedings of the International Joint Conference on Neural Networks , pp. 357-363. IEEE Press, 1989.
  47. Nortmann, Nora, Rekauzke, Sascha, Onat, Selim, Knig, Peter, and Jancke, Dirk. Primary visual cortex represents the difference between past and present. Cerebral Cortex , 25(6):1427-1440, 2015. doi: 10.1093/cercor/ bht318. URL http://dx.doi.org/10.1093/ cercor/bht318 . · DOI
  48. Oh, J., Guo, X., Lee, H., Lewis, R., and Singh, S. Actionconditional video prediction using deep networks in atari games. ArXiv preprint , July 2015. URL https:// arxiv.org/abs/1507.08750 . · arXiv
  49. Oudeyer, P., Kaplan, F., and Hafner, V. Intrinsic motivation systems for autonomous mental development. Trans. Evol. Comp , apr 2007. doi: 10.1109/TEVC.2006.890271. URL http://www.pyoudeyer.com/ims.pdf . · DOI · Paper page
  50. Parker, M. and Bryant, B. Neuro-visual control in the quake ii environment. IEEE Transactions on Computational Intelligence and AI in Games , 2012. URL https://www.cse.unr.edu/˜bdbryant/ papers/parker-2012-tciaig.pdf .
  51. Pathak, D., Agrawal, P., A., Efros, and Darrell, T. Curiositydriven exploration by self-supervised prediction. ArXiv preprint , May 2017. URL https://pathak22. github.io/noreward-rl/ .
  52. Pi, H., Hangya, B., Kvitsiani, D., Sanders, J., Huang, Z., and Kepecs, A. Cortical interneurons that specialize in disinhibitory control. Nature , November 2013. doi: 10.1038/nature12676. URL http://dx.doi.org/ 10.1038/nature12676 . · DOI · Paper page
  53. Quiroga, R., Reddy, L., Kreiman, G., Koch, C., and Fried, I. Invariant visual representation by single neurons in the human brain. Nature , 2005. doi: 10.1038/nature03687. URL http://www.nature.com/nature/journal/ v435/n7045/abs/nature03687.html . · DOI · Paper page
  54. Ratcliff, Rodney Mark. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review , 97 2:285-308, 1990.
  55. Rezende, D., Mohamed, S., and Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. ArXiv preprint , 2014. URL https: //arxiv.org/abs/1401.4082 .
  56. Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. Evolution strategies as a scalable alternative to reinforcement learning. ArXiv preprint , 2017. URL https://arxiv.org/abs/1703.03864 . · arXiv
  57. Schmidhuber, J. An on-line algorithm for dynamic reinforcement learning and planning in reactive environments. 1990 IJCNN International Joint Conference on Neural Networks , pp. 253-258 vol.2, June 1990b. doi: 10.1109/IJCNN.1990.137723. URL ftp://ftp. idsia.ch/pub/juergen/ijcnn90.ps.gz . · DOI
  58. Schmidhuber, J. A possibility for implementing curiosity and boredom in model-building neural controllers. Proceedings of the First International Conference on Simulation of Adaptive Behavior on From Animals to Animats , pp. 222-227, 1990c. URL ftp://ftp.idsia.ch/ pub/juergen/curiositysab.pdf .
  59. Schmidhuber, J. Reinforcement learning in markovian and non-markovian environments. Advances in Neural Information Processing Systems 3 , pp. 500-506, 1991a. URL https://goo.gl/ij1uYQ .
  60. Schmidhuber, J. Curious model-building control systems. In Proc. International Joint Conference on Neural Networks, Singapore , pp. 1458-1463, 1991b.
  61. Schmidhuber, J. Learning complex, extended sequences using the principle of history compression. Neural Computation , 4(2):234-242, 1992. (Based on TR FKI-148-91, TUM, 1991).
  62. Schmidhuber, J. Optimal ordered problem solver. ArXiv preprint , July 2002. URL https://arxiv.org/ abs/cs/0207097 . · arXiv
  63. Schmidhuber, J. Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science , 18(2):173-187, 2006.
  64. Schmidhuber, J. Formal theory of creativity, fun, and intrinsic motivation (1990-2010). IEEE Trans. Autonomous Mental Development , 2010. URL http://people. idsia.ch/˜juergen/creativity.html .
  65. Schmidhuber, J. Powerplay: Training an increasingly general problem solver by continually searching for the simplest still unsolvable problem. Frontiers in Psychology , 4:313, 2013. ISSN 1664-1078. doi: 10.3389/fpsyg.2013. 00313. URL https://www.frontiersin.org/ article/10.3389/fpsyg.2013.00313 . · DOI
  66. Schmidhuber, J. On learning to think: Algorithmic information theory for novel combinations of reinforcement learning controllers and recurrent neural world models. ArXiv preprint , 2015a. URL https://arxiv.org/ abs/1511.09249 . · arXiv
  67. Schmidhuber, J. Deep learning in neural networks: An overview. Neural Networks , 61:85-117, 2015b. doi: 10.1016/j.neunet.2014.09.003. Published online 2014; based on TR arXiv:1404.7828 [cs.NE]. · DOI · Paper page
  68. Schmidhuber, J. One big net for everything. Preprint arXiv:1802.08864 [cs.AI] , February 2018. URL https: //arxiv.org/abs/1802.08864 . · arXiv
  69. Schmidhuber, J. and Huber, R. Learning to generate artificial fovea trajectories for target detection. International Journal of Neural Systems , 2(1-2):125-134, 1991. doi: 10.1142/S012906579100011X. URL ftp://ftp. idsia.ch/pub/juergen/attention.pdf . · DOI
  70. Sehnke, F., Osendorfer, C., Ruckstieb, T., Graves, A., Peters, J., and Schmidhuber, J. Parameter-exploring policy gradients. Neural Networks , 23(4):551559, 2010. doi: 10.1016/j.neunet.2009.12.004. URL http://citeseerx.ist.psu. edu/viewdoc/download;jsessionid= A64D1AE8313A364B814998E9E245B40A? doi=10.1.1.180.7104&rep=rep1&type=pdf . · DOI
  71. Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. ArXiv preprint , January 2017. URL https://arxiv. org/abs/1701.06538 .
  72. Silver, D., van Hasselt, H., Hessel, M., Schaul, T., Guez, A., Harley, T., Dulac-Arnold, G., Reichert, D., Rabinowitz, N., Barreto, A., and Degris, T. The predictron: Endto-end learning and planning. ArXiv preprint , December 2016. URL https://arxiv.org/abs/1612. 08810 .
  73. Srivastava, R., Steunebrink, B., and Schmidhuber, J. First experiments with powerplay. ArXiv preprint , October 2012. URL https://arxiv.org/abs/1210.8385 . · arXiv
  74. Stanley, Kenneth O. and Miikkulainen, Risto. Evolving neural networks through augmenting topologies. Evolutionary Computation , 10(2):99-127, 2002. URL http: //nn.cs.utexas.edu/?stanley:ec02 .
  75. Suarez, Joseph. Language modeling with recurrent highway hypernetworks. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30 , pp. 3269-3278. Curran Associates, Inc., 2017. URL https://goo.gl/4nqHXw .
  76. Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., and Fergus, R. Intrinsic motivation and automatic curricula via asymmetric self-play. ArXiv preprint , October 2017. URL https://arxiv.org/abs/1703. 05407 .
  77. Sutton, Richard S. and Barto, Andrew G. Introduction to Reinforcement Learning . MIT Press, Cambridge, MA, USA, 1st edition, 1998. ISBN 0262193981. URL http://ufal.mff.cuni. cz/˜straka/courses/npfl114/2016/ sutton-bookdraft2016sep.pdf . http://ufal.mff.cuni.cz/~straka/courses/npfl114/2016/sutton-bookdraft2016sep.pdf
  78. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A, Kaiser, L., and Polosukhin, I. Attention is all you need. ArXiv preprint , June 2017. URL https: //arxiv.org/abs/1706.03762 .
  79. Wahlstrm, N., Schn, T., and Deisenroth, M. Learning deep dynamical models from image pixels. ArXiv preprint , October 2014. URL https://arxiv.org/abs/ 1410.7550 .
  80. Wahlstrm, N., Schn, T., and Deisenroth, M. From pixels to torques: Policy learning with deep dynamical models. ArXiv preprint , June 2015. URL https://arxiv. org/abs/1502.02251 .
  81. Watter, M., Springenberg, J., Boedecker, J., and Riedmiller, M. Embed to control: A locally linear latent dynamics model for control from raw images. ArXiv preprint , June 2015. URL https://arxiv.org/abs/1506. 07365 .
  82. Watters, N., Tacchetti, A., Weber, T., Pascanu, R., Battaglia, P., and Zoran, D. Visual interaction networks. ArXiv preprint , June 2017. URL https://arxiv.org/ abs/1706.01433 . · arXiv
  83. Weber, T., Racanire, S., Reichert, D., Buesing, L., Guez, A., Rezende, D., Badia, A., Vinyals, O., Heess, N., Li, Y., Pascanu, R., Battaglia, P., Silver, D., and Wierstra, D. Imagination-augmented agents for deep reinforcement learning. ArXiv preprint , July 2017. URL https:// arxiv.org/abs/1707.06203 . · arXiv
  84. Werbos, P. J. Learning how the world works: Specifications for predictive networks in robots and brains. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, N.Y. , 1987.
  85. Werbos, P. J. Neural networks for control and system identification. In Proceedings of IEEE/CDC Tampa, Florida , 1989.
  86. Werbos, Paul J. Applications of advances in nonlinear sensitivity analysis. In System modeling and optimization , pp. 762-770. Springer, 1982.
  87. Wiering, Marco and van Otterlo, Martijn. Reinforcement Learning . Springer, 2012.

Cited by

Sorted by citations

Authors
Ha, D., & Schmidhuber, J.
Affiliations
  • Google Brain
  • NNAISENSE
ORCID
Venue
Zenodo
Publisher
Zenodo
Published
2018-03-28
ISSN
2331-8422 (ISSN-L)
DOI
10.5281/zenodo.1207631
OpenAlex
W2795843265
Cited by
187
References
4
Citations by year
2026: 25 · 2025: 17 · 2024: 18 · 2023: 23 · 2022: 18 · 2021: 31 · 2020: 33 · 2019: 17 · 2018: 5
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
  • Zenodo (CERN European Organization for Nuclear Research) (cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
  • Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Database Systems and Queries
  • Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
Keywords
procedural videos, deterministic decoding, Rust implementation, digital twin, scene graph, neural radiance fields, event logs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review