Pascanu, Razvan

Overcoming catastrophic forgetting in neural networks

Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences

Domain Adaptation and Few-Shot LearningReinforcement Learning in Robotics

Cited by 7,899

On the difficulty of training recurrent neural networks

Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv

Neural Networks and ApplicationsStochastic Gradient Optimization Techniques

Cited by 4,310

Progressive Neural Networks

Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R. (2016) · arXiv

Reinforcement Learning in RoboticsDomain Adaptation and Few-Shot Learning

Cited by 1,307

A simple neural network module for relational reasoning

Santoro, A., Raposo, D., Barrett, D. G. T., Malinowski, M., Pascanu, R., Battaglia, P., & Lillicrap, T. (2017) · arXiv

Multimodal Machine Learning ApplicationsLogic, Reasoning, and Knowledge

Cited by 1,281

Theano: new features and speed improvements

Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bouchard, N., Warde-Farley, D., & Bengio, Y. (2012) · arXiv

Parallel Computing and Optimization TechniquesNeural Networks and Applications

Cited by 1,005

Interaction Networks for Learning about Objects, Relations and Physics

Battaglia, P. W., Pascanu, R., Lai, M., Rezende, D., & Kavukcuoglu, K. (2016) · arXiv

Model Reduction and Neural NetworksReinforcement Learning in Robotics

Cited by 969

On the Number of Linear Regions of Deep Neural Networks

Montúfar, G., Pascanu, R., Cho, K., & Bengio, Y. (2014) · arXiv

Neural Networks and ApplicationsStochastic Gradient Optimization Techniques

Cited by 840

Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

Dauphin, Y., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., & Bengio, Y. (2014) · arXiv

Stochastic Gradient Optimization TechniquesAdvanced Optimization Algorithms Research

Cited by 827

Embracing Change: Continual Learning in Deep Neural Networks

Hadsell, R., Rao, D., Rusu, A. A., & Pascanu, R. (2020) · Trends in Cognitive Sciences

Domain Adaptation and Few-Shot LearningData Stream Mining Techniques

Cited by 467

Combining modality specific deep neural networks for emotion recognition in video

Kahou, S. E., Pal, C., Bouthillier, X., Froumenty, P., Gülçehre, Ç., Memisevic, R., Vincent, P., Courville, A., Bengio, Y., Ferrari, R. C., et al. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction

Emotion and Mood RecognitionFace recognition and analysis

Cited by 391

Sim-to-Real Robot Learning from Pixels with Progressive Nets

Rusu, A. A., Vecerik, M., Rothörl, T., Heess, N., Pascanu, R., & Hadsell, R. (2016) · arXiv

Reinforcement Learning in RoboticsRobot Manipulation and Learning

Cited by 385

Policy Distillation

Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., & Hadsell, R. (2015) · arXiv

Reinforcement Learning in RoboticsDomain Adaptation and Few-Shot Learning

Cited by 270

Stabilizing Transformers for Reinforcement Learning

Parisotto, E., Song, H. F., Rae, J. W., Pascanu, R., Gulcehre, C., Jayakumar, S. M., Jaderberg, M., Kaufman, R. L., Clark, A., Noury, S., et al. (2019) · arXiv

Reinforcement Learning in RoboticsDomain Adaptation and Few-Shot Learning

Cited by 199

On the number of response regions of deep feed forward networks with piece-wise linear activations

Pascanu, R., Montufar, G., & Bengio, Y. (2013) · arXiv

Neural Networks and ApplicationsStochastic Gradient Optimization Techniques

Cited by 153

Resurrecting Recurrent Neural Networks for Long Sequences

Orvieto, A., Smith, S. L., Gu, A., Fernando, A., Gulcehre, C., Pascanu, R., & De, S. (2023) · arXiv

Neural Networks and ApplicationsTopic Modeling

Cited by 76

BYOL works even without batch statistics

Richemond, P. H., Grill, J.-B., Altché, F., Tallec, C., Strub, F., Brock, A., Smith, S., De, S., Pascanu, R., Piot, B., et al. (2020) · arXiv

Domain Adaptation and Few-Shot LearningAdvanced Neural Network Applications

Cited by 51

Meta-learning of Sequential Strategies

Ortega, P. A., Wang, J. X., Rowland, M., Genewein, T., Kurth-Nelson, Z., Pascanu, R., Heess, N., Veness, J., Pritzel, A., Sprechmann, P., et al. (2019) · arXiv

Reinforcement Learning in RoboticsDomain Adaptation and Few-Shot Learning

Cited by 47

Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?

Tomasev, N., Bica, I., McWilliams, B., Buesing, L., Pascanu, R., Blundell, C., & Mitrovic, J. (2022) · arXiv

Domain Adaptation and Few-Shot LearningAdvanced Neural Network Applications

Cited by 22

Understanding plasticity in neural networks

Lyle, C., Zheng, Z., Nikishin, E., Pires, B. A., Pascanu, R., & Dabney, W. (2023) · arXiv

Reinforcement Learning in RoboticsNeural Networks and Applications

Cited by 11

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

De, S., Smith, S. L., Fernando, A., Botev, A., Cristian-Muraru, G., Gu, A., Haroun, R., Berrada, L., Chen, Y., Srinivasan, S., et al. (2024) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 8

Disentangling the Causes of Plasticity Loss in Neural Networks

Lyle, C., Zheng, Z., Khetarpal, K., van Hasselt, H., Pascanu, R., Martens, J., & Dabney, W. (2024) · arXiv

Neural Networks and ApplicationsDomain Adaptation and Few-Shot Learning

Cited by 5

Uncovering mesa-optimization algorithms in Transformers

von Oswald, J., Schlegel, M., Meulemans, A., Kobayashi, S., Niklasson, E., Zucchet, N., Scherrer, N., Miller, N., Sandler, M., Arcas, B. A. Y., et al. (2023) · arXiv

Topic ModelingDomain Adaptation and Few-Shot Learning

Cited by 5

RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

Botev, A., De, S., Smith, S. L., Fernando, A., Muraru, G.-C., Haroun, R., Berrada, L., Pascanu, R., Sessa, P. G., Dadashi, R., et al. (2024) · arXiv

Topic ModelingNatural Language Processing Techniques

Cited by 5

Deep Reinforcement Learning with Plasticity Injection

Nikishin, E., Oh, J., Ostrovski, G., Lyle, C., Pascanu, R., Dabney, W., & Barreto, A. (2023) · arXiv

Reinforcement Learning in RoboticsDomain Adaptation and Few-Shot Learning

Cited by 4