The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
FiLM: Visual Reasoning with a General Conditioning Layer
Open accessAbstract
We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation. FiLM layers influence neural network computation via a simple, feature-wise affine transformation based on conditioning information. We show that FiLM layers are highly effective for visual reasoning - answering image-related questions which require a multi-step, high-level process - a task which has proven difficult for standard deep learning methods that do not explicitly model reasoning. Specifically, we show on visual reasoning tasks that FiLM layers 1) halve state-of-the-art error for the CLEVR benchmark, 2) modulate features in a coherent manner, 3) are robust to ablations and architectural modifications, and 4) generalize well to challenging, new data from few examples or even zero-shot.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“While MI-VECTOR (i.e., diagonal parameters in a MI layer which results in a vector output) corresponds to the most basic implementation of FILM, the original FILM layer uses multiple non-linear layers instead of a single linear transformation in MI-VECTOR which has been shown to improve performance [120].”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [120]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“By appropriately constraining the rank and structure of these parameters, MI recovers HyperNetworks [61] (unconstrained parameters resulting in a matrix output), Feature-wise linear modulation (FiLM) [120, 188] (diagonal parameters resulting in vector output), and Sigmoid units [37] (scalar parameters resulting in scalar output).”
From MultiBench: Multiscale Benchmarks for Multimodal Representation Learning (2021) · cites this paper as [120]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Squeeze-and-Excitation NetworksPreprint
Hu, J., Shen, L., & Sun, G. (2017) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1709.01507 · Cited by 37,845
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
(2008) · Journal of Machine Learning Research · Cited by 33,051
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
(2013) · Cited by 5,726
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Huang, X., & Belongie, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.167 · Cited by 5,076
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850
Hierarchical Mixtures of Experts and the EM Algorithm
Jordan, M. I., & Jacobs, R. A. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.181 · Cited by 2,786
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.670 · Cited by 2,287
Stacked Attention Networks for Image Question Answering
Yang, Z., He, X., Gao, J., Deng, L., & Smola, A. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.10 · Cited by 2,072
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05328 · Cited by 2,032
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781
Hierarchical Question-Image Co-Attention for Visual Question AnsweringPreprint
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,357
A simple neural network module for relational reasoning
Santoro, A., Raposo, D., Barrett, D. G. T., Malinowski, M., Pascanu, R., Battaglia, P., & Lillicrap, T. (2017) · arXiv · doi:10.48550/arxiv.1706.01427 · Cited by 1,281
A Learned Representation For Artistic StylePreprint
Dumoulin, V., Shlens, J., & Kudlur, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.07629 · Cited by 1,114
Self-citationAndreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.12 · Cited by 1,012
VQA: Visual Question AnsweringPreprint
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Batra, D., & Parikh, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00468 · Cited by 968
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017) · arXiv · doi:10.48550/arxiv.1701.06538 · Cited by 933
Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images
Malinowski, M., Rohrbach, M., & Fritz, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.9 · Cited by 632
Learning to Reason: End-to-End Module Networks for Visual Question Answering
Hu, R., Andreas, J., Rohrbach, M., Darrell, T., & Saenko, K. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.93 · Cited by 538
Inferring and Executing Programs for Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L. J. P., Hoffman, J., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.325 · Cited by 502
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Agrawal, A., Summers-Stay, D., Batra, D., & Parikh, D. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1116-0 · Cited by 487
Learning to Compose Neural Networks for Question Answering
Andreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1181 · Cited by 450
A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain InputPreprint
Malinowski, M., & Fritz, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0210 · Cited by 432
FiLM: Visual Reasoning with a General Conditioning LayerPreprint
Perez, E., Strub, F., de Vries, H., Dumoulin, V., & Courville, A. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.07871 · Cited by 432
Self-citationLearning Factored Representations in a Deep Mixture of ExpertsPreprint
Eigen, D., Ranzato, M., & Sutskever, I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.4314 · Cited by 401
Modulating early visual processing by languagePreprint
de Vries, H., Strub, F., Mary, J., Larochelle, H., Pietquin, O., & Courville, A. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.00683 · Cited by 397
Self-citationGuessWhat?! Visual Object Discovery through Multi-modal Dialogue
de Vries, H., Strub, F., Chandar, S., Pietquin, O., Larochelle, H., & Courville, A. C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.475 · Cited by 385
Self-citationTraversing Knowledge Graphs in Vector Space
Guu, K., Miller, J., & Liang, P. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1038 · Cited by 319
Visual Turing test for computer vision systems
Geman, D., Geman, S., Hallonquist, N., & Laurent Younes, B. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1422953112 · Cited by 319
Exploring the structure of a real-time, arbitrary neural artistic stylization network
Ghiasi, G., Lee, H., Kudlur, M., Dumoulin, V., & Shlens, J. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.114 · Cited by 306
Self-citationVisual Interaction NetworksPreprint
Watters, N., Tacchetti, A., Weber, T., Pascanu, R., Battaglia, P., & Zoran, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.01433 · Cited by 267
Bottom-Up and Top-Down Attention for Image Captioning and VQA.Preprint
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2017) · arXiv (Cornell University) · Cited by 252
Arbitrary Style Transfer in Real-time with Adaptive Instance NormalizationPreprint
Huang, X., & Belongie, S. (2017) · arXiv (Cornell University) · Cited by 205
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement LearningPreprint
Oh, J., Singh, S. P., Lee, H., & Kohli, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05064 · Cited by 182
Parallel Multiscale Autoregressive Density EstimationPreprint
Reed, S., van den Oord, A., Kalchbrenner, N., Colmenarejo, S. G., Wang, Z., Belov, D., & de Freitas, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.03664 · Cited by 149
Learning to Reason: End-to-End Module Networks for Visual Question AnsweringPreprint
Hu, R., Andreas, J., Rohrbach, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.05526 · Cited by 109
Learning to Compose Neural Networks for Question AnsweringPreprint
Andreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.01705 · Cited by 100
Dynamic Layer Normalization for Adaptive Neural Acoustic Modeling in Speech Recognition
Kim, T., Song, I., & Bengio, Y. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-556 · Cited by 76
Learning Visual Reasoning Without Strong Priors
Perez, E., de Vries, H., Strub, F., Dumoulin, V., & Courville, A. C. (2017) · HAL · doi:10.48550/arxiv.1707.03017 · Cited by 28
Self-citationHierarchical Mixtures of Experts and the EM Algorithm
Jordan, M. I., & Jacobs, R. A. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/3349.003.0014 · Cited by 10
Inferring and Executing Programs for Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Hoffman, J., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · arXiv · doi:10.48550/arxiv.1705.03633
Exploring the structure of a real-time, arbitrary neural artistic stylization network
Ghiasi, G., Lee, H., Kudlur, M., Dumoulin, V., & Shlens, J. (2017) · arXiv · doi:10.48550/arxiv.1705.06830
Self-citation
Research works printed but not linked: 3
- [1][1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, 'Imagenet classification with deep convolutional neural networks,' in Proc. of NIPS , 2012. -
- [4][4] I. Sutskever, O. Vinyals, and Q. V. Le, 'Sequence to sequence learning with neural networks,' in Proc. of NIPS , 2014. -
- [33][33] S. Reed, A. van den Oord, N. Kalchbrenner, V. Bapst, M. Botvinick, and N. de Freitas, 'Generating interpretable images with controllable structure,' in Proc. of ICLR , 2017.
Cited by
Sorted by citations
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319
Scalable Diffusion Models with Transformers
Peebles, W., & Xie, S. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00387 · Cited by 2,284
Metadata
- Authors
- Perez, E., Strub, F., De Vries, H., Dumoulin, V., & Courville, A.
- Affiliations
- Université de Montréal (CA)
- Centre National de la Recherche Scientifique (FR)
- Mila - Quebec Artificial Intelligence Institute (CA)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-29
- Volume
- 32
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2760103357
- Cited by
- 2,490
- References
- 55
- Citations by year
- 2026: 823 · 2025: 412 · 2024: 280 · 2023: 221 · 2022: 158 · 2021: 244 · 2020: 179 · 2019: 120 · 2018: 52
- FWCI
- 44.92
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Funders
- Nvidia
- Fonds de recherche du Québec – Nature et technologies
- CHIST-ERA
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- cinema, feature-wise linear modulation, visual reasoning, visual question answering, feature modulation, zero-shot generalization, few-shot learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
