Cited by 848Advanced Neural Network ApplicationsNeural Networks and ApplicationsDomain Adaptation and Few-Shot Learning

Dynamic Neural Networks: A Survey

Open access
Han, Yizeng, Huang, Gao, Song, Shiji, Yang, Le, Wang, Honghui, Wang, Yulin·2022·IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

Dynamic neural network is an emerging research topic in deep learning. Compared to static models which have fixed computational graphs and parameters at the inference stage, dynamic networks can adapt their structures or parameters to different inputs, leading to notable advantages in terms of accuracy, computational efficiency, adaptiveness, etc. In this survey, we comprehensively review this rapidly developing area by dividing dynamic networks into three main categories: 1) sample-wise dynamic models that process each sample with data-dependent architectures or parameters; 2) spatial-wise dynamic networks that conduct adaptive computation with respect to different spatial locations of image data; and 3) temporal-wise dynamic models that perform adaptive inference along the temporal dimension for sequential data such as videos and texts. The important research problems of dynamic networks, e.g., architecture design, decision making scheme, optimization technique and applications, are reviewed systematically. Finally, we discuss the open problems in this field together with interesting future research directions.

Integrity signals

Registry

Crossref · 10.1109/tpami.2021.3117837

Cites

212 of 212 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Han, Y., Huang, G., Song, S., Yang, L., Wang, H., & Wang, Y.
Affiliations
  • Tsinghua University (CN)
  • Beijing Academy of Artificial Intelligence (CN)
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2022-11-01
Volume
44
Issue
11
Pages
7436-7456
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
OpenAlex
W3204647170
Cited by
848
References
212
Citations by year
2026: 162 · 2025: 213 · 2024: 191 · 2023: 170 · 2022: 74 · 2021: 36 · 2020: 2
FWCI
7.34
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science and Technology Major Project (2018AAA0100701)
  • National Natural Science Foundation of China (61906106, 62022048)
  • Tsinghua University
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
dynamic neural networks, adaptive computation, adaptive inference, architectural design, decision-making scheme, optimization methods, sequential data, video analysis, text analysis
MeSH terms
Algorithms, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review