Herding dynamical weights to learn
Abstract
A new "herding" algorithm is proposed which directly converts observed moments into a sequence of pseudo-samples. The pseudo-samples respect the moment constraints and may be used to estimate (unobserved) quantities of interest. The procedure allows us to sidestep the usual approach of first learning a joint model (which is intractable) and then sampling from that model (which can easily get stuck in a local mode). Moreover, the algorithm is fully deterministic, avoiding random number generation) and does not need expensive operations such as exponentiation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The original definition of herding provides a better approximation of the actual class center compared to a static selection.
“The advantage of the original definition is that it provides a better approximation of the actual class center compared to a static selection.”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [74]
Cites
Information theory and statistical mechanics
Jaynes, E. T. (1957) · Physical Review · doi:10.1103/physrev.106.620 · Cited by 12,931 · see citing works
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 1× · see citing works
“There have been many attempts to approximate this learning problem which have proven useful in a wide range of real world applications (Besag, 1977; Geman & Geman, 1984; Zhu et al., 1997; Lafferty, 1999; Zhu & Liu, 2002; Hinton, 2002; Hyvarinen, 2005; Tieleman, 2008).”
Flagged first, then by citations
Cited by
Sorted by citations
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988 · see citing works
A Strategy for an Uncompromising Incremental Learner
Venkatesan, R., Venkateswara, H., Panchanathan, S., & Li, B. (2017) · arXiv · doi:10.48550/arxiv.1705.00744 · Cited by 37 · see citing works
A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks
Belouadah, E., Popescu, A., & Kanellos, I. (2020) · arXiv · doi:10.48550/arxiv.2011.01844 · Cited by 235 · see citing works
“Introduced in [74] and first used in an IL context by [56], its usefulness was questioned in [14, 29, 37] where it was reported to provide only marginal improvement compared to random selection.”
BackgroundMethod
- Authors
- Welling, M.
- Affiliations
- University of California, Irvine (US)
- Corresponding author
- Welling, Max
- ORCID
- Welling, Max
- Venue
- Proceedings of the 26th Annual International Conference on Machine Learning
- Publisher
- ACM
- Published
- 2009-06-14
- Pages
- 1121-1128
- OpenAlex
- W1997865285
- Cited by
- 401
- References
- 2
- Citations by year
- 2026: 38 · 2025: 62 · 2024: 70 · 2023: 58 · 2022: 38 · 2021: 39 · 2020: 27 · 2019: 7 · 2018: 4 · 2017: 9 · 2016: 11 · 2015: 6 · 2014: 8 · 2013: 7 · 2012: 8
- FWCI
- 17.51
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Funders
- Division of Information and Intelligent Systems (IIS-0447903IIS-0535278)
- Office of Naval Research (00014-06-1-073)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Target Tracking and Data Fusion in Sensor Networks
- Keywords
- moment constraints, pseudo-samples, deterministic sampling, moment matching
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review