The DOI matches its DataCite record; 5 of the 5 references it cites were found in a registry or an index.
Distributed Weight Consolidation: A Brain Segmentation Case Study
Abstract
Collecting the large datasets needed to train deep neural networks can be very difficult, particularly for the many applications for which sharing and pooling data is complicated by practical, ethical, or legal concerns. However, it may be the case that derivative datasets or predictive models developed within individual sites can be shared and combined with fewer restrictions. Training on distributed data and combining the resulting networks is often viewed as continual learning, but these methods require networks to be trained sequentially. In this paper, we introduce distributed weight consolidation (DWC), a continual learning method to consolidate the weights of separate neural networks, each trained on an independent dataset. We evaluated DWC with a brain segmentation case study, where we consolidated dilated convolutional neural networks trained on independent structural magnetic resonance imaging (sMRI) datasets from different sites. We found that DWC led to increased performance on test sets from the different sites, while maintaining generalization performance for a very large and completely independent multi-site dataset, compared to an ensemble baseline.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Orlando al. [669] | et | 2019 | U-Net | Segmentation | OCT scans | 2 | MC dropout”
From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [669]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Keeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994
The ENIGMA Consortium: large-scale collaborative analyses of neuroimaging and genetic data
the Alzheimer’s Disease Neuroimaging Initiative, E. C., Thompson, P. M., Stein, J. L., Medland, S. E., Hibar, D. P., Vasquez, A. A., Renteria, M. E., Toro, R., Jahanshad, N., Schumann, G., et al. (2014) · Brain Imaging and Behavior · doi:10.1007/s11682-013-9269-5 · Cited by 838
Variational Continual Learning
Turner, R., Bui, T., Li, Y., & Cuong, N. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.35471 · Cited by 438
Cited by
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
Si, Y., Du, J., Li, Z., Jiang, X., Miller, T., Wang, F., Jim Zheng, W., & Roberts, K. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103671 · Cited by 235
Federated Gradient Averaging for Multi-Site Training with Momentum-Based Optimizers
Remedios, S. W., Butman, J. A., Landman, B. A., & Pham, D. L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60548-3_17 · Cited by 19
“Bayesian neural networks, in which parameters are not point estimates but distributions of parameters, allow techniques such as distributed weight consolidation [18] to consider multi-site training as a continual learning problem.”
Background
Metadata
- Authors
- McClure, P., Zheng, C. Y., Kaczmarzyk, J. R., Lee, J. A., Ghosh, S. S., Nielson, D., Bandettini, P., & Pereira, F.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- arXiv
- 1805.10863
- References
- 5
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
