PreprintCited by 342Explainable Artificial Intelligence (XAI)Neural Networks and ApplicationsStochastic Gradient Optimization Techniques

The DOI matches its DataCite record, with a small difference in title or year.

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability

Open access

Raghu, Maithra, Gilmer, Justin, Yosinski, Jason, Sohl-Dickstein, Jascha·2017·arXiv

DOI 10.48550/arxiv.1706.05806arXiv 1706.05806

Open this paper in CoChat — free account Open PDF

Abstract

We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less. Code: https://github.com/google/svcca/

Integrity signals

Registry

DataCite · 10.48550/arxiv.1706.05806

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Still being checked.

Sorted by citations

A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
Transfusion: Understanding Transfer Learning for Medical ImagingPreprint
Raghu, M., Zhang, C., Kleinberg, J. M., & Samy Bengio, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07208 · Cited by 901
Self-citation
The Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591
The need for uncertainty quantification in machine-assisted medical decision making
Begoli, E., Bhattacharya, T., & Kusnezov, D. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0004-1 · Cited by 597
Explainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., van Gerven, M. A. J., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 551
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLPreprint
Raghu, A., Raghu, M., Samy Bengio, A., & Vinyals, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09157 · Cited by 409
Self-citation
Multi-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346
High-Performance Large-Scale Image Recognition Without NormalizationPreprint
Brock, A., De, S., Smith, S. L., & Simonyan, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06171 · Cited by 394
Zoom In: An Introduction to Circuits
Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., & Carter, S. (2020) · Distill · doi:10.23915/distill.00024.001 · Cited by 288
Layer-Wise Analysis of a Self-Supervised Speech Representation Model
Pasad, A., Chou, J.-C., & Livescu, K. (2021) · 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru51503.2021.9688093 · Cited by 262
From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers
Lauscher, A., Ravishankar, V., Vulić, I., & Glavaš, G. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.363 · Cited by 283
A Survey on Canonical Correlation Analysis
Yang, X., Liu, W., Liu, W., & Tao, D. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2958342 · Cited by 209
Emerging Cross-lingual Structure in Pretrained Language Models
Conneau, A., Wu, S., Li, H., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.536 · Cited by 253
Separability and geometry of object manifolds in deep neural networks
Cohen, U., Chung, S., Lee, D. D., & Sompolinsky, H. (2020) · Nature Communications · doi:10.1038/s41467-020-14578-5 · Cited by 201
Intrinsic dimension of data representations in deep neural networksPreprint
Ansuini, A., Laio, A., Macke, J. H., & Zoccolan, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12784 · Cited by 205
Review Study of Interpretation Methods for Future Interpretable Machine Learning
Mi, J., Li, A., & Zhou, L. (2020) · IEEE Access · doi:10.1109/access.2020.3032756 · Cited by 168
Transfer learning in medical image segmentation: New insights from analysis of the dynamics of model parameters and learned representations
Karimi, D., Warfield, S. K., & Gholipour, A. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102078 · Cited by 181
Classification of Explainable Artificial Intelligence Methods through Their Output Formats
Vilone, G., & Longo, L. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030032 · Cited by 182
Generative Feature Replay For Class-Incremental Learning
Liu, X., Wu, C., Menta, M., Herranz, L., Raducanu, B., Bagdanov, A. D., Jui, S., & van de Weijer, J. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00121 · Cited by 142
A Survey of Deep Learning for Scientific DiscoveryPreprint
Raghu, M., & Schmidt, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11755 · Cited by 221
Self-citation
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives
Voita, E., Sennrich, R., & Titov, I. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1448 · Cited by 192
On Learning the Geodesic Path for Incremental Learning
Simon, C., Koniusz, P., & Harandi, M. T. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00164 · Cited by 132
Optimal anticipatory control as a theory of motor preparation: A thalamo-cortical circuit model
Kao, T.-C., Sadabadi, M. S., & Hennequin, G. (2021) · Neuron · doi:10.1016/j.neuron.2021.03.009 · Cited by 139
Analyzing biological and artificial neural networks: challenges with opportunities for synergy?
Barrett, D. G. T., Morcos, A. S., & Macke, J. H. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.01.007 · Cited by 122
Do Vision Transformers See Like Convolutional Neural Networks?
Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., & Dosovitskiy, A. (2021) · arXiv · doi:10.48550/arxiv.2108.08810 · Cited by 663
Self-citation
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
Räuker, T., Ho, A., Casper, S., & Hadfield-Menell, D. (2023) · 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00039 · Cited by 122
Comparative analysis of molecular fingerprints in prediction of drug combination effects
Zagidullin, B., Wang, Z., Guan, Y., Pitkänen, E., & Tang, J. P. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab291 · Cited by 110
Assessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107
A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping
Michaels, J. A., Schaffelhofer, S., Agudelo-Toro, A., & Scherberger, H. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2005087117 · Cited by 103
Investigating Multilingual NMT Representations at Scale
Kudugunta, S., Bapna, A., Caswell, I. R., & Fırat, O. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1167 · Cited by 103
Rethinking pre-training on medical imaging
Wen, Y., Chen, L., Deng, Y., & Zhou, C. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103145 · Cited by 92
Explainable Artificial Intelligence: a Systematic Review
Vilone, G., & Longo, L. (2020) · arXiv · doi:10.48550/arxiv.2006.00093 · Cited by 269
A Systematic Review of Transformer-Based Pre-Trained Language Models through Self-Supervised Learning
Kotei, E., & Thirunavukarasu, R. (2023) · Information · doi:10.3390/info14030187 · Cited by 89
On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment
Wang, Z., Lipton, Z. C., & Tsvetkov, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.359 · Cited by 116
Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning
Lee, S., Zhang, T., & Avestimehr, A. S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26023 · Cited by 84
Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks
Bansal, T., Jha, R., Munkhdalai, T., & McCallum, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.38 · Cited by 87
BERT is Not an Interlingua and the Bias of Tokenization
Singh, J., McCann, B., Socher, R., & Xiong, C. (2019) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/d19-6106 · Cited by 79
Universality and individuality in neural dynamics across large populations of recurrent networks
Maheswaranathan, N., Williams, A. H., Golub, M. D., Ganguli, S., & Sussillo, D. (2019) · PubMed · doi:10.48550/arxiv.1907.08549 · Cited by 103
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task
Tang, Y., Pino, J., Li, X., Wang, C., & Genzel, D. (2021) · doi:10.18653/v1/2021.acl-long.328 · Cited by 71
Quantifying the separability of data classes in neural networks
Schilling, A., Maier, A., Gerum, R. C., Metzner, C., & Krauß, P. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.035 · Cited by 76
RePr: Improved Training of Convolutional Filters
Prakash, A., Storer, J. A., Florencio, D. A. F., & Zhang, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01092 · Cited by 64
Analyzing Redundancy in Pretrained Transformer Models
Dalvi, F., Sajjad, H., Durrani, N., & Belinkov, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.398 · Cited by 97
Evaluating Feature Importance EstimatesPreprint
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · Cited by 60
ModelDiff: testing-based DNN similarity comparison for model reuse detection
Li, Y., Zhang, Z., Liu, B., Yang, Z., & Liu, Y. (2021) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3460319.3464816 · Cited by 64
Interpretability of deep neural networks: A review of methods, classification and hardware
Antamis, T., Drosou, A., Vafeiadis, T., Nizamis, A., Ioannidis, D., & Tzovaras, D. K. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128204 · Cited by 61
Similarity Analysis of Contextual Word Representation Models
Wu, J., Belinkov, Y., Sajjad, H., Durrani, N., Dalvi, F., & Glass, J. R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.422 · Cited by 69
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
Ramasesh, V., Dyer, E., & Raghu, M. (2021) · International Conference on Learning Representations · Cited by 53
Self-citation
Ps and Qs: Quantization-Aware Pruning for Efficient Low Latency Neural Network Inference
Hawks, B., Duarte, J. M., Fraser, N. J., Pappalardo, A., Tran, N. V., & Umuroglu, Y. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.676564 · Cited by 81
Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop
Alishahi, A., Chrupała, G., & Linzen, T. (2019) · Natural Language Engineering · doi:10.1017/s135132491900024x · Cited by 59
Deep model compression for mobile platforms: A survey
Nan, K., Liu, S., Du, J., & Liu, H. (2019) · Tsinghua Science & Technology · doi:10.26599/tst.2018.9010103 · Cited by 48
Ecco: An Open Source Library for the Explainability of Transformer Language Models
Alammar, J. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.30 · Cited by 53
Transfer Learning for Radio Frequency Machine Learning: A Taxonomy and Survey
Wong, L. J., & Michaels, A. J. (2022) · Sensors · doi:10.3390/s22041416 · Cited by 46
Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation
Dankers, V., Lucas, C., & Titov, I. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.252 · Cited by 73
MURAL: Multimodal, Multitask Representations Across Languages
Jain, A., Guo, M., Srinivasan, K., Chen, T., Kudugunta, S., Jia, C., Yang, Y., & Baldridge, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.293 · Cited by 45
YOGA: Adaptive Layer-Wise Model Aggregation for Decentralized Federated Learning
Liu, J., Liu, J., Xu, H., Liao, Y., Wang, Z., & Ma, Q. (2023) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2023.3329005 · Cited by 40
Topic Modeling with Contextualized Word Representation ClustersPreprint
Thompson, L., & Mimno, D. M. (2020) · arXiv.org · doi:10.48550/arxiv.2010.12626 · Cited by 111
A clustered federated learning framework for collaborative fault diagnosis of wind turbines
Zhou, R., Li, Y., & Lin, X. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124532 · Cited by 39
Neuron-level Interpretation of Deep NLP Models: A Survey
Sajjad, H., Durrani, N., & Dalvi, F. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00519 · Cited by 52
Improving Zero-Shot Translation by Disentangling Positional Information
Liu, D., Niehues, J., Cross, J., Guzmán, F., & Li, X. (2021) · doi:10.18653/v1/2021.acl-long.101 · Cited by 41
META-EEG: Meta-learning-based class-relevant EEG representation learning for zero-calibration brain–computer interfaces
Han, J.-W., Bak, S., Kim, J.-M., Choi, W., Shin, D.-H., Son, Y.-H., & Kam, T. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121986 · Cited by 36
What Do Self-Supervised Speech Models Know About Words?
Pasad, A., Chien, C.-M., Settle, S., & Livescu, K. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00656 · Cited by 54
Similarity Analysis of Self-Supervised Speech Representations
Chung, Y.-A., Belinkov, Y., & Glass, J. R. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414321 · Cited by 37
Open Questions about Generative Adversarial Networks
Odena, A. (2019) · Distill · doi:10.23915/distill.00018 · Cited by 37
Measuring Dependence with Matrix-based Entropy Functional
Yu, S., Alesiani, F., Yu, X., Jenssen, R., & Príncipe, J. C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17288 · Cited by 32
Distributionally robust chance constrained programming with generative adversarial networks (GANs)
Zhao, S., & You, F. (2020) · AIChE Journal · doi:10.1002/aic.16963 · Cited by 32
Pretrained Language Model Embryology: The Birth of ALBERT
Chiang, C.-H., Huang, S.-F., & Lee, H.-Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.553 · Cited by 42
Orthogonal Model Division Multiple Access
Liang, H., Liu, K., Liu, X., Jiang, H., Dong, C., Xu, X., Niu, K., & Zhang, P. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3384421 · Cited by 34
Neural network interpretation using descrambler groups
Amey, J. L., Keeley, J., Choudhury, T., & Kuprov, I. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016917118 · Cited by 30
One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking
Chen, M., Fu, J., & Ling, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01626 · Cited by 30
The Geometry of Multilingual Language Model Representations
Chang, T. A., Tu, Z., & Bergen, B. K. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.9 · Cited by 60
The Platonic Representation Hypothesis
Huh, M., Cheung, B., Wang, T., & Isola, P. (2024) · arXiv · doi:10.48550/arxiv.2405.07987 · Cited by 35
A Generic Layer Pruning Method for Signal Modulation Recognition Deep Learning Models
Lu, Y., Zhu, Y., Li, Y., Xu, D., Lin, Y., Xuan, Q., & Yang, X. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3520958 · Cited by 32
On the geometry of generalization and memorization in deep neural networksPreprint
Stephenson, C., Padhy, S., Ganesh, A., Hui, Y., Tang, H., & Chung, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.14602 · Cited by 58
Agree to Disagree: When Deep Learning Models With Identical Architectures Produce Distinct Explanations
Watson, M., Hasan, B. A. S., & Moubayed, N. A. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00159 · Cited by 28
On Self-Supervised Multimodal Representation Learning: An Application To Alzheimer’s Disease
Fedorov, A., Wu, L., Sylvain, T., Luck, M. M., DeRamus, T. P., Bleklov, D., Plis, S., & Calhoun, V. D. (2021) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi48211.2021.9434103 · Cited by 25
The Shape of Data: Intrinsic Distance for Data DistributionsPreprint
Tsitsulin, A., Munkhoeva, M., Mottin, D., Karras, P., Bronstein, A. M., Oseledets, I. V., & Müller, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11141 · Cited by 26
A Gradient Flow Framework For Analyzing Network PruningPreprint
Lubana, E. S., & Dick, R. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.11839 · Cited by 26
Ptolemy: Architecture Support for Robust Deep Learning
Gan, Y., Qiu, Y., Leng, J., Guo, M., & Zhu, Y. (2020) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro50266.2020.00031 · Cited by 25
Building an adaptive interface via unsupervised tracking of latent manifolds
Rizzoglio, F., Casadio, M., Santis, D. D., & Mussa-Ivaldi, F. A. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.01.009 · Cited by 25
Fed2A: Federated Learning Mechanism in Asynchronous and Adaptive Modes
Liu, S., Chen, Q., & You, L. (2022) · Electronics · doi:10.3390/electronics11091393 · Cited by 24
Achieving small-batch accuracy with large-batch scalability via Hessian-aware learning rate adjustment
Lee, S., He, C., & Avestimehr, S. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.11.007 · Cited by 25
Teacher Guided Architecture Search
Bashivan, P., Tensen, M., & DiCarlo, J. J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00542 · Cited by 23
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
Augenstein, I., Gella, S., Ruder, S., Kann, K., Can, B., Conneau, A., Welbl, J., Ren, X., & Rei, M. (2019) · doi:10.18653/v1/w19-43 · Cited by 24
Investigating Learning Dynamics of BERT Fine-Tuning
Hao, Y., Dong, L., Wei, F., & Xu, K. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2020.aacl-main.11 · Cited by 25
Deep multimodal fusion for 3D mineral prospectivity modeling: Integration of geological models and simulation data via canonical-correlated joint fusion networks
Zheng, Y., Deng, H., Wu, J., Xie, S., Li, X., Chen, Y., Li, N., Xiao, K., Pfeifer, N., & Mao, X. (2024) · Computers & Geosciences · doi:10.1016/j.cageo.2024.105618 · Cited by 22
Duality Diagram Similarity: A Generic Framework for Initialization Selection in Task Transfer Learning
Dwivedi, K., Huang, J., Cichy, R. M., & Roig, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_30 · Cited by 25
Correcting for experiment-specific variability in expression compendia can remove underlying signals
Lee, A. J., Park, Y., Doing, G., Hogan, D. A., & Greene, C. S. (2020) · GigaScience · doi:10.1093/gigascience/giaa117 · Cited by 22
Explainable Clinical Decision Support from Text
Feng, J., Shaib, C., & Rudzicz, F. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.115 · Cited by 23
A Comprehensive Survey of Multilingual Neural Machine TranslationPreprint
Dabre, R., Chu, C.-H., & Kunchukuttan, A. (2020) · arXiv.org · doi:10.48550/arxiv.2001.01115 · Cited by 36
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop
(2021) · doi:10.18653/v1/2021.acl-srw · Cited by 22
DirectQE: Direct Pretraining for Machine Translation Quality Estimation
Cui, Q., Huang, S., Li, J., Geng, X., Zheng, Z., Huang, G., & Chen, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17506 · Cited by 21
Stitchable Neural Networks
Pan, Z., Cai, J., & Zhuang, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01545 · Cited by 21
Emerging Cross-lingual Structure in Pretrained Language ModelsPreprint
Wu, S., Conneau, A., Li, H., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.01464 · Cited by 21
Emergence of Separable Manifolds in Deep Language RepresentationsPreprint
Mamou, J., Le, H., Rio, M. D., Stephenson, C., Tang, H., Kim, Y., & Chung, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01095 · Cited by 25
Automatic Layer Freezing for Communication Efficiency in Cross-Device Federated Learning
Malan, E., Peluso, V., Calimera, A., Macii, E., & Montuschi, P. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3309691 · Cited by 20
DeepBase
Sellam, T., Lin, K., Huang, I., Yang, M. Y., Vondrick, C., & Wu, E. (2019) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3299869.3300073 · Cited by 20
Understanding community structure in layered neural networks
Watanabe, C., Hiramatsu, K., & Kashino, K. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.08.020 · Cited by 24
COALA: Co-Aligned Autoencoders for Learning Semantically Enriched Audio Representations
Favory, X., Drossos, K., Virtanen, T., & Serra, X. (2020) · arXiv · doi:10.48550/arxiv.2006.08386 · Cited by 20
Anatomy of Catastrophic Forgetting: Hidden Representations and Task SemanticsPreprint
Ramasesh, V., Dyer, E., & Raghu, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07400 · Cited by 20
Self-citation

Authors: Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J.
Affiliations: Google (United States) (US)
Uber AI (United States) (US)
ORCID: Gilmer, Justin
Venue: arXiv
Publisher: arXiv
Published: 2017
Volume: 30
Pages: 6076-6085
ISSN: 2331-8422 (ISSN-L)
DOI: 10.48550/arxiv.1706.05806
arXiv: 1706.05806
OpenAlex: W2767204723
Cited by: 335
References: 0
Citations by year: 2026: 4 · 2025: 13 · 2024: 33 · 2023: 55 · 2022: 40 · 2021: 77 · 2020: 65 · 2019: 41 · 2018: 7
Type: posted content
Language: en
Open access: green
arxiv.org
Licence: arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations: arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
neural information processing systems
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords: representation comparison, affine invariance, overparameterization, learning dynamics, training dynamics, interpretability, training protocols, overfitting

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks