SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
Open accessAbstract
We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less. Code: https://github.com/google/svcca/
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
t-SNE and SVCCA play a significant role in directing towards the subspace of input for error-free predictions.
“Several other methods, such as t-distributed stochastic neighbor embedding (t-SNE) and correlation-based singular vector canonical correlation analysis (SVCCA) [48], play a significant role in directing towards the subspace of input for error-free predictions.”
From Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2024) · cites this paper as [48]
Cites
Still being checked.
Cited by
Sorted by citations
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,062 · see citing works
Transfusion: Understanding Transfer Learning for Medical ImagingPreprint
Raghu, M., Zhang, C., Kleinberg, J. M., & Samy Bengio, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07208 · Cited by 845 · see citing works
Self-citationThe Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591 · see citing works
The need for uncertainty quantification in machine-assisted medical decision making
Begoli, E., Bhattacharya, T., & Kusnezov, D. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0004-1 · Cited by 597 · see citing works
Explainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., van Gerven, M. A. J., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 486 · see citing works
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLPreprint
Raghu, A., Raghu, M., Samy Bengio, A., & Vinyals, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09157 · Cited by 409 · see citing works
Self-citationMulti-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346 · see citing works
High-Performance Large-Scale Image Recognition Without NormalizationPreprint
Brock, A., De, S., Smith, S. L., & Simonyan, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06171 · Cited by 356 · see citing works
Zoom In: An Introduction to Circuits
Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., & Carter, S. (2020) · Distill · doi:10.23915/distill.00024.001 · Cited by 288 · see citing works
Layer-Wise Analysis of a Self-Supervised Speech Representation Model
Pasad, A., Chou, J.-C., & Livescu, K. (2021) · 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru51503.2021.9688093 · Cited by 262 · see citing works
From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers
Lauscher, A., Ravishankar, V., Vulić, I., & Glavaš, G. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.363 · Cited by 250 · see citing works
A Survey on Canonical Correlation Analysis
Yang, X., Liu, W., Liu, W., & Tao, D. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2958342 · Cited by 202 · see citing works
Emerging Cross-lingual Structure in Pretrained Language Models
Conneau, A., Wu, S., Li, H., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.536 · Cited by 216 · see citing works
Separability and geometry of object manifolds in deep neural networks
Cohen, U., Chung, S., Lee, D. D., & Sompolinsky, H. (2020) · Nature Communications · doi:10.1038/s41467-020-14578-5 · Cited by 185 · see citing works
Intrinsic dimension of data representations in deep neural networksPreprint
Ansuini, A., Laio, A., Macke, J. H., & Zoccolan, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12784 · Cited by 187 · see citing works
Review Study of Interpretation Methods for Future Interpretable Machine Learning
Mi, J., Li, A., & Zhou, L. (2020) · IEEE Access · doi:10.1109/access.2020.3032756 · Cited by 162 · see citing works
Karimi, D., Warfield, S. K., & Gholipour, A. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102078 · Cited by 181 · see citing works
Classification of Explainable Artificial Intelligence Methods through Their Output Formats
Vilone, G., & Longo, L. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030032 · Cited by 157 · see citing works
Generative Feature Replay For Class-Incremental Learning
Liu, X., Wu, C., Menta, M., Herranz, L., Raducanu, B., Bagdanov, A. D., Jui, S., & van de Weijer, J. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00121 · Cited by 135 · see citing works
A Survey of Deep Learning for Scientific DiscoveryPreprint
Raghu, M., & Schmidt, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11755 · Cited by 174 · see citing works
Self-citationVoita, E., Sennrich, R., & Titov, I. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1448 · Cited by 147 · see citing works
On Learning the Geodesic Path for Incremental Learning
Simon, C., Koniusz, P., & Harandi, M. T. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00164 · Cited by 132 · see citing works
Optimal anticipatory control as a theory of motor preparation: A thalamo-cortical circuit model
Kao, T.-C., Sadabadi, M. S., & Hennequin, G. (2021) · Neuron · doi:10.1016/j.neuron.2021.03.009 · Cited by 139 · see citing works
Analyzing biological and artificial neural networks: challenges with opportunities for synergy?
Barrett, D. G. T., Morcos, A. S., & Macke, J. H. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.01.007 · Cited by 111 · see citing works
Do Vision Transformers See Like Convolutional Neural Networks?
Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., & Dosovitskiy, A. (2021) · arXiv · doi:10.48550/arxiv.2108.08810 · Cited by 663 · see citing works
Self-citationToward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
Räuker, T., Ho, A., Casper, S., & Hadfield-Menell, D. (2023) · 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00039 · Cited by 122 · see citing works
Comparative analysis of molecular fingerprints in prediction of drug combination effects
Zagidullin, B., Wang, Z., Guan, Y., Pitkänen, E., & Tang, J. P. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab291 · Cited by 105 · see citing works
Assessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107 · see citing works
A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping
Michaels, J. A., Schaffelhofer, S., Agudelo-Toro, A., & Scherberger, H. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2005087117 · Cited by 103 · see citing works
Investigating Multilingual NMT Representations at Scale
Kudugunta, S., Bapna, A., Caswell, I. R., & Fırat, O. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1167 · Cited by 96 · see citing works
Rethinking pre-training on medical imaging
Wen, Y., Chen, L., Deng, Y., & Zhou, C. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103145 · Cited by 92 · see citing works
Explainable Artificial Intelligence: a Systematic Review
Vilone, G., & Longo, L. (2020) · arXiv · doi:10.48550/arxiv.2006.00093 · Cited by 269 · see citing works
Kotei, E., & Thirunavukarasu, R. (2023) · Information · doi:10.3390/info14030187 · Cited by 89 · see citing works
On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment
Wang, Z., Lipton, Z. C., & Tsvetkov, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.359 · Cited by 84 · see citing works
Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning
Lee, S., Zhang, T., & Avestimehr, A. S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26023 · Cited by 84 · see citing works
Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks
Bansal, T., Jha, R., Munkhdalai, T., & McCallum, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.38 · Cited by 76 · see citing works
BERT is Not an Interlingua and the Bias of Tokenization
Singh, J., McCann, B., Socher, R., & Xiong, C. (2019) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/d19-6106 · Cited by 74 · see citing works
Universality and individuality in neural dynamics across large populations of recurrent networks
Maheswaranathan, N., Williams, A. H., Golub, M. D., Ganguli, S., & Sussillo, D. (2019) · PubMed · doi:10.48550/arxiv.1907.08549 · Cited by 72 · see citing works
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task
Tang, Y., Pino, J., Li, X., Wang, C., & Genzel, D. (2021) · doi:10.18653/v1/2021.acl-long.328 · Cited by 71 · see citing works
Quantifying the separability of data classes in neural networks
Schilling, A., Maier, A., Gerum, R. C., Metzner, C., & Krauß, P. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.035 · Cited by 68 · see citing works
RePr: Improved Training of Convolutional Filters
Prakash, A., Storer, J. A., Florencio, D. A. F., & Zhang, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01092 · Cited by 64 · see citing works
Analyzing Redundancy in Pretrained Transformer Models
Dalvi, F., Sajjad, H., Durrani, N., & Belinkov, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.398 · Cited by 71 · see citing works
Evaluating Feature Importance EstimatesPreprint
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · Cited by 61 · see citing works
ModelDiff: testing-based DNN similarity comparison for model reuse detection
Li, Y., Zhang, Z., Liu, B., Yang, Z., & Liu, Y. (2021) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3460319.3464816 · Cited by 55 · see citing works
Interpretability of deep neural networks: A review of methods, classification and hardware
Antamis, T., Drosou, A., Vafeiadis, T., Nizamis, A., Ioannidis, D., & Tzovaras, D. K. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128204 · Cited by 55 · see citing works
Similarity Analysis of Contextual Word Representation Models
Wu, J., Belinkov, Y., Sajjad, H., Durrani, N., Dalvi, F., & Glass, J. R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.422 · Cited by 52 · see citing works
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
Ramasesh, V., Dyer, E., & Raghu, M. (2021) · International Conference on Learning Representations · Cited by 53 · see citing works
Self-citationPs and Qs: Quantization-Aware Pruning for Efficient Low Latency Neural Network Inference
Hawks, B., Duarte, J., Fraser, N. J., Pappalardo, A., Tran, N., & Umuroglu, Y. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.676564 · Cited by 53 · see citing works
Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop
Alishahi, A., Chrupała, G., & Linzen, T. (2019) · Natural Language Engineering · doi:10.1017/s135132491900024x · Cited by 53 · see citing works
Deep model compression for mobile platforms: A survey
Nan, K., Liu, S., Du, J., & Liu, H. (2019) · Tsinghua Science & Technology · doi:10.26599/tst.2018.9010103 · Cited by 49 · see citing works
Ecco: An Open Source Library for the Explainability of Transformer Language Models
Alammar, J. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.30 · Cited by 46 · see citing works
Transfer Learning for Radio Frequency Machine Learning: A Taxonomy and Survey
Wong, L. J., & Michaels, A. J. (2022) · Sensors · doi:10.3390/s22041416 · Cited by 45 · see citing works
Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation
Dankers, V., Lucas, C., & Titov, I. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.252 · Cited by 44 · see citing works
MURAL: Multimodal, Multitask Representations Across Languages
Jain, A., Guo, M., Srinivasan, K., Chen, T., Kudugunta, S., Jia, C., Yang, Y., & Baldridge, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.293 · Cited by 43 · see citing works
YOGA: Adaptive Layer-Wise Model Aggregation for Decentralized Federated Learning
Liu, J., Liu, J., Xu, H., Liao, Y., Wang, Z., & Ma, Q. (2023) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2023.3329005 · Cited by 40 · see citing works
Topic Modeling with Contextualized Word Representation ClustersPreprint
Thompson, L., & Mimno, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12626 · Cited by 63 · see citing works
A clustered federated learning framework for collaborative fault diagnosis of wind turbines
Zhou, R., Li, Y., & Lin, X. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124532 · Cited by 39 · see citing works
Neuron-level Interpretation of Deep NLP Models: A Survey
Sajjad, H., Durrani, N., & Dalvi, F. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00519 · Cited by 41 · see citing works
Improving Zero-Shot Translation by Disentangling Positional Information
Liu, D., Niehues, J., Cross, J., Guzmán, F., & Li, X. (2021) · doi:10.18653/v1/2021.acl-long.101 · Cited by 35 · see citing works
Han, J.-W., Bak, S., Kim, J.-M., Choi, W., Shin, D.-H., Son, Y.-H., & Kam, T. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121986 · Cited by 35 · see citing works
What Do Self-Supervised Speech Models Know About Words?
Pasad, A., Chien, C.-M., Settle, S., & Livescu, K. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00656 · Cited by 36 · see citing works
Similarity Analysis of Self-Supervised Speech Representations
Chung, Y.-A., Belinkov, Y., & Glass, J. R. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414321 · Cited by 35 · see citing works
Open Questions about Generative Adversarial Networks
Odena, A. (2019) · Distill · doi:10.23915/distill.00018 · Cited by 35 · see citing works
Measuring Dependence with Matrix-based Entropy Functional
Yu, S., Alesiani, F., Yu, X., Jenssen, R., & Príncipe, J. C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17288 · Cited by 32 · see citing works
Distributionally robust chance constrained programming with generative adversarial networks (GANs)
Zhao, S., & You, F. (2020) · AIChE Journal · doi:10.1002/aic.16963 · Cited by 32 · see citing works
Pretrained Language Model Embryology: The Birth of ALBERT
Chiang, C.-H., Huang, S.-F., & Lee, H.-Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.553 · Cited by 34 · see citing works
Orthogonal Model Division Multiple Access
Liang, H., Liu, K., Liu, X., Jiang, H., Dong, C., Xu, X., Niu, K., & Zhang, P. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3384421 · Cited by 29 · see citing works
Neural network interpretation using descrambler groups
Amey, J. L., Keeley, J., Choudhury, T., & Kuprov, I. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016917118 · Cited by 29 · see citing works
One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking
Chen, M., Fu, J., & Ling, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01626 · Cited by 29 · see citing works
The Geometry of Multilingual Language Model Representations
Chang, T. A., Tu, Z., & Bergen, B. K. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.9 · Cited by 31 · see citing works
The Platonic Representation Hypothesis
Huh, M., Cheung, B., Wang, T., & Isola, P. (2024) · arXiv · doi:10.48550/arxiv.2405.07987 · Cited by 35 · see citing works
A Generic Layer Pruning Method for Signal Modulation Recognition Deep Learning Models
Lu, Y., Zhu, Y., Li, Y., Xu, D., Lin, Y., Xuan, Q., & Yang, X. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3520958 · Cited by 27 · see citing works
On the geometry of generalization and memorization in deep neural networksPreprint
Stephenson, C., Padhy, S., Ganesh, A., Hui, Y., Tang, H., & Chung, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.14602 · Cited by 27 · see citing works
Watson, M., Hasan, B. A. S., & Moubayed, N. A. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00159 · Cited by 27 · see citing works
On Self-Supervised Multimodal Representation Learning: An Application To Alzheimer’s Disease
Fedorov, A., Wu, L., Sylvain, T., Luck, M. M., DeRamus, T. P., Bleklov, D., Plis, S., & Calhoun, V. D. (2021) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi48211.2021.9434103 · Cited by 25 · see citing works
The Shape of Data: Intrinsic Distance for Data DistributionsPreprint
Tsitsulin, A., Munkhoeva, M., Mottin, D., Karras, P., Bronstein, A. M., Oseledets, I. V., & Müller, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11141 · Cited by 26 · see citing works
A Gradient Flow Framework For Analyzing Network PruningPreprint
Lubana, E. S., & Dick, R. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.11839 · Cited by 26 · see citing works
Ptolemy: Architecture Support for Robust Deep Learning
Gan, Y., Qiu, Y., Leng, J., Guo, M., & Zhu, Y. (2020) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro50266.2020.00031 · Cited by 25 · see citing works
Building an adaptive interface via unsupervised tracking of latent manifolds
Rizzoglio, F., Casadio, M., Santis, D. D., & Mussa-Ivaldi, F. A. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.01.009 · Cited by 22 · see citing works
Fed2A: Federated Learning Mechanism in Asynchronous and Adaptive Modes
Liu, S., Chen, Q., & You, L. (2022) · Electronics · doi:10.3390/electronics11091393 · Cited by 24 · see citing works
Lee, S., He, C., & Avestimehr, S. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.11.007 · Cited by 24 · see citing works
Teacher Guided Architecture Search
Bashivan, P., Tensen, M., & DiCarlo, J. J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00542 · Cited by 23 · see citing works
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
Augenstein, I., Gella, S., Ruder, S., Kann, K., Can, B., Conneau, A., Welbl, J., Ren, X., & Rei, M. (2019) · doi:10.18653/v1/w19-43 · Cited by 24 · see citing works
Investigating Learning Dynamics of BERT Fine-Tuning
Hao, Y., Dong, L., Wei, F., & Xu, K. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2020.aacl-main.11 · Cited by 24 · see citing works
Zheng, Y., Deng, H., Wu, J., Xie, S., Li, X., Chen, Y., Li, N., Xiao, K., Pfeifer, N., & Mao, X. (2024) · Computers & Geosciences · doi:10.1016/j.cageo.2024.105618 · Cited by 22 · see citing works
Dwivedi, K., Huang, J., Cichy, R. M., & Roig, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_30 · Cited by 22 · see citing works
Correcting for experiment-specific variability in expression compendia can remove underlying signals
Lee, A. J., Park, Y., Doing, G., Hogan, D. A., & Greene, C. S. (2020) · GigaScience · doi:10.1093/gigascience/giaa117 · Cited by 22 · see citing works
Explainable Clinical Decision Support from Text
Feng, J., Shaib, C., & Rudzicz, F. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.115 · Cited by 22 · see citing works
A Comprehensive Survey of Multilingual Neural Machine TranslationPreprint
Dabre, R., Chu, C.-H., & Kunchukuttan, A. (2020) · arXiv.org · doi:10.48550/arxiv.2001.01115 · Cited by 36 · see citing works
DirectQE: Direct Pretraining for Machine Translation Quality Estimation
Cui, Q., Huang, S., Li, J., Geng, X., Zheng, Z., Huang, G., & Chen, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17506 · Cited by 21 · see citing works
Pan, Z., Cai, J., & Zhuang, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01545 · Cited by 21 · see citing works
Emerging Cross-lingual Structure in Pretrained Language ModelsPreprint
Wu, S., Conneau, A., Li, H., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.01464 · Cited by 21 · see citing works
Emergence of Separable Manifolds in Deep Language RepresentationsPreprint
Mamou, J., Le, H., Rio, M. D., Stephenson, C., Tang, H., Kim, Y., & Chung, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01095 · Cited by 23 · see citing works
Automatic Layer Freezing for Communication Efficiency in Cross-Device Federated Learning
Malan, E., Peluso, V., Calimera, A., Macii, E., & Montuschi, P. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3309691 · Cited by 20 · see citing works
Sellam, T., Lin, K., Huang, I., Yang, M. Y., Vondrick, C., & Wu, E. (2019) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3299869.3300073 · Cited by 20 · see citing works
Understanding community structure in layered neural networks
Watanabe, C., Hiramatsu, K., & Kashino, K. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.08.020 · Cited by 20 · see citing works
COALA: Co-Aligned Autoencoders for Learning Semantically Enriched Audio Representations
Favory, X., Drossos, K., Virtanen, T., & Serra, X. (2020) · arXiv · doi:10.48550/arxiv.2006.08386 · Cited by 20 · see citing works
Anatomy of Catastrophic Forgetting: Hidden Representations and Task SemanticsPreprint
Ramasesh, V., Dyer, E., & Raghu, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07400 · Cited by 20 · see citing works
Self-citation
- Authors
- Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J.
- Affiliations
- Google (United States) (US)
- Uber AI (United States) (US)
- ORCID
- Gilmer, Justin
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- Volume
- 30
- Pages
- 6076-6085
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1706.05806
- OpenAlex
- W2767204723
- Cited by
- 335
- References
- 0
- Citations by year
- 2026: 4 · 2025: 13 · 2024: 33 · 2023: 55 · 2022: 40 · 2021: 77 · 2020: 65 · 2019: 41 · 2018: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- neural information processing systems
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- representation comparison, affine invariance, overparameterization, learning dynamics, training dynamics, interpretability, training protocols, overfitting
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review