The DOI matches its DataCite record.
High Fidelity Neural Audio Compression
Open accessAbstract
We introduce a state-of-the-art real-time, high-fidelity, audio codec leveraging neural networks. It consists in a streaming encoder-decoder architecture with quantized latent space trained in an end-to-end fashion. We simplify and speed-up the training by using a single multiscale spectrogram adversary that efficiently reduces artifacts and produce high-quality samples. We introduce a novel loss balancer mechanism to stabilize training: the weight of a loss now defines the fraction of the overall gradient it should represent, thus decoupling the choice of this hyper-parameter from the typical scale of the loss. Finally, we study how lightweight Transformer models can be used to further compress the obtained representation by up to 40%, while staying faster than real time. We provide a detailed description of the key design choices of the proposed model including: training objective, architectural changes and a study of various perceptual loss functions. We present an extensive subjective evaluation (MUSHRA tests) together with an ablation study for a range of bandwidths and audio domains, including speech, noisy-reverberant speech, and music. Our approach is superior to the baselines methods across all evaluated settings, considering both 24 kHz monophonic and 48 kHz stereophonic audio. Code and models are available at github.com/facebookresearch/encodec.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The most challenging codecs for our non-SSL based detectors were codec-4 (Encodec [26]), 7 (mp3+Encodec) and 10 (speex) with pooled min DCF of 0.675, 0.683 and 0.701.”
From Robust audio deepfake detection: exploring front-/back-end combinations and data augmentation strategies for the ASVspoof5 Challenge (2024) · cites this paper as [26]
Citations
Cites
Still being checked.
Cited by
Sorted by citations
Items 101–200 of 290
SoundSpring: Loss-Resilient Audio Transceiver With Dual-Functional Masked Language Modeling
Yao, S., Dai, J., Qin, X., Wang, S., Wang, S., Niu, K., & Zhang, P. (2025) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2025.3531406 · Cited by 6
A Streamable Neural Audio Codec With Residual Scalar-Vector Quantization for Real-Time Communication
Jiang, X.-H., Ai, Y., Zheng, R.-C., & Ling, Z.-H. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3560172 · Cited by 6
Harmonizing AI-Generated Music: Integrating Symbolic and Audio Models for Text-to-Music Generation
Yan, X., Ma, C., & Hao, Y. (2023) · International Conference on Artificial Intelligence and Intelligent Information Processing (AIIIP) · doi:10.1109/aiiip61647.2023.00030 · Cited by 6
Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations
Jiang, X., Peng, X., Zhang, Y., & Lu, Y. (2024) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2024.3488557 · Cited by 6
End-To-End Streaming Model For Low-Latency Speech Anonymization
Quamer, W., & Gutiérrez‐Osuna, R. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832303 · Cited by 6
Ren, W., Lin, Y., Chou, H.-C., Wu, H., Wu, Y.-C., Lee, C.-C., Lee, H.-Y., Wang, H., & Tsao, Y. (2024) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc63619.2025.10849259 · Cited by 6
Audio Codec Augmentation for Robust Collaborative Watermarking of Speech Synthesis
Juvela, L., & Wang, X. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888976 · Cited by 7
Pang, H., Ding, T., He, L., Tao, M., Zhang, L., & Gan, Q. (2025) · International Conference on Computer Graphics and Virtuality (ICCGV) · doi:10.1117/12.3060395 · Cited by 6
Loss Masking Is Not Needed In Decoder-Only Transformer For Discrete-Token-Based ASR
Chen, Q., Wang, W., Zhang, Q., Zheng, S., Zhang, S. S., Deng, C., Ma, Y., Yu, H., Liu, J., & Zhang, C. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447296 · Cited by 6
Srcodec: Split-Residual Vector Quantization for Neural Speech Codec
Zheng, Y., Tu, W., Li, X., & Xu, X. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445966 · Cited by 6
Regeneration Learning: A Learning Paradigm for Data Generation
Tan, X., Qin, T., Bian, J., Liu, T., & Bengio, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30271 · Cited by 10
NeuralVC: Any-to-Any Voice Conversion Using Neural Networks Decoder for Real-Time Voice Conversion
Cao, D., Zhang, Z., & Zhang, J. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3439469 · Cited by 6
Preference Alignment Improves Language Model-Based TTS
Tian, J., Zhang, C., Shi, J., Zhang, H., Yu, J., Watanabe, S., & Yu, D. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890510 · Cited by 19
Mehta, S., Deichler, A., O’Regan, J., Moëll, B., Beskow, J., Henter, G. E., & Alexanderson, S. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00201 · Cited by 7
An Investigation of Time-Frequency Representation Discriminators for High-Fidelity Vocoders
Gu, Y., Zhang, X., Xue, L., Li, H., & Wu, Z. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3468005 · Cited by 7
Improving Audio Explanations Using Audio Language Models
Akman, A., Sun, Q., & Schuller, B. W. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3532218 · Cited by 8
Zhao, S., Zhou, K., Pan, Z., Ma, Y., Zhang, C., & Ma, B. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888627 · Cited by 5
VMAs: Video-to-Music Generation via Semantic Alignment in Web Music Videos
Lin, Y.-B., Tian, Y., Yang, L., Bertasius, G., & Wang, H. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00120 · Cited by 21
Cross-Modal Multi-Tasking for Speech-to-Text Translation via Hard Parameter Sharing
Yan, B., Chang, X., Anastasopoulos, A., Fujita, Y., & Watanabe, S. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447926 · Cited by 5
A Mel Spectrogram Enhancement Paradigm Based on CWT in Speech Synthesis
Hu, G., Tan, H., & Li, R. (2024) · International Conference on Asian Language Processing · doi:10.1109/ialp63756.2024.10661192 · Cited by 6
Casanova, E., Langman, R., Neekhara, P., Hussain, S., Li, J., Ghosh, S., Jukić, A., & Lee, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888202 · Cited by 5
Latent Watermarking of Audio Generative Models
Roman, R. S., Fernandez, P., Deleforge, A., Adi, Y., & Serizel, R. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889782 · Cited by 12
Self-citationEmotion Recognition ToolKit (ERTK): Standardising Tools For Emotion Recognition Research
Keesing, A., Koh, Y. S., Yogarajan, V., & Witbrock, M. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3613459 · Cited by 5
SelfRemaster: Self-Supervised Speech Restoration for Historical Audio Resources
Saeki, T., Takamichi, S., Nakamura, T., Tanji, N., & Saruwatari, H. (2023) · IEEE Access · doi:10.1109/access.2023.3345027 · Cited by 5
High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models
Qiang, C., Li, H., Tian, Y., Zhao, Y. F., Zhang, Y., Wang, L., & Dang, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448495 · Cited by 5
Efficient Parallel Audio Generation Using Group Masked Language Modeling
Jeong, M., Kim, M., Lee, J. Y., & Kim, N. S. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3381910 · Cited by 10
Naturalistic Music Decoding from EEG Data via Latent Diffusion Models
Postolache, E., Polouliakh, N., Kitano, H., Connelly, A., Rodolà, E., Cosmo, L., & Akama, T. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887735 · Cited by 10
SF-Speech: Straightened Flow for Zero-Shot Voice Clone
Li, X., Shang, Z., Hua, H., Shi, P., Yang, C., Wang, L., & Zhang, P. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3557242 · Cited by 8
Delgado, P. M., & Herre, J. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3477291 · Cited by 4
WMCodec: End-to-End Neural Speech Codec with Deep Watermarking for Authenticity Verification
Zhou, J., Yi, J., Ren, Y. S., Tao, J., Wang, T., & Zhang, C. Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888430 · Cited by 4
Yang, D., Huang, R., Wang, Y., Guo, H., Chong, D., Liu, S., Wu, X., & Meng, H. M. L. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3574847 · Cited by 15
Personalized Neural Speech Codec
Jang, I., Yang, H., Lım, W., Beack, S., & Kim, M. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446067 · Cited by 4
Audio-Journey: Open Domain Latent Diffusion Based Text-To-Audio Generation
Michaels, J., Li, J. B., Yao, L., Yu, L., Wood-Doughty, Z., & Metze, F. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448220 · Cited by 5
Does Current Deepfake Audio Detection Model Effectively Detect ALM-Based Deepfake Audio?
Xie, Y., Xiong, C., Wang, X., Wang, Z., Lu, Y., Qi, X., Fu, R., Liu, Y., Wen, Z., Tao, J., et al. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800375 · Cited by 4
NDVQ: Robust Neural Audio Codec With Normal Distribution-Based Vector Quantization
Niu, Z., Chen, S., Zhou, L., Ma, Z., Chen, X., & Liu, S. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832324 · Cited by 7
Accelerating Codec-based Speech Synthesis with Multi-Token Prediction and Speculative Decoding
Nguyen, T. D., Kim, J., Choi, J., Choi, S., Park, J., Lee, Y., & Chung, J. S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887855 · Cited by 4
Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding
Li, B., Wang, H., Zhang, S., Guo, Y., & Yu, K. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888194 · Cited by 11
Audio Explanation Synthesis with Generative Foundation Models
Akman, A., Sun, Q., & Schuller, B. W. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890370 · Cited by 4
Direct speech-to-speech neural machine translation: A survey
Gupta, M., Dutta, M., & Maurya, C. K. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103317 · Cited by 11
Towards General-Purpose Text-Instruction-Guided Voice Conversion
Kuan, C.-Y., Li, C.-A., Hsu, T.-Y., Lin, T.-Y., Chung, H.-L., Chang, K., Chang, S.-Y., & Lee, H.-Y. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389672 · Cited by 6
Li, D., Ma, Y., Wei, W., Kong, Q., Wu, Y., Che, M., Xia, F., Benetos, E., & Li, W. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447445 · Cited by 4
Leveraging Pre-Trained Autoencoders for Interpretable Prototype Learning of Music Audio
Alonso-Jiménez, P., Pepino, L., Batlle-Roca, R., Zinemanas, P., Bogdanov, D., Serra, X., & Rocamora, M. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10669909 · Cited by 4
Biologically informed cortical models predict optogenetic perturbations
Sourmpis, C., Petersen, C. C. H., Wulfram Gerstner, B., & Bellec, G. (2026) · eLife · doi:10.7554/elife.106827.3 · Cited by 4
Du, H.-P., Ai, Y., Zheng, R.-C., & Ling, Z.-H. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800013 · Cited by 4
Low-Latency Deep Analog Speech Transmission Using Joint Source Channel Coding
Bokaei, M., Jensen, J., Doclo, S., & Østergaard, J. (2024) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2024.3521277 · Cited by 6
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Sadok, S., Leglaive, S., Laurent Girin, B., Gaël Richard, A., & Alameda-Pineda, X. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887856 · Cited by 4
Personalized and Controllable Voice Style Transfer With Speech Diffusion Transformer
Choi, H.-Y., Lee, S.-H., & Lee, S.-W. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3533362 · Cited by 4
EnvId: A Metric Learning Approach for Forensic Few-Shot Identification of Unseen Environments
Moussa, D., Hirsch, G., & Rieß, C. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3541534 · Cited by 4
DiffCSS: Diverse and Expressive Conversational Speech Synthesis with Diffusion Models
Wu, W., Lin, Z., Zhou, Y., Li, J., Niu, R., Wu, Q., Cao, S., Ma, L., & Wu, Z. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890208 · Cited by 5
Coarse-to-Fine Text-to-Music Latent Diffusion
Lanzendörfer, L. A., Lu, T., Perraudin, N., Herremans, D., & Wattenhofer, R. P. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888056 · Cited by 4
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing
Qiang, C., Wang, G., Zhao, Y., Fu, R., Wang, T., Cheng, G., Wang, T., Liu, Q., Yi, J., Wen, Z., et al. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3564168 · Cited by 6
Speech Separation using Neural Audio Codecs with Embedding Loss
Yip, J. Q., Kwok, C. Y., Ma, B., & Chng, E. S. (2024) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc63619.2025.10848753 · Cited by 4
MusicGen-Stem: Multi-stem music generation and edition through autoregressive modeling
Rouard, S., Roman, R. S., Adi, Y., & Röebel, A. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890052 · Cited by 3
Self-citationNoise-Robust Speaker Verification With Attenuated Speech Restoration and Consistency Training
Han, S., Ahn, Y., & Shin, J. W. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3567758 · Cited by 3
High Resolution Seismic Waveform Generation Using Denoising Diffusion
Palgunadi, K. H., Bergmeister, A., Bosisio, A., Ermert, L., Koroni, M., Perraudin, N., Dirmeier, S., & Meier, M. (2025) · Journal of Geophysical Research Machine Learning and Computation · doi:10.1029/2025jh000862 · Cited by 12
Low Bitrate Loss Resilience Scheme for a Speech Enhancing Neural Codec
Kolundžija, M., Kavalekalam, M. S., Balic, I. J., Mao, M., & Casas, R. A. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445746 · Cited by 3
Continual Gated Adapter for Bilingual Codec Text-to-Speech
Yang, L.-J., & Chien, J. (2024) · International Conference on Speech Database and Assessments · doi:10.1109/o-cocosda64382.2024.10800072 · Cited by 3
Lightweight Language Model for Speech Synthesis: Attempts and Analysis
Wu, Z., Liu, D., & Li, M. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800708 · Cited by 3
Attention-Constrained Inference For Robust Decoder-Only Text-to-Speech
Wang, H., Du, C., Guo, Y., Wang, S., Chen, X., & Yu, K. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832301 · Cited by 6
Benchmarking Music Generation Models and Metrics via Human Preference Studies
Grötschla, F., Solak, A., Lanzendörfer, L. A., & Wattenhofer, R. P. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887745 · Cited by 6
Textless Streaming Speech-to-Speech Translation using Semantic Speech Tokens
Zhao, J., Moritz, N., Lakomkin, E., Xie, R., Xiu, Z., Žmolíková, K., Ahmed, Z. A., Gaur, Y., Le, D., & Fuegen, C. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889740 · Cited by 3
ComplexDec: A Domain-robust High-fidelity Neural Audio Codec with Complex Spectrum Modeling
Wu, Y.-C., Marković, D., Krenn, S., Gebru, I. D., & Richard, A. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888771 · Cited by 8
InstantSpeech: Instant Synchronous Text-to-Speech Synthesis for LLM-driven Voice Chatbots
Du, M., Liu, C., & Lai, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890120 · Cited by 3
Evidential-TTS: High Fidelity Zero-Shot Text-to-Speech Using Evidential Deep Learning
Jeong, M., Kim, M., Kim, S., & Kim, N. S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889279 · Cited by 3
Latent Diffusion Bridges for Unsupervised Musical Audio Timbre Transfer
Mancusi, M., Halychanskyi, Y., Cheuk, K. W., Moliner, E., Lai, C.-H., Uhlich, S., Koo, J., Martínez-Ramírez, M. A., Liao, W., Fabbro, G., et al. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890708 · Cited by 7
Enhancing Zero-Shot Multi-Speaker TTS with Negated Speaker Representations
Jeon, Y., Kim, Y., & Lee, G. G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29793 · Cited by 3
Emotion-Conditioned MusicLM: Enhancing Emotional Resonance in Music Generation
Sun, Y., Kuo, M. M. Y., Wang, X., Li, W., & Bai, Q. (2024) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec60901.2024.10612075 · Cited by 5
Indonesian Voice Cloning Text-to-Speech System With Vall-E-Based Model and Speech Enhancement
Roosadi, H. R. P., Ginanjar, R. R., & Lestari, D. P. (2024) · IEEE Access · doi:10.1109/access.2024.3519870 · Cited by 3
SMITIN: Self-Monitored Inference-Time INtervention for Generative Music Transformers
Koo, J., Wichern, G., Germain, F. G., Khurana, S., & Roux, J. L. (2025) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2025.3534686 · Cited by 8
Synthetic Aircraft Trajectory Generation Using Time-Based VQ-VAE
Murad, A., & Ruocco, M. (2025) · Integrated Communications, Navigation and Surveillance Conference (ICNS) · doi:10.1109/icns65417.2025.10976929 · Cited by 3
Generative Semantic Communication for Text-to-Speech Synthesis
Zheng, J., Ren, J., Xu, P., Yuan, Z., Xu, J., Wang, F., Gui, G., & Cui, S. R. (2024) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshp64532.2024.11101684 · Cited by 3
Guo, H., Xie, F., Kang, J., Xiao, Y., Wu, X., & Meng, H. M. L. (2024) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3414342 · Cited by 4
Evaluation of Objective Quality Models on Neural Audio Codecs
Muller, T., Ragot, S., Barriac, V., & Scalart, P. (2024) · International Workshop on Acoustic Signal Enhancement · doi:10.1109/iwaenc61483.2024.10694655 · Cited by 2
Expressive Speech Synthesis with Theme-Oriented Few-Shot Learning in ICAGC 2024
Wang, H., Wang, T., Gong, C., Jiang, Y., Liu, Q., Wang, L., & Dang, J. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800403 · Cited by 2
SSR-Speech: Towards Stable, Safe and Robust Zero-shot Text-based Speech Editing and Synthesis
Wang, H., Meng, Y., Hai, J., Chen, C., Hu, Y., Chen, R., Dehak, N., & Yu, D. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889119 · Cited by 9
FlowMAC: Conditional Flow Matching for Audio Coding at Low Bit Rates
Pia, N., Strauss, M., Multrus, M., & Edler, B. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888898 · Cited by 2
TokenSynth: A Token-based Neural Synthesizer for Instrument Cloning and Text-to-Instrument
Kim, K., Koo, J., Lee, S., Joung, H., & Lee, K. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888403 · Cited by 2
Tokenized Generative Speech Enhancement With Language Model and Flow Matching
Yang, D.-H., Lee, J., & Chang, J. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3589128 · Cited by 2
HamNava: A Dataset for Multi‑Label Instrument Classification
Mohseni, P., BabaAli, B., & Asadi, H. (2025) · Transactions of the International Society for Music Information Retrieval · doi:10.5334/tismir.257 · Cited by 2
The detection of algebraic auditory structures emerges with self-supervised learning
Orhan, P., Boubenec, Y., & King, J.-R. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013271 · Cited by 5
Toward Guided Musical Form: Structuring Generative Music with LLMs
Atassi, L. (2024) · Computer Music Journal · doi:10.1162/comj.a.682 · Cited by 2
Native Multi-Band Audio Coding Within Hyper-Autoencoded Reconstruction Propagation Networks
Petermann, D., Jang, I., & Kim, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094593 · Cited by 2
Federated Neural Compression Under Heterogeneous Data
Lei, E., Hassani, H., & Bidokhti, S. S. (2023) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit54713.2023.10206457 · Cited by 2
Music Conditioned Generation for Human-Centric Video
Zhao, Z., Zuo, B., & Wang, Y. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3358978 · Cited by 3
Mapache: Masked Parallel Transformer for Advanced Speech Editing and Synthesis
Cámbara, G., Tobing, P. L., Babianski, M., Vipperla, R., Wang, D., Shmelkin, R., Coccia, G., Angelini, O., Joly, A., Łajszczak, M., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448121 · Cited by 2
Scalable Ensemble-Based Detection Method Against Adversarial Attacks For Speaker Verification
Wu, H., Kuo, H.-C., Tsao, Y., & Lee, H.-Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447441 · Cited by 2
Improving Language Model-Based Zero-Shot Text-to-Speech Synthesis with Multi-Scale Acoustic Prompts
Lei, S., Zhou, Y., Chen, L., Luo, D., Wu, Z., Wu, X., Kang, S., Jiang, T., Zhou, Y., Han, Y., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447815 · Cited by 3
Quantization Noise Masking in Perceptual Neural Audio Coder
Shin, S., Byun, J., Sung, J., Beack, S., & Park, Y.-C. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446359 · Cited by 2
Stack-and-Delay: A New Codebook Pattern for Music Generation
Lan, G. L., Nagaraja, V., Chang, E., Kant, D., Ni, Z., Shi, Y., Iandola, F., & Chandra, V. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447392 · Cited by 8
Channel-Configurable Deep Wireless Speech Transmission
Bokaei, M., Jensen, J., Doclo, S., & Østergaard, J. (2024) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc57260.2024.10570880 · Cited by 2
An Ultra-Low Bitrate Neural Audio Codec Under NB-IoT
Xu, Q., Yuan, X., Zhu, X., Ouyang, M., & Liao, G. (2024) · International Conference on Electronic Communication and Artificial Intelligence (ICECAI) · doi:10.1109/icecai62591.2024.10675006 · Cited by 2
High-Fidelity Diffusion-Based Audio Codec
Zhang, Z., Feng, J., Mao, Y., Zhu, Y., Shi, J., Ye, X., Liu, S., Liu, D., & Huang, C. (2024) · International Workshop on Acoustic Signal Enhancement · doi:10.1109/iwaenc61483.2024.10694300 · Cited by 2
Stahl, B., Windtner, S., & Sontacchi, A. (2024) · IEEE Access · doi:10.1109/access.2024.3482359 · Cited by 2
Perceptual Neural Audio Coding With Modified Discrete Cosine Transform
Lim, H., Lee, J., Kim, B. H., Jang, I., & Kang, H.-G. (2024) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2024.3491576 · Cited by 2
Sheferaw, G. K., Mwangi, W. R., Kimwele, M. W., Mamuye, A. L., & Salau, A. O. (2024) · IEEE Access · doi:10.1109/access.2024.3493604 · Cited by 2
Audio Compression Using Qubits and Quantum Neural Network
Aher, R. N., & Aher, N. D. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.10.307 · Cited by 2
Improved Alias-and-Separate Speech Coding Framework With Minimal Algorithmic Delay
Lee, E., Beack, S., & Shin, J. W. (2024) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2024.3501681 · Cited by 2
Evolutionary Music Synthesis: A Generative AI System with Interactive User Feedback
Ohya, K., Ayedoun, E., & Tokumaru, M. (2024) · Joint International Conference on Soft Computing and Intelligent Systems and International Symposium on Advanced Intelligent Systems (SCIS&ISIS) · doi:10.1109/scisisis61014.2024.10759906 · Cited by 2
The ISCSLP 2024 Conversational Voice Clone (CoVoC) Challenge: Tasks, Results and Findings
Xia, K., Guo, D., Yao, J., Xue, L., Li, H., Wang, S., Guo, Z., Xie, L., Zhang, Q., Luo, L., et al. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800694 · Cited by 2
Instructsing: High-Fidelity Singing Voice Generation Via Instructing Yourself
Zeng, C., Wang, C., Miao, X., Zhao, J., Jiang, Z., & Chen, Y. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832285 · Cited by 2
Metadata
- Authors
- Défossez, A., Copet, J., Synnaeve, G., & Adi, Y.
- Affiliations
- Meta (Israel) (IL)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2210.13438
- OpenAlex
- W4307323391
- Cited by
- 282
- References
- 0
- Citations by year
- 2026: 4 · 2025: 112 · 2024: 145 · 2023: 21
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Data Compression Techniques
- Keywords
- audio codec, encoder-decoder architecture, loss balancing, lightweight transformer, perceptual loss, MUSHRA, stereo audio
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
