Mustango: Toward Controllable Text-to-Music Generation
Open accessAbstract
Jan Melechovsky, Zixun Guo, Deepanway Ghosal, Navonil Majumder, Dorien Herremans, Soujanya Poria. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MusicBench contains around 50k audio samples.
“Other datasets are often much smaller, such as MusicBench containing around 50k audio samples (Melechovsky et al., 2024).”
From SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning (2025) · cites this paper as Melechovsky et al., 2024
Training a model on the MusicBench dataset enriched with music features improved the model to include dedicated musical features in the captions.
“They found that training the model on such an enriched dataset improved the model to include dedicated musical feature sin the captions.”
From SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning (2025) · cites this paper as Melechovsky et al., 2024
Music captioning remains an open challenge due to the lack of an extensive dataset.
“Ironically, this task remains an open challenge due to the lack of an extensive dataset.”
From SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning (2025) · cites this paper as Melechovsky et al., 2024
The MusicBench training set excludes samples that are part of the MusicCaps evaluation set.
“Finally, we use the MusicBench training set which consists of around 26k clips (after excluding those samples that are part of MusicCaps eval set) with paired captions (10s).”
From SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning (2025) · cites this paper as Melechovsky et al., 2024
All text caption (NLP) evaluations were performed on the MusicCaps evaluation set, while all music-feature specific evaluations were performed on the MusicBench test set, following the MusicCaps split.
“All text caption (NLP) evaluations were performed on the MusicCaps evaluation set (Agostinelli et al., 2023), while all music-feature specific evaluations were performed on the MusicBench test set (Melechovsky et al., 2024), following the MusicCaps split (Agostinelli et al., 2023).”
From SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning (2025) · cites this paper as Melechovsky et al., 2024
Cites
Still being checked.
Cited by
Sorted by citations
AI-Based Affective Music Generation Systems: A Review of Methods and Challenges
Dash, A., & Agres, K. (2024) · ACM Computing Surveys · doi:10.1145/3672554 · Cited by 115 · see citing works
Video2Music: Suitable music generation from videos using an Affective Multimodal Transformer model
Kang, J., Poria, S., & Herremans, D. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123640 · Cited by 62 · see citing works
Self-citationTango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization
Majumder, N., Hung, C.-Y., Ghosal, D., Hsu, W.-N., Mihalcea, R. F., & Poria, S. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681688 · Cited by 58 · see citing works
Self-citationKang, J., & Herremans, D. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3583505 · Cited by 21 · see citing works
Self-citationMusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models
Zhang, Y., Ikemiya, Y., Xia, G., Murata, N., Martínez-Ramírez, M. A., Liao, W., Mitsufuji, Y., & Dixon, S. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/864 · Cited by 19 · see citing works
Comanducci, L., Bestagini, P., & Tubaro, S. (2025) · Journal of Imaging · doi:10.3390/jimaging11070242 · Cited by 15 · see citing works
Taming Data and Transformers for Audio Generation
Haji-Ali, M., Menapace, W., Siarohin, A., Balakrishnan, G., & Ordóñez, V. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02632-y · Cited by 15 · see citing works
Dance-to-Music Generation with Encoder-based Textual Inversion
Li, S., Dong, W., Zhang, Y., Tang, F., Ma, C., Deußen, O., Lee, T., & Xu, C. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687562 · Cited by 11 · see citing works
PAGURI: A User Experience Study of Creative Interaction with Text-to-Music Models
Ronchini, F., Comanducci, L., Perego, G., & Antonacci, F. (2025) · Electronics · doi:10.3390/electronics14173379 · Cited by 10 · see citing works
DiffRhythm+: Controllable and Flexible Full-Length Song Generation with Preference Optimization
Chen, H., Jiang, Y., Ma, G., Hao, C. X., Wang, S., Yao, J., Ning, Z., Meng, M., Luan, J., & Xie, L. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434644 · Cited by 8 · see citing works
Zheng, J., Cao, M., & Zhang, C. (2025) · Multimedia Systems · doi:10.1007/s00530-025-02023-w · Cited by 7 · see citing works
Benchmarking Music Generation Models and Metrics via Human Preference Studies
Grötschla, F., Solak, A., Lanzendörfer, L. A., & Wattenhofer, R. P. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887745 · Cited by 6 · see citing works
Simultaneous Music Separation and Generation Using Multi-Track Latent Diffusion Models
Karchkhadze, T., Izadi, M. R., & Dubnov, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889421 · Cited by 6 · see citing works
Multi-track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Model
Karchkhadze, T., Izadi, M. R., Chen, K., Assayag, G., & Dubnov, S. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-97257-7_6 · Cited by 6 · see citing works
Visual signatures for music mood and timbre
Wang, H., & Sourin, A. (2024) · The Visual Computer · doi:10.1007/s00371-024-03417-z · Cited by 6 · see citing works
Wang, Z., Li, S., Zhang, T., Wang, Q., Yu, P., Luo, J., Liu, Y., Ming, X., & Zhang, K. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/860 · Cited by 4 · see citing works
NOTA: Multimodal Music Notation Understanding for Visual Large Language Model
Tang, M., Li, J., Lu, Y., Zhang, Z., Tian, J., Li, Z., Zhang, L., & Wang, P. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.399 · Cited by 4 · see citing works
Coarse-to-Fine Text-to-Music Latent Diffusion
Lanzendörfer, L. A., Lu, T., Perraudin, N., Herremans, D., & Wattenhofer, R. P. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888056 · Cited by 4 · see citing works
Self-citationBootstrapping Language-Audio Pre-training for Music Captioning
Lanzendörfer, L. A., Pinkl, C., Perraudin, N., & Wattenhofer, R. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10887618 · Cited by 3 · see citing works
Zheng, J., Cao, M., & Zhang, C. (2025) · IEEE Access · doi:10.1109/access.2025.3572954 · Cited by 3 · see citing works
AudioGenie: A Training-Free Multi-Agent Framework for Diverse Multimodality-to-Multiaudio Generation
Rong, Y., Wang, J., Lei, G., Yang, S., & Liu, L. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755758 · Cited by 3 · see citing works
JamendoMaxCaps: A Large Scale Music-caption Dataset with Imputed Metadata
Roy, A., Liu, R., Lu, T., & Herremans, D. (2025) · 2025 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228034 · Cited by 2 · see citing works
Self-citationInterpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew’s Treatise
Karchkhadze, T., Shao, K., & Dubnov, S. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825824 · Cited by 2 · see citing works
Deep Composer: Improving the String Quartet Music Generation Task
Galajda, J. E., Bach, V. H. B., & Hua, K. A. (2025) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr67560.2025.00022 · Cited by 2 · see citing works
Effectively Obtaining Acoustic, Visual, and Textual Data from Videos
León, J. E., & Carrasco, M. (2025) · Applied Sciences · doi:10.3390/app152312654 · Cited by 2 · see citing works
Generative Music Models’ Alignment with Professional and Amateur Users’ Expectations
Wang, Z., Yu, J., Liu, H., Zheng, Z., Jin, Y., Li, S., Ji, S., & Zhang, K. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.360 · Cited by 2 · see citing works
Research on Polyphonic Music Generation Algorithm Based on GPT Large Model
Zhu, L., & Zhang, W. (2025) · IEEE Access · doi:10.1109/access.2025.3588847 · Cited by 2 · see citing works
Effective Techniques for Scaling Audio Encoder Pretraining
Kim, B., Bydlon, A., Tang, Q., Phan, H., Kao, C.-C., Zhang, T., & Wang, C. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890012 · Cited by 1 · see citing works
Wu, J., Novack, Z., Namburi, A., Dong, H., Chen, C., Dai, J., & McAuley, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888485 · Cited by 1 · see citing works
Customized Condition Controllable Generation for Video Soundtrack
Qi, F., Ma, K., & Xu, C. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02227 · Cited by 1 · see citing works
User Intent-Based Music Generation Model Combining Actor-Critic Approach With MusicVAE
Jang, S., & Lee, J. (2025) · IEEE Access · doi:10.1109/access.2025.3597741 · Cited by 1 · see citing works
Enhancing MusicGen with Prompt Tuning
Shin, H., Im, J., & Sung, Y. (2025) · Applied Sciences · doi:10.3390/app15158504 · Cited by 1 · see citing works
ERLD-HC: Entropy-Regularized Latent Diffusion for Harmony-Constrained Symbolic Music Generation
Li, Y. (2025) · Entropy · doi:10.3390/e27090901 · Cited by 1 · see citing works
Yang, Y., Jia, Y., Xu, J., Song, K., Dong, D., & He, J. (2025) · International Workshop on Multimedia Content Generation and Evaluation: New Methods and Practice (McGE) · doi:10.1145/3746278.3759380 · Cited by 1 · see citing works
Exploring Adapter Design Tradeoffs for Low Resource Music Generation
Mehta, A., Chauhan, S., & Choudhury, M. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755766 · Cited by 1 · see citing works
Controllable Video-to-Music Generation with Multiple Time-Varying Conditions
Wu, J., You, W., Zuo, H., Zhang, D., Chen, P., & Sun, L. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755656 · Cited by 1 · see citing works
Zhao, Z., Guo, Z., Du, J., Ma, F., & Pan, J. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209133 · Cited by 1 · see citing works
Self-citationAligning Generative Music AI with Human Preferences: Methods and Challenges
Herremans, D., & Roy, A. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i46.41323 · Cited by 1 · see citing works
Self-citationRecent advances in music generation: methods, evaluation, and challenges
段玮, Yu, Y., & Briot, J.-P. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11582-x · Cited by 1 · see citing works
Zero-Effort Image-to-Music Generation: An Interpretable RAG-based VLM Approach
Zhao, Z., Jin, D., & Zhou, Z. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810723 · Cited by 1 · see citing works
Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editing
Tian, Z., Yang, B., Liu, Z. T., Zhang, J., Yuan, R., Yin, H., Chen, Q., Li, C., Lyu, J., Xue, W., et al. (2026) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3799902.3811191 · Cited by 1 · see citing works
SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning
Chopra, A., Roy, A., & Herremans, D. (2025) · arXiv · doi:10.48550/arxiv.2506.15154
“By training the proposed multitask captioning model on the Magna-Tag-A-Tune dataset (Law et al., 2009) with MosiacML (MosaicML NLP Team et al., 2023) captions and MIRFLEX-based (Chopra et al., 2024) features, and finetuning on the MusicBench (Melechovsky et al., 2024) dataset expanded with MIRFLEX-based features, we c…”
InfluentialSelf-citationDiffusion based Text-to-Music Generation with Global and Local Text based Conditioning
Zhang, J., Parada, P. P., Jalal, M. A., & Saravanan, K. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889289
PixMus : Video and Text Conditioned Background Music Generation Using Latent Diffusion
K., T., Kaushik, A. R., & Ratha, N. (2025) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai64502.2025.00136
Generating Music With Retention Based Latent Transformer Models
Viswanathan, V., Veeravalli, B., & Mohan, G. B. (2025) · International Conference on Communication, Security and Artificial Intelligence (ICCSAI) · doi:10.1109/iccsai64074.2025.11063879
[Paper] TPSMG: Text-Controllable Polyphonic Symbolic Music Generation
Zhu, J., Sakurai, K., Togo, R., OGAWA, T., & Haseyama, M. (2025) · ITE Transactions on Media Technology and Applications · doi:10.3169/mta.14.110
Bad Designs by Good Talkers: Chatbots Failing to Architect Audio Encoders for Image Synthesis
León, J. E., Carrasco, M., & Peters, A. A. (2026) · IEEE Access · doi:10.1109/access.2026.3668130
Mel-Refine: A Plug-and-Play Approach to Refine Mel-Spectrogram in Audio Generation
Guo, H., Fu, R., Geng, Y., Shi, S., Wang, T., Qiang, C., Li, Y., Wen, Z., Liu, Y., Liu, X., et al. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434719
Advancing Controllable Music Generation with Latent Rectified Flow Guided by Rhythm and Harmony
Yu, H., Zhou, J., Wang, W., Wang, Z., Zhu, H., & Qian, Y. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434607
Li, J., Yu, Y., Shao, Y., & Cheema, A. A. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3749
Repulsive Guidance for Memorization Mitigation in Text-to-Music Diffusion Models
Kim, T., Lee, H., Ahn, C. W., & Kim, M.-J. (2026) · Mathematics · doi:10.3390/math14091512
Multi-style music generation via multi-modal conditional encoding and flow matching
Zhang, Y., & Du, G. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02368-w
Go, G., Han, S., Choi, A., Choi, E., Nam, J., & Park, J. M. (2026) · Research Square · doi:10.21203/rs.3.rs-9027741/v1
Narrative2Nanyin: Generative Applied Intelligence for Heritage Nanyin Music
Xiahou, J., Zhai, W., & Cui, X. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-7087-4_27
Yang, C., Tan, W., Chen, H., Li, G., Lei, S., Yu, J., Wang, S. Q., & Li, H. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3705684
Auditing Training Data in Generative Music Models via Black-Box Membership Inference
Liu, Y. C., Yu, J., Cao, K., Meerza, S. I. A., Movva, T., & Liu, J. (2026) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw72489.2026.00027
FIGMA: Towards FIne-Grained Music retrievAl
Anand, N., Seth, A., Ghosh, S., Manocha, D., & Ramani Duraiswami, B. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.2197
AudioStealer: Extracting Audio Prompts via Shapley Value-Guided Query Search
Jin, Y., Du, X., Luo, H., Wang, Z., Hu, H., Wang, X., & Li, X. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.688
SyncCLIPAgent: leveraging LLMs for automated video editing with audiovisual synchronization
Yuan, D., & Liang, G. (2026) · Cluster Computing · doi:10.1007/s10586-026-06308-1
Hu, X., & Wang, X. (2026) · International Conference on Computer Technology and Sustainable Development · doi:10.1145/3814210.3814244
Huang, J. (2026) · SHS Web of Conferences · doi:10.1051/shsconf/202623602002
MUSICAL REPRESENTATION OF GENRE INFORMATION IN GENERATIVE ARTIFICIAL INTELLIGENCE: THE CASE OF SUNO
Yücebağ, H. (2026) · The Turkish Online Journal of Design Art and Communication · doi:10.7456/tojdac.1993191
Wang, Y., Zhang, X., & Wang, Y. (2026) · Information · doi:10.3390/info17101035
陶蕾伃, & Chen, X. (2026) · Current Psychology · doi:10.1007/s12144-026-10181-x
- Authors
- Melechovsky, J., Guo, Z., Ghosal, D., Majumder, N., Herremans, D., & Poria, S.
- Venue
- Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
- Published
- 2024
- Pages
- 8293-8316
- OpenAlex
- W4401043564
- Cited by
- 48
- References
- 0
- Citations by year
- 2026: 17 · 2025: 28 · 2024: 3
- FWCI
- 9.94
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Music Technology and Sound Studies
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- text-to-music generation, controllable music generation, music generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
