Learning to Localize Sound Source in Visual Scenes

Senocak, Arda, Oh, Tae-Hyun, Kim, Junsik, Yang, Ming-Hsuan, Kweon, In So·2018·2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
DOI 10.1109/cvpr.2018.00458

Cited by (326)

Items 201–300 of 326

  1. Weakly Paired Associative Learning for Sound and Image Representations via Bimodal Associative Memory
    Sangmin Lee, Hyung-Il Kim, Yong Man Ro · 2022 · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings
  2. MMAL: Multi-Modal Analytic Learning for Exemplar-Free Audio-Visual Class Incremental Tasks
    Xianghu Yue, Xueyi Zhang, Yiming Chen, Chengwei Zhang, Mingrui Lao, Huiping Zhuang, Xinyuan Qian, Haizhou Li · 2024 · ACM International Conference on Multimedia (ACM MM)
  3. MEERKAT: Audio-Visual Large Language Model for Grounding in Space and Time
    Sanjoy Chowdhury, Sayan Nag, Subhrajyoti Dasgupta, Jun Chen, Mohamed Elhoseiny, Ruohan Gao, Dinesh Manocha · 2024 · Lecture notes in computer science
  4. Multi-Modal and Multi-Scale Temporal Fusion Architecture Search for Audio-Visual Video Parsing
    Jiayi Zhang, Weixin Li · 2023 · ACM International Conference on Multimedia (ACM MM)
  5. Scanning dial: the instantaneous audio classification transformer
    Huawei Jiang, Husna Mutahira, Unsang Park, Mannan Saeed Muhammad · 2024 · Discover Applied Sciences
  6. Learning to Visually Localize Sound Sources from Mixtures without Prior Source Knowledge
    Dong-Jin Kim, Sung Jin Um, Sangmin Lee, Jung Uk Kim · 2024 · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings
  7. CRATI: Contrastive representation-based multimodal sound event localization and detection
    Shichao Wu, Yongru Wang, Yushan Jiang, Qianyi Zhang, Jingtai Liu · 2024 · Knowledge-Based Systems
  8. FunnyNet: Audiovisual Learning of Funny Moments in Videos
    Zhi-Song Liu, Robin Courant, Vicky S. Kalogeiton · 2023 · Lecture notes in computer science
  9. Listen to Look Into the Future: Audio-Visual Egocentric Gaze Anticipation
    Bolin Lai, Fiona Ryan, Wenqi Jia, Miao Liu, James Matthew Rehg · 2024 · Lecture notes in computer science
  10. Object-aware Sound Source Localization via Audio-Visual Scene Understanding
    Sung Jin Um, Dong-Jin Kim, Sang‐Min Lee, Jung Uk Kim · 2025 · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings
  11. Self-Supervised Learning for Heterogeneous Audiovisual Scene Analysis
    Di Hu, Zheng Wang, Feiping Nie, Rong Wang, Xuelong Li · 2022 · IEEE Transactions on Multimedia
  12. Self-Motion As Supervision For Egocentric Audiovisual Localization
    Calvin Murdock, Ishwarya Ananthabhotla, Hao Lü, Vamsi Krishna Ithapu · 2024 · IEEE International Conference on Acoustics Speech and Signal Processing
  13. Saying the Unseen: Video Descriptions via Dialog Agents
    Ye Zhu, Yu Wu, Yi Yang, Yan Yan · 2021 · IEEE Transactions on Pattern Analysis and Machine Intelligence
  14. Online Audio-Visual Source Association for Chamber Music Performances
    Bochen Li, Karthik Dinesh, Chenliang Xu, Gaurav Sharma A, Zhiyao Duan · 2019 · Transactions of the International Society for Music Information Retrieval
  15. Separating Sounds from a Single Image.
    Lingyu Zhu, Esa Rahtu · 2020 · arXiv (Cornell University)
  16. Cross Pseudo-Labeling for Semi-Supervised Audio-Visual Source Localization
    Yuxin Guo, Shijie Ma, Yuhao Zhao, Hu Su, Wei Zou · 2024 · IEEE International Conference on Acoustics Speech and Signal Processing
  17. SeCo: Separating Unknown Musical Visual Sounds with Consistency Guidance
    Xinchi Zhou, Dongzhan Zhou, Wanli Ouyang, Hang Zhou, Di Hu · 2023 · IEEE Winter Conference on Applications of Computer Vision
  18. Deep Learning Based Audio-Visual Multi-Speaker DOA Estimation Using Permutation-Free Loss Function
    Qing Wang, Hang Chen, Ya Ping Jiang, Zhe Wang, Yuyang Wang, Jun Du, Chin‐Hui Lee · 2022 · International Symposium on Chinese Spoken Language Processing (ISCSLP)
  19. Multi Event Localization by Audio-Visual Fusion with Omnidirectional Camera and Microphone Array
    Wenru Zheng, Ryota Yoshihashi, Rei Kawakami, Ikuro Sato, Asako Kanezaki · 2023 · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  20. Audio-Visual Event Localization using Multi-task Hybrid Attention Networks for Smart Healthcare Systems
    Han Liang, Jincai Chen, Fazlullah Khan, Gautam Srivastava, Jiangfeng Zeng · 2024 · ACM Transactions on Internet Technology
  21. Can CLIP Help Sound Source Localization?
    Sooyoung Park, Arda Senocak, Joon Son Chung · 2024 · IEEE Winter Conference on Applications of Computer Vision
  22. Audio–Visual Segmentation based on robust principal component analysis
    Shun Fang, Qile Zhu, Qi Wu, Shiqian Wu, Shoulie Xie · 2024 · Expert Systems with Applications
  23. Self-Supervised Learning for Audio-Visual Relationships of Videos With Stereo Sounds
    Tomoya Sato, Yusuke Sugano, Yoichi Sato · 2022 · IEEE Access
  24. The Head Turning Modulation System: An Active Multimodal Paradigm for Intrinsically Motivated Exploration of Unknown Environments
    Benjamin Cohen-Lhyver, Sylvain Argentieri, Bruno Gas · 2018 · Frontiers in Neurorobotics
  25. Noisy Agents: Self-supervised Exploration by Predicting Auditory Events
    Chuang Gan, Xiaoyu Chen, Phillip Isola, Antonio B. Torralba, Joshua B. Tenenbaum · 2022 · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  26. Decompose the Sounds and Pixels, Recompose the Events
    Varshanth R. Rao, Md Ibrahim Khalil, Haoda Li, Peng Dai, Juwei Lu · 2022 · Proceedings of the AAAI Conference on Artificial Intelligence
  27. Deep Co-Clustering for Unsupervised Audiovisual Learning.
    Di Hu, Feiping Nie, Xuelong Li · 2018 · arXiv (Cornell University)
  28. Cross-Modal Attention Consistency for Video-Audio Unsupervised Learning
    Shaobo Min, Dai, Qi, Hongtao Xie, Chuang Gan, Yongdong Zhang, Jingdong Wang · 2021 · arXiv (Cornell University)
  29. Contrastive Learning of Global-Local Video Representations
    Shuang Ma, Zhaoyang Zeng, Daniel McDuff, Yale Song · 2021 · arXiv (Cornell University)
  30. TransAVS: End-to-End Audio-Visual Segmentation with Transformer
    Yuhang Ling, Yuxi Li, Zhenye Gan, Jiangning Zhang, Mingmin Chi, Yabiao Wang · 2024 · IEEE International Conference on Acoustics Speech and Signal Processing
  31. Multi-speaker DoA Estimation Using Audio and Visual Modality
    Yulin Wu, Ruimin Hu, Xiaochen Wang, Shanfa Ke · 2023 · Neural Processing Letters
  32. Flowgrad: Using Motion for Visual Sound Source Localization
    Rajsuryan Singh, Pablo Zinemanas, Xavier Serra, Juan Pablo Bello, Magdalena Fuentes · 2023 · IEEE International Conference on Acoustics Speech and Signal Processing
  33. Audio-Visual Spatial Integration and Recursive Attention for Robust Sound Source Localization
    Sung Jin Um, Dongjin Kim, Jung Uk Kim · 2023 · ACM International Conference on Multimedia (ACM MM)
  34. Chaotic World: A Large and Challenging Benchmark for Human Behavior Understanding in Chaotic Events
    Kian Eng Ong, Xun Long Ng, Yanchao Li, Wenjie Ai, Kuangyi Zhao, Si Yong Yeo, Jun Liu · 2023 · IEEE/CVF International Conference on Computer Vision (ICCV)
  35. Alignment and Fusion Using Distinct Sensor Data for Multimodal Aerial Scene Classification
    Xu Guang Sun, Junyu Gao, Yuan Yuan · 2024 · IEEE Transactions on Geoscience and Remote Sensing
  36. Rule-Embedded Network for Audio-Visual Voice Activity Detection in Live Musical Video Streams
    Yuanbo Hou, Yi Deng, Bilei Zhu, Zejun Ma, Dick B. Botteldooren · 2021 · IEEE International Conference on Acoustics Speech and Signal Processing
  37. PoseKernelLifter: Metric Lifting of 3D Human Pose using Sound
    Zhijian Yang, Xiaoran Fan, Volkan Isler, Hyun Soo Park · 2022 · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings
  38. Induction Network: Audio-Visual Modality Gap-Bridging for Self-Supervised Sound Source Localization
    Tianyu Liu, Peng Zhang, Wei Huang, Yufei Zha, Tao You, Yanning Zhang · 2023 · ACM International Conference on Multimedia (ACM MM)
  39. Learning Fine-Grained Semantics in Spoken Language Using Visual Grounding
    Xinsheng Wang, Tian Tian, Jihua Zhu, Odette Scharenborg · 2021 · IEEE International Symposium on Circuits and Systems proceedings
  40. Toward Interactive Sound Source Localization: Better Align Sight and Sound!
    Arda Senocak, Hyeonggon Ryu, Junsik Kim, Tae-Hyun Oh, Hanspeter Pfister, Joon Son Chung · 2025 · IEEE Transactions on Pattern Analysis and Machine Intelligence
  41. Unveiling Visual Biases in Audio-Visual Localization Benchmarks
    Liangyu Chen, Zihao Yue, Boshen Xu, Qin Jin · 2025 · Lecture notes in computer science
  42. Complementary Cues from Audio Help Combat Noise in Weakly-Supervised Object Detection
    Cagri Gungor, Adriana Kovashka · 2023 · IEEE Winter Conference on Applications of Computer Vision
  43. Cross-Modal Feature Enhancement Networks for Audio-Visual Event Localization
    Qian Xu, Yunxia Wu, Jiachen Zeng, Zhihan Wang, Hailan Zhang · 2025 · IEEE Sensors Journal
  44. Listen To The Pixels
    Sanjoy Chowdhury, Subhrajyoti Dasgupta, Sudip Kumar Das, Ujjwal Bhattacharya · 2021 · Proceedings - International Conference on Image Processing
  45. Bio-Inspired Audiovisual Multi-Representation Integration via Self-Supervised Learning
    Zhaojian Li, Bin Zhao, Yuan Yuan · 2023 · ACM International Conference on Multimedia (ACM MM)
  46. Sound3DVDet: 3D Sound Source Detection using Multiview Microphone Array and RGB Images
    Yuhang He, Sangyun Shin, Anoop Cherian, Niki Trigoni, Andrew Markham · 2024 · IEEE Winter Conference on Applications of Computer Vision
  47. CoPL:Parameter-Efficient Collaborative Prompt Learning for Audio-Visual Tasks
    Yihan Zhao, Wei Xi, Yuhang Cui, Gairui Bai, Xinhui Liu, Jizhong Zhao · 2024 · ACM International Conference on Multimedia (ACM MM)
  48. Active Speaker Detection and Localization in Videos Using Low-Rank and Kernelized Sparsity
    Jie Xin Pu, Yannis Panagakis, Maja Pantić · 2020 · IEEE Signal Processing Letters
  49. How to Listen? Rethinking Visual Sound Localization
    Ho-Hsiang Wu, Magdalena Fuentes, Prem Seetharaman, Juan Pablo Bello · 2022 · Interspeech (USB)
  50. Expectation Learning for Stimulus Prediction Across Modalities Improves Unisensory Classification
    Pablo V. A. Barros, Manfred Eppe, German Ignacio Parisi, Xun Liu, Stefan Wermter · 2019 · Frontiers in Robotics and AI
  51. Audio-Visual 3D Reconstruction Framework for Dynamic Scenes
    Takashi Konno, Kenji Nishida, Katsutoshi Itoyama, Kazuhiro Nakadai · 2020 · IEEE/SICE International Symposium on System Integration (SII)
  52. Cross-modal Variational Auto-encoder with Distributed Latent Spaces and Associators
    Dae Ung Jo, Byeong‐Ju Lee, Jongwon Choi, Haanju Yoo, Jin Young Choi · 2019 · arXiv (Cornell University)
  53. Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation
    Yapeng Tian, Di Hu, Chenliang Xu · 2021 · arXiv (Cornell University)
  54. Multi-target DoA Estimation with an Audio-visual Fusion Mechanism
    Xinyuan Qian, Maulik C. Madhavi, Zexu Pan, Jiadong Wang, Haizhou Li · 2021 · arXiv (Cornell University)
  55. Multi-Modulation Network for Audio-Visual Event Localization
    Hao Wang, Zheng-Jun Zha, Liang Li, Xuejin Chen, Jiebo Luo · 2021 · arXiv (Cornell University)
  56. Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes
    Hyeonggon Ryu, Seongyu Kim, Joon Son Chung, Arda Senocak · 2025 · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings
  57. AV-Unified: A Unified Framework for Audio-Visual Scene Understanding
    G Li, Xin Wang, Wenwu Zhu · 2026 · IEEE Transactions on Multimedia
  58. Frame-Level Multiple Sound Sources Localization Based on Visual Understanding
    Lixuezi Fang, Long Ye, Xin Ma, Ruiqi Wang, Zhong Wei, Qin Zhang · 2021 · Asian Conference on Artificial Intelligence Technology (ACAIT)
  59. Audio-Visual Event and Sound Source Localization Based on Spatial-Channel Feature Fusion
    Xiaolong Zheng, Ying Wei · 2022 · 2022 7th International Conference on Signal and Image Processing (ICSIP)
  60. Multimodal Network with Cross-Modal Attention for Audio-Visual Event Localization
    Qianchao Tan · 2022 · International Workshop on Human-centric Multimedia Analysis (HCMA)
  61. Enhancing Sound Source Localization via False Negative Elimination
    Zengjie Song, Jiangshe Zhang, Yuxi Wang, Junsong Fan, Zhaoxiang Zhang · 2024 · IEEE Transactions on Pattern Analysis and Machine Intelligence
  62. Audio-Visual Generalized Zero-Shot Learning the Easy Way
    Shentong Mo, Pedro M. Morgado · 2024 · Lecture notes in computer science
  63. SoundLoc3D: Invisible 3D Sound Source Localization and Classification Using a Multimodal RGB-D Acoustic Camera
    Yuhang He, Sangyun Shin, Anoop Cherian, Niki Trigoni, Andrew Markham · 2025 · IEEE Winter Conference on Applications of Computer Vision
  64. Unsupervised Sounding Pixel Learning
    Yining Zhang, Yanli Ji, Yang Yang · 2023 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  65. Audio Compensation Network: to Improve the Quality of Low-Energy Audio in Visual Sound Separation
    Yining Gao, Pengyuan Zhang, Zejia Tian · 2021 · International Conference on Consumer Electronics and Computer Engineering (ICCECE)
  66. Unsupervised Co-Segmentation for Athlete Movements and Live Commentaries Using Crossmodal Temporal Proximity
    Yasunori Ohishi, Yuki Tanaka, Kunio Kashino · 2021 · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition
  67. The Sound of Bounding-Boxes
    Takashi Oya, Shohei Iwase, Shigeo Morishima · 2022 · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition
  68. SEMIPL: A Semi-Supervised Method for Event Sound Source Localization
    Yue Li, Baiqiao Yin, Jinfu Liu, Jiajun Wen, Jiaying Lin, Mengyuan Liu · 2024 · IEEE International Conference on Multimedia and Expo workshops
  69. Look, Listen and Infer
    Ruijian Jia, Xinsheng Wang, Shanmin Pang, Jihua Zhu, Jianru Xue · 2020 · ACM International Conference on Multimedia (ACM MM)
  70. Swoosh! Rattle! Thump! -- Actions that Sound
    Dhiraj Gandhi, Abhinav Gupta, Lerrel Pinto · 2020 · arXiv (Cornell University)
  71. Geometry-aware multi-task learning for binaural audio generation from video
    Garg, Rishabh · 2021 · arXiv (Cornell University)
  72. Joint Learning of Visual-Audio Saliency Prediction and Sound Source Localization on Multi-face Videos
    Minglang Qiao, Yufan Liu, Mai Xu, Xin Deng, Bing Li, Weiming Hu, Ali Borji · 2021 · arXiv (Cornell University)
  73. Direction-of-Arrival Estimation for Mobile Agents Utilizing the Relationship Between Agent’s Trajectory and Binaural Audio
    Tomoya Sato, Yusuke Sugano, Yoichi Sato · 2024 · IEEE Access
  74. Multimodal learning audio-visual detection for obtaining object-level sound sources in Japanese-language teaching room
    Lu Li, Xiuxiu Bai, Jun Xu, Dingkang Wang, Tao Jiang · 2025 · Scientific Reports
  75. Progressive Confident Masking Attention Network for Audio–Visual Segmentation
    Yuxuan Wang, Jinchao Zhu, Feng Dan Dong, Shuyue Zhu · 2025 · Knowledge-Based Systems
  76. Hearing and Seeing Through CLIP: A Framework for Self-Supervised Sound Source Localization
    Sooyoung Park, Arda Senocak, Joon Son Chung · 2026 · International Journal of Computer Vision
  77. Cross-Modal Semantic Relevance is an Efficient Gatekeeper for Audio-Visual Video Parsing
    Jinxing Zhou, Z Li, Shijie Hao, Pengcheng Zhao, Yuxin Mao, Y Zhou, Ruohao Guo, Yiran Zhong, Xiaojun Chang · 2026 · IEEE Transactions on Multimedia
  78. Weakly-Supervised Action Detection Guided by Audio Narration
    Keren Ye, Adriana Kovashka · 2022 · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  79. Audio-Visual Class Association Based on Two-stage Self-supervised Contrastive Learning towards Robust Scene Analysis
    Kei Suzuki, Katsutoshi Itoyama, Kenji Nishida, Kazuhiro Nakadai · 2023 · IEEE/SICE International Symposium on System Integration (SII)
  80. An Active Speaker Detection Method in Videos using Standard Deviations of Color Histogram
    Adekunle A. Akinrinmade, Emmanuel Adetiba, Joke A. Badejo, Oluwadamilola I. Oshin · 2023 · International Conference on Science, Engineering and Business for Sustainable Development Goals (SEB-SDG)
  81. Self-Supervised Sound Promotion Method of Sound Localization from Video
    Yang Li, Xiaoli Zhao, Zhuoyao Zhang · 2023 · Electronics
  82. A Critical Assessment of Visual Sound Source Localization Models Including Negative Audio
    Xavier Juanola, Gloria Haro, Magdalena Fuentes · 2025 · IEEE International Conference on Acoustics Speech and Signal Processing
  83. Audio-visual localization based on spatial relative sound order
    Tomoya Sato, Yusuke Sugano, Yoichi Sato · 2025 · Machine Vision and Applications
  84. Unsupervised Sound Source Localization From Audio-Image Pairs Using Input Gradient Map
    Tomohiro Tanaka, Takahiro Shinozaki · 2021 · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition
  85. Unveiling unexpected training data in internet video
    Tali Dekel, Noah Snavely · 2021 · Communications of the ACM
  86. Learning Self-supervised Audio-Visual Representations for Sound Recommendations
    Sudha Krishnamurthy · 2021 · Lecture notes in computer science
  87. Sound Can Help Us See More Clearly
    Yongsheng Li, Tengfei Tu, Hua Zhang, Jishuai Li, Zhengping Jin, Qiaoyan Wen · 2022 · Sensors
  88. 2.5D Visual Sound
    Ruohan Gao, Kristen Lorraine Grauman · 2018 · arXiv (Cornell University)
  89. Can audio-visual integration strengthen robustness under multimodal attacks?
    Yapeng Tian, Chenliang Xu · 2021 · arXiv (Cornell University)
  90. Read and Attend: Temporal Localisation in Sign Language Videos
    Gül Varol, Liliane Momeni, Samuel Albanie, Triantafyllos Afouras, Andrew Zisserman · 2021 · arXiv (Cornell University)
  91. Progressive Confident Masking Attention Network for Audio-Visual Segmentation
    Yuxuan Wang, Jinchao Zhu, Feng Dan Dong · 2023 · SSRN Electronic Journal
  92. Controllable Attention for Structured Layered Video Decomposition
    Jean-Baptiste Alayrac, João Carreira, Relja Arandjelović, Andrew Zisserman · 2019 · arXiv (Cornell University)
  93. End-to-End Lip Synchronisation.
    You Jin Kim, Hee Soo Heo, Soo-Whan Chung, Bong‐Jin Lee · 2020 · arXiv (Cornell University)
  94. Do We Need Sound for Sound Source Localization?
    Takashi Oya, Shohei Iwase, Ryota Natsume, Takahiro Itazuri, Shugo Yamaguchi, Shigeo Morishima · 2020 · arXiv (Cornell University)
  95. Rule-embedded network for audio-visual voice activity detection in live musical video streams
    Yuanbo Hou, Yi Deng, Bilei Zhu, Zejun Ma, Dick B. Botteldooren · 2020 · arXiv (Cornell University)
  96. Learning Audio-Visual Correlations from Variational Cross-Modal Generation
    Ye Zhu, Yu Wu, Hugo M. Latapie, Yi Yang, Yan Yan · 2021 · arXiv (Cornell University)
  97. Dual Normalization Multitasking for Audio-Visual Sounding Object Localization
    Tokuhiro Nishikawa, Daiki Shimada, Jerry Jun Yokono · 2021 · arXiv (Cornell University)
  98. Bio-Inspired Audio-Visual Cues Integration for Visual Attention Prediction.
    Yuan Yuan, Hailong Ning, Bin Zhao · 2021 · arXiv (Cornell University)
  99. The Impact of Spatiotemporal Augmentations on Self-Supervised Audiovisual Representation Learning
    Haider Al-Tahan, Yalda Mohsenzadeh · 2021 · arXiv (Cornell University)
  100. On Off-Screen Speech Separation based on Deep Learning of Audio-Video Fusion
    Yang Yu, Yang Liu, Ying Wei · 2021 · International Conference on Computational Intelligence and Applications (ICCIA)