The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
The visual microphone: passive recovery of sound from video
Open accessAbstract
When sound hits an object, it causes small vibrations of the object's surface. We show how, using only high-speed video of the object, we can extract those minute vibrations and partially recover the sound that produced them, allowing us to turn everyday objects---a glass of water, a potted plant, a box of tissues, or a bag of chips---into visual microphones. We recover sounds from high-speed footage of a variety of objects with different properties, and use both real and simulated data to examine some of the factors that affect our ability to visually recover sound. We evaluate the quality of recovered sounds using intelligibility and SNR metrics and provide input and recovered audio samples for direct comparison. We also explore how to leverage the rolling shutter in regular consumer cameras to recover audio from standard frame-rate videos, and use the spatial resolution of our method to visualize how sound-related vibrations vary over an object's surface, which we can use to recover the vibration modes of an object.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Suppression of acoustic noise in speech using spectral subtraction
Boll, S. (1979) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1979.1163209 · Cited by 4,634
What energy functions can be minimized via graph cuts?
Kolmogorov, V., & Zabih, R. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1262177 · Cited by 3,127
An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy Speech
Taal, C. H., Hendriks, R. C., Heusdens, R., & Jensen, J. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2114881 · Cited by 2,254
YIN, a fundamental frequency estimator for speech and music
de Cheveigné, A., & Kawahara, H. (2002) · The Journal of the Acoustical Society of America · doi:10.1121/1.1458024 · Cited by 1,920
A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients
Portilla, J., & Simoncelli, E. P. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026553619983 · Cited by 1,806
Eulerian video magnification for revealing subtle changes in the world
Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., & Freeman, W. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185561 · Cited by 1,357
Self-citationShiftable multiscale transforms
Simoncelli, E. P., Freeman, W. T., Adelson, E. H., & Heeger, D. J. (1992) · IEEE Transactions on Information Theory · doi:10.1109/18.119725 · Cited by 1,333
Self-citationPhase-based video motion processing
Wadhwa, N., Rubinstein, M., Durand, F., & Freeman, W. T. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2461966 · Cited by 853
Self-citationInteractive digital photomontage
Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., & Cohen, M. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015718 · Cited by 829
Objective measures of speech quality
Quackenbush, S. (1995) · UPT. Syiah Kuala University Library (Syiah Kuala University) · Cited by 735
Image Sensors and Signal Processing for Digital Still Cameras
Nakamura, J. (2017) · CRC Press eBooks · doi:10.1201/9781420026856 · Cited by 530
Content-preserving warps for 3D video stabilization
Liu, F., Gleicher, M., Jin, H., & Agarwala, A. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531350 · Cited by 467
A phase-based approach to the estimation of the optical flow field using spatial filtering
Gautama, T., & Van Hulle, M. A. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2002.1031944 · Cited by 335
MODAL TESTING USING A SCANNING LASER DOPPLER VIBROMETER
STANBRIDGE, A. B., & EWINS, D. J. (1999) · Mechanical Systems and Signal Processing · doi:10.1006/mssp.1998.1209 · Cited by 302
Liu, C., Torralba, A., Freeman, W. T., Durand, F., & Adelson, E. H. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073223 · Cited by 301
Self-citationZalevsky, Z., Beiderman, Y., Margalit, I., Gingold, S., Teicher, M., Mico, V., & Garcia, J. (2009) · Optics Express · doi:10.1364/oe.17.021566 · Cited by 229
A multilevel banded graph cuts method for fast image segmentation
Lombaert, H., Sun, Y., Grady, L., & Xu, C. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.13 · Cited by 220
Riesz pyramids for fast phase-based video magnification
Wadhwa, N., Rubinstein, M., Durand, F., & Freeman, W. T. (2014) · 2014 IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccphot.2014.6831820 · Cited by 210
Self-citationCalibration-free rolling shutter removal
Grundmann, M., Kwatra, V., Castro, D., & Essa, I. (2012) · 2012 IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccphot.2012.6215213 · Cited by 172
Geometric Models of Rolling-Shutter CamerasPreprint
Meingast, M., Geyer, C., & Sastry, S. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0503076 · Cited by 165
SfM with MRFs: Discrete-Continuous Optimization for Large-Scale Structure from Motion
Crandall, D., Owens, A., Snavely, N., & Huttenlocher, D. P. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.218 · Cited by 157
Laser vibrometry: Pseudo-vibrations
Rothberg, S., Baker, J. R., & Halliwell, N. A. (1989) · Journal of Sound and Vibration · doi:10.1016/0022-460x(89)90705-0 · Cited by 114
Structural Modal Identification Through High Speed Camera Video: Motion Magnification
Chen, J. G., Wadhwa, N., Cha, Y., Durand, F., Freeman, W. T., & Büyüköztürk, O. (2014) · Conference proceedings of the Society for Experimental Mechanics · doi:10.1007/978-3-319-04753-9_19 · Cited by 65
Self-citationNew Image Processing Tools for Structural Dynamic Monitoring
Morlier, J., Salom, P., & Bos, F. (2007) · Key engineering materials · doi:10.4028/www.scientific.net/kem.347.239 · Cited by 25
Cited by
Speech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends
Schuller, B. W. (2018) · Communications of the ACM · doi:10.1145/3129340 · Cited by 541
Owens, A., Isola, P., McDermott, J., Torralba, A., Adelson, E. H., & Freeman, W. T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.264 · Cited by 346
Self-citationPhase-based frame interpolation for video
Meyer, S., Wang, O., Zimmer, H., Grosse, M., & Sorkine-Hornung, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298747 · Cited by 294
Metadata
- Authors
- Davis, A., Rubinstein, M., Wadhwa, N., Mysore, G. J., Durand, F., & Freeman, W. T.
- Venue
- ACM Transactions on Graphics
- Published
- 2014-07-27
- Volume
- 33
- Issue
- 4
- Pages
- 1-10
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W1977985044
- Cited by
- 359
- References
- 24
- FWCI
- 30.17
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.