Simulating Low-Cost Cameras for Augmented Reality Compositing
Abstract
Video see-through Augmented Reality adds computer graphics to the real world in real time by overlaying graphics onto a live video feed. To achieve a realistic integration of the virtual and real imagery, the rendered images should have a similar appearance and quality to those produced by the video camera. This paper describes a compositing method which models the artifacts produced by a small low-cost camera, and adds these effects to an ideal pinhole image produced by conventional rendering methods. We attempt to model and simulate each step of the imaging process, including distortions, chromatic aberrations, blur, Bayer masking, noise, sharpening, and color-space compression, all while requiring only an RGBA image and an estimate of camera velocity as inputs.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Applying camera artifacts to rendered objects makes augmented-reality objects blend well into real images.
“As a result, augmented-reality objects blended well into the real images; without this adjustment, the rendered objects stood out because they looked unnaturally clean and clear.”
From What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation? (2018) · cites this paper as Klein and Murray (2010)
Cites
(1981) · Cited by 11,963 · see citing works
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · see citing works
Self-citationPorter, T., & Duff, T. (1984) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/800031.808606 · Cited by 1,186 · see citing works
Porter, T., & Duff, T. (1984) · ACM SIGGRAPH Computer Graphics · doi:10.1145/964965.808606 · Cited by 728 · see citing works
Straight lines have to be straight
Devernay, F., & Faugeras, O. D. (2001) · Machine Vision and Applications · doi:10.1007/pl00013269 · Cited by 701 · see citing works
Improving the Agility of Keyframe-Based SLAM
Klein, G., & Murray, D. W. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88688-4_59 · Cited by 338 · see citing works
Self-citationStylized Augmented Reality for Improved Immersion
Fischer, J., Bartz, D., & Straßer, W. (2006) · Proceedings of the IEEE Virtual Reality Annual International Symposium · doi:10.1109/vr.2005.71 · Cited by 85 · see citing works
Classification of Illumination Methods for Mixed Reality
Jacobs, K., & Loscos, C. (2006) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2006.00816.x · Cited by 84 · see citing works
A Technique for Evaluation of CCD Video-Camera Noise
Irie, K., McKinnon, A. E., Unsworth, K., & Woodhead, I. M. (2008) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2007.913972 · Cited by 77 · see citing works
Using texture maps to correct for optical distortion in head-mounted displays
Watson, B. A., & Hodges, L. F. (2002) · doi:10.1109/vrais.1995.512493 · Cited by 72 · see citing works
Augmented reality based on estimation of defocusing and motion blurring from captured images
Okumura, B., Kanbara, M., & Yokoya, N. (2006) · IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2006.297817 · Cited by 37 · see citing works
Klein, G., & Murray, D. W. (2008) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2008.4637324 · Cited by 34 · see citing works
Self-citationStylized augmented reality for improved immersion
Fischer, J., Bartz, D., & Strasser, W. (2005) · doi:10.1109/vr.2005.1492774 · Cited by 25 · see citing works
Enhanced visual realism by incorporating camera image effects
Fischer, J., Bartz, D., & Straßer, W. (2006) · IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2006.297815 · Cited by 22 · see citing works
A loose and sketchy approach in a mediated reality environment
Haller, M., Landerl, F., & Billinghurst, M. (2005) · International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (GRAPHITE) · doi:10.1145/1101389.1101463 · Cited by 19 · see citing works
Watercolor inspired non-photorealistic rendering for augmented reality
Chen, J., Turk, G., & MacIntyre, B. (2008) · ACM Symposium on Virtual Reality Software and Technology (VRST) · doi:10.1145/1450579.1450629 · Cited by 13 · see citing works
Simulation of a Digital Camera Pipeline
Toadere, F. (2007) · International Symposium on Signals, Circuits and Systems · doi:10.1109/isscs.2007.4292789 · Cited by 6 · see citing works
Flagged first, then by citations
1 of the 18 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Billinghurst, M., Clark, A., & Lee, G. (2015) · now publishers, Inc. eBooks · doi:10.1561/9781601989215 · Cited by 536 · see citing works
Augmented reality technology in the manufacturing industry: A review of the last decade
Bottani, E., & Vignali, G. (2019) · IISE Transactions · doi:10.1080/24725854.2018.1493244 · Cited by 441 · see citing works
What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?
Mayer, N., Ilg, E., Fischer, P., Hazirbas, C., Cremers, D., Dosovitskiy, A., & Brox, T. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1082-6 · Cited by 191 · see citing works
Syed, T. A., Siddiqui, M. S., Abdullah, H. B., Jan, S., Namoun, A., Alzahrani, A., Nadeem, A., & Alkhodre, A. B. (2022) · Sensors · doi:10.3390/s23010146 · Cited by 235 · see citing works
Real-Time Camera Tracking: When is High Frame-Rate Best?
Handa, A., Newcombe, R. A., Angeli, A., & Davison, A. J. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33786-4_17 · Cited by 139 · see citing works
On rendering synthetic images for training an object detector
Rozantsev, A., Lepetit, V., & Fua, P. (2015) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2014.12.006 · Cited by 116 · see citing works
Visualization Techniques in Augmented Reality: A Taxonomy, Methods and Patterns
Zollmann, S., Langlotz, T., Grasset, R., Lo, W. H., Mori, S., & Regenbrecht, H. T. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.2986247 · Cited by 79 · see citing works
Colour Mapping: A Review of Recent Methods, Extensions and Applications
Faridul, H. S., Pouli, T., Chamaret, C., Stauder, J., Reinhard, E., Kuzovkin, D., & Trémeau, A. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12671 · Cited by 75 · see citing works
Visualization Techniques for Augmented Reality
Kalkofen, D., Sandor, C., White, S., & Schmalstieg, D. (2011) · Springer eBooks · doi:10.1007/978-1-4614-0064-6_3 · Cited by 60 · see citing works
Learning Lightprobes for Mixed Reality Illumination
Mandl, D., Yi, K. M., Mohr, P., Roth, P. M., Fua, P., Lepetit, V., Schmalstieg, D., & Kalkofen, D. (2017) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2017.25 · Cited by 56 · see citing works
Lee, M., Kim, H., & Paik, J. (2019) · IEEE Access · doi:10.1109/access.2019.2908451 · Cited by 52 · see citing works
Pursuit of “X-Ray Vision” for Augmented Reality
Livingston, M. A., Dey, A., Sandor, C., & Thomas, B. H. (2012) · Springer eBooks · doi:10.1007/978-1-4614-4205-9_4 · Cited by 45 · see citing works
Visuo-Haptic Mixed Reality with Unobstructed Tool-Hand Integration
Cosco, F., Garre, C., Bruno, F., Muzzupappa, M., & Otaduy, M. Á. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.107 · Cited by 39 · see citing works
Real-time illumination estimation from faces for coherent rendering
Knorr, S. B., & Kurz, D. (2014) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2014.6948416 · Cited by 37 · see citing works
Fast and Stable Color Balancing for Images and Augmented Reality
Oskam, T., Hornung, A., Sumner, R. W., & Groß, M. H. (2012) · International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission · doi:10.1109/3dimpvt.2012.36 · Cited by 29 · see citing works
Park, J., Kim, H., & Kim, L. (2012) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2223873 · Cited by 27 · see citing works
Reciprocal shading for mixed reality
Knecht, M., Traxler, C. T., Mattausch, O., & Wimmer, M. H. (2012) · Computers & Graphics · doi:10.1016/j.cag.2012.04.013 · Cited by 26 · see citing works
Analysis of Augmented Reality application based on cloud computing
Ming, C., Chen, L., & Zhang, W. (2011) · International Congress on Image and Signal Processing (CISP) · doi:10.1109/cisp.2011.6100311 · Cited by 23 · see citing works
Handling Motion-Blur in 3D Tracking and Rendering for Augmented Reality
Park, Y., Lepetit, V., & Woo, W. (2011) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.158 · Cited by 20 · see citing works
Wearable-Assisted Localization and Inspection Guidance System Using Egocentric Stereo Cameras
Qian, K., Zhao, W., Ma, Z., Ma, J., Ma, X., & Yu, H. (2017) · IEEE Sensors Journal · doi:10.1109/jsen.2017.2773487 · Cited by 20 · see citing works
Augmented reality systems for weed economic thresholds applications
Vidal, N. R., & Vidal, R. A. (2010) · Planta Daninha · doi:10.1590/s0100-83582010000200025 · Cited by 18 · see citing works
CAD-based Learning for Egocentric Object Detection in Industrial Context
Cohen, J., Crispim-Junior, C. F., Grange-Faivre, C., & Tougne, L. (2020) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0008975500002514 · Cited by 15 · see citing works
Preaching Voxels: An Alternative Approach to Mixed Reality
Regenbrecht, H. T., Park, J.-W. (., Ott, C., Mills, S. J., Cook, M., & Langlotz, T. (2019) · Frontiers in ICT · doi:10.3389/fict.2019.00007 · Cited by 14 · see citing works
Adaptive camera-based color mapping for mixed-reality applications
Knecht, M., Traxler, C. T., Purgathofer, W., & Wimmer, M. H. (2011) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2011.6092382 · Cited by 12 · see citing works
Neural Cameras: Learning Camera Characteristics for Coherent Mixed Reality Rendering
Mandl, D., Roth, P. M., Langlotz, T., Ebner, C., Mori, S., Zollmann, S., Mohr, P., & Kalkofen, D. (2021) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar52148.2021.00068 · Cited by 11 · see citing works
Real-Time Mixed Reality Rendering for Underwater 360° Videos
Thompson, S., Chalmers, A., & Rhee, T. J. (2019) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2019.00-24 · Cited by 10 · see citing works
Pixel-wise closed-loop registration in video-based augmented reality
Zheng, F., Schmalstieg, D., & Welch, G. (2014) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2014.6948419 · Cited by 8 · see citing works
Teaching Children Musical Perception with MUSIC-AR
Martins, V. F., Gomez, L., & Corrêa, A. G. D. (2015) · ICST Transactions on e-Education and e-Learning · doi:10.4108/el.2.5.e3 · Cited by 7 · see citing works
Perceptual Evaluation of Photo-realism in Real-time 3D Augmented Reality
Borg, M., Paprocki, M. M., & Madsen, C. B. (2014) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0004683203770386 · Cited by 5 · see citing works
Neural Bokeh: Learning Lens Blur for Computational Videography and Out-of-Focus Mixed Reality
Mandl, D., Mori, S., Mohr, P., Peng, Y., Langlotz, T., Schmalstieg, D., & Kalkofen, D. (2024) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr58804.2024.00106 · Cited by 5 · see citing works
Artificial Defocus for Displaying Markers in Microscopy Z-Stacks
Giusti, A., Taddei, P., Giorgio Corani, A., Gambardella, L. M., Magli, M. C., & Gianaroli, L. (2011) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.168 · Cited by 5 · see citing works
The Design and Implementation of On-Line Multi-User Augmented Reality Integrated System
Wang, H.-S., & Chiu, C.-W. (2011) · InTech eBooks · doi:10.5772/26505 · Cited by 5 · see citing works
Coherent Rendering of Virtual Smile Previews with Fast Neural Style Transfer
Vasiliu, V., & Sörös, G. (2019) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2019.00-25 · Cited by 4 · see citing works
Real Time 3D Face-Ear Recognition on Mobile Devices: New Scenarios for 3D Biometrics “in-the-Wild”
Nappi, M., Ricciardi, S., & Massimo Tistarelli, A. (2017) · Elsevier eBooks · doi:10.1016/b978-0-08-100705-1.00003-8 · Cited by 4 · see citing works
Non-Marker based Mobile Augmented Reality and its Applications using Object Recognition
Kim, D., & Hwang, D. (2020) · TUGraz OPEN Library (Graz University of Technology) · doi:10.3217/jucs-018-20-2832 · Cited by 4 · see citing works
Influence of animated reality mixing techniques on user experience
Zünd, F., Lancelle, M., Ryffel, M., Sumner, R. W., Mitchell, K., & Groß, M. H. (2014) · Motion in Games · doi:10.1145/2668064.2668088 · Cited by 3 · see citing works
Interactive multimedia for engineering tele-operation
Jailly, B., Preda, M., Gravier, C., & Fayolle, J. (2011) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2011.6012127 · Cited by 3 · see citing works
Prakash, S., Walton, D. R., dos Anjos, R. K., Steed, A. J., & Ritschel, T. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3549541 · Cited by 2 · see citing works
Spatio-Temporal Fish-Eye Image Processing Based on Neural Network
Wu, Y., & Zhang, L. (2020) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs49078.2020.9118472 · Cited by 2 · see citing works
Influence of animated reality mixing techniques on user experience
Zünd, F., Lancelle, M., Ryffel, M., Sumner, R. W., Mitchell, K., & Groß, M. H. (2014) · Motion in Games · doi:10.1145/2668084.2668088 · Cited by 2 · see citing works
Mimicking the In-Camera Color Pipeline for Camera-Aware Object CompositingPreprint
Gao, J., Li, X., Wang, L., Fidler, S., & Lin, S. C.-F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.11248 · Cited by 2 · see citing works
Hardware Accelerator for Feature Point Detection and Matching
Park, J., & Kim, L. (2014) · Springer eBooks · doi:10.1007/978-94-017-9075-8_8 · Cited by 1 · see citing works
Development and Application of the Virtual Intelligent Digital Camera Teaching System
Yang-na, S. (2011) · Advances in intelligent and soft computing · doi:10.1007/978-3-642-25986-9_82
Real-Time Blur Rendering of Moving Objects in an Augmented Reality Environment
Yue, Z., Jiao, L. J., Ni, L. Z., & Ma, J. (2014) · International Conference on Intelligent Human-Machine Systems and Cybernetics · doi:10.1109/ihmsc.2014.136
Adaptive camera-based color mapping for mixed-reality applications
Knecht, M., Traxler, C. T., Purgathofer, W., & Wimmer, M. H. (2011) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar.2011.6162884
Parameter-Free Neural Lens Blur Rendering for High-Fidelity Composites
Ruan, L., Chen, B., & Rhee, T. J. (2025) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar67309.2025.00145
- Authors
- Klein, G., & Murray, D. W.
- Affiliations
- University of Oxford (GB)
- ORCID
- Murray, D.W.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2010-05
- Volume
- 16
- Issue
- 3
- Pages
- 369-380
- ISSN
- 1077-2626 (ISSN-L)
- OpenAlex
- W2120943485
- Cited by
- 47
- References
- 18
- Citations by year
- 2025: 2 · 2024: 1 · 2022: 1 · 2021: 1 · 2020: 4 · 2019: 5 · 2018: 3 · 2017: 3 · 2015: 4 · 2014: 7 · 2012: 6
- FWCI
- 5.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Visualization and Computer Graphics (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Engineering and Physical Sciences Research Council (GR/S97774/01, EP/D037077/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Augmented Reality Applications
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Physical Sciences › Engineering › Media Technology › Advanced Optical Imaging Technologies
- Keywords
- augmented reality, video see-through, image compositing, lens distortion, chromatic aberration, blur, noise, sharpening
- MeSH terms
- Computer Graphics, Computer Simulation, Image Interpretation, Computer-Assisted, Models, Theoretical, Photography, User-Computer Interface, Video Recording, Equipment Failure Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
