Cited by 169Image Enhancement TechniquesImage and Video Quality AssessmentAdvanced Image Processing Techniques

Dehazing Evaluation: Real-World Benchmark Datasets, Criteria, and Baselines

Zhao, Shiyu, Zhang, Lin, Huang, Shuaiyi, Shen, Ying, Zhao, Shengjie·2020·IEEE Transactions on Image Processing

Abstract

On benchmark images, modern dehazing methods are able to achieve very comparable results whose differences are too subtle for people to qualitatively judge. Thus, it is imperative to adopt quantitative evaluation on a vast number of hazy images. However, existing quantitative evaluation schemes are not convincing due to a lack of appropriate datasets and poor correlations between metrics and human perceptions. In this work, we attempt to address these issues, and we make two contributions. First, we establish two benchmark datasets, i.e., the BEnchmark Dataset for Dehazing Evaluation (BeDDE) and the EXtension of the BeDDE (exBeDDE), which had been lacking for a long period of time. The BeDDE is used to evaluate dehazing methods via full reference image quality assessment (FR-IQA) metrics. It provides hazy images, clear references, haze level labels, and manually labeled masks that indicate the regions of interest (ROIs) in image pairs. The exBeDDE is used to assess the performance of dehazing evaluation metrics. It provides extra dehazed images and subjective scores from people. To the best of our knowledge, the BeDDE is the first dehazing dataset whose image pairs were collected in natural outdoor scenes without any simulation. Second, we provide a new insight that dehazing involves two separate aspects, i.e., visibility restoration and realness restoration, which should be evaluated independently; thus, to characterize them, we establish two criteria, i.e., the visibility index (VI) and the realness index (RI), respectively. The effectiveness of the criteria is verified through extensive experiments. Furthermore, 14 representative dehazing methods are evaluated as baselines using our criteria on BeDDE. Our datasets and relevant code are available at https://github.com/xiaofeng94/BeDDE-for-defogging.

Integrity signals

Registry

Crossref · 10.1109/tip.2020.2995264

Cites

65 of 65 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Zhao, S., Zhang, L., Huang, S., Shen, Y., & Zhao, S.
Affiliations
  • Tongji University (CN)
  • ShanghaiTech University (CN)
Venue
IEEE Transactions on Image Processing
Published
2020
Volume
29
Pages
6947-6962
ISSN
  • 1057-7149 (ISSN-L)
  • 1941-0042
OpenAlex
W3028365735
Cited by
169
References
65
Citations by year
2026: 22 · 2025: 36 · 2024: 34 · 2023: 37 · 2022: 22 · 2021: 14 · 2020: 4
FWCI
7.48
Type
journal article
Language
en
Open access
closed
Locations
IEEE Transactions on Image Processing (publishedVersion)
Funders
  • National Natural Science Foundation of China (61672380, 61973235, 61936014, 61972285)
  • Natural Science Foundation of Shanghai (19ZR1461300)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Enhancement Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
Keywords
image dehazing, benchmark dataset, quantitative evaluation, full-reference image quality assessment, visibility restoration, visibility index, outdoor scenes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review