Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 202 of the 207 references it cites were found in a registry or an index, and 1 are not checked yet.

On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID

Open access
Long, Yang, Xia, Gui-Song, Li, Shengyang, Yang, Wen, Yang, Michael Ying, Zhu, Xiao Xiang, Zhang, Liangpei, Li, Deren · 2021 · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

Abstract

The past years have witnessed great progress on remote sensing (RS) image interpretation and its wide applications. With RS images becoming more accessible than ever before, there is an increasing demand for the automatic interpretation of these images. In this context, the benchmark datasets serve as an essential prerequisites for developing and testing intelligent interpretation algorithms. After reviewing existing benchmark datasets in the research community of RS image interpretation, this article discusses the problem of how to efficiently prepare a suitable benchmark dataset for RS image interpretation. Specifically, we first analyze the current challenges of developing intelligent algorithms for RS image interpretation with bibliometric investigations. We then present the general guidances on creating benchmark datasets in efficient manners. Following the presented guidances, we also provide an example on building RS image dataset, i.e., Million Aerial Image Dataset (Online. Available: https://captain-whu.github.io/DiRS/0), a new large-scale benchmark dataset containing a million instances for RS image scene classification. Several challenges and perspectives in RS image annotation are finally discussed to facilitate the research in benchmark dataset construction. We do hope this article will provide the RS community an overall perspective on constructing large-scale and practical image datasets for further research, especially data-driven ones.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/jstars.2021.3070368

Cites

202 of 203 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Tag cloud of RS image interpretation.
Fig. 1 Tag cloud of RS image interpretation.CC-BY · From the PDF
Fig. 2 DiRS properties: di versity , r ichness , and s calability . DiRS formulates the basic properties which can be considered as basic desirability in the construction of datasets for RS image interpretation. We believe that these properties are complementary to each other. That is, the improvement of dataset in one aspect can simultaneously promote the dataset quality reflected in other properties.
Fig. 2 DiRS properties: di versity , r ichness , and s calability . DiRS formulates the basic properties which can be considered as basic desirability in the construction of datasets for RS image interpretation. We believe that these properties are complementary to each other. That is, the improvement of dataset in one aspect can simultaneously promote the dataset quality reflected in other properties.CC-BY · From the PDF
Fig. 3 General workflow of semiautomatic annotation in RS images.
Fig. 3 General workflow of semiautomatic annotation in RS images.CC-BY · From the PDF
Fig. 4 Hierarchical scene category network of Million-AID. All categories are hierarchically organized in a three-level tree: 51 leaf nodes fall into 28 parent nodes at the second level which are grouped into eight nodes at the first level, representing the 8 underlying scene categories of agriculture land, commercial land, industrial land, public service land, residential land, transportation land, unutilized land, and water area.
Fig. 4 Hierarchical scene category network of Million-AID. All categories are hierarchically organized in a three-level tree: 51 leaf nodes fall into 28 parent nodes at the second level which are grouped into eight nodes at the first level, representing the 8 underlying scene categories of agriculture land, commercial land, industrial land, public service land, residential land, transportation land, unutilized land, and water area.CC-BY · From the PDF
Fig. 5 Points of searched tennis courts shown in Google Earth Pro ( ©2020 Google LLC. ), where the top-left and bottom-right coordinates are ( 34 . 1071 ◦ N, 118 . 3605 ◦ W) and ( 33 . 9823 ◦ N, 118 . 3605 ◦ W), respectively. We consider the tennis courts as point ground features. The red marks show the searched locations of tennis courts. The eagle window shows the detail of a tennis court scene, which confirms the validity of collecting semantic coordinates by our proposed method.
Fig. 5 Points of searched tennis courts shown in Google Earth Pro ( ©2020 Google LLC. ), where the top-left and bottom-right coordinates are ( 34 . 1071 ◦ N, 118 . 3605 ◦ W) and ( 33 . 9823 ◦ N, 118 . 3605 ◦ W), respectively. We consider the tennis courts as point ground features. The red marks show the searched locations of tennis courts. The eagle window shows the detail of a tennis court scene, which confirms the validity of collecting semantic coordinates by our proposed method.CC-BY · From the PDF
Fig. 6 Points of wind turbines extracted from USWTDB and integrated in Google Earth Pro ( ©2020 Google LLC. ), where the geographic range is indicated with the top-left coordinates ( 41 . 2695 ◦ N, 90 . 3315 ◦ W) and bottom-right coordinates ( 41 . 1421 ◦ N, 90 . 0424 ◦ W). The eagle window shows the details of two wind turbines.
Fig. 6 Points of wind turbines extracted from USWTDB and integrated in Google Earth Pro ( ©2020 Google LLC. ), where the geographic range is indicated with the top-left coordinates ( 41 . 2695 ◦ N, 90 . 3315 ◦ W) and bottom-right coordinates ( 41 . 1421 ◦ N, 90 . 0424 ◦ W). The eagle window shows the details of two wind turbines.CC-BY · From the PDF
Fig. 7 River lines within a local area of China are extracted from OSM and displayed in Google Earth Pro ( ©2020 Google LLC. ), where the upper-left and bottom-right coordinates of geographic range are ( 32 . 2826 ◦ N, 111 . 1027 ◦ E) and ( 28 . 7477 ◦ N, 118 . 5372 ◦ E), respectively. The zoomed image shows the details of river lines.
Fig. 7 River lines within a local area of China are extracted from OSM and displayed in Google Earth Pro ( ©2020 Google LLC. ), where the upper-left and bottom-right coordinates of geographic range are ( 32 . 2826 ◦ N, 111 . 1027 ◦ E) and ( 28 . 7477 ◦ N, 118 . 5372 ◦ E), respectively. The zoomed image shows the details of river lines.CC-BY · From the PDF
Fig. 8 Illustration of searching scenes of airports around the world. An airport in OSM contains a large amount of tags, which can be employed to search airports with specific semantic key-value labels, e.g., aeroway and name . More than 5000 world airports in forms like way and relation can be obtained with accurate geographic coordinates, using English and Chinese semantic tags.
Fig. 8 Illustration of searching scenes of airports around the world. An airport in OSM contains a large amount of tags, which can be employed to search airports with specific semantic key-value labels, e.g., aeroway and name . More than 5000 world airports in forms like way and relation can be obtained with accurate geographic coordinates, using English and Chinese semantic tags.CC-BY · From the PDF
Fig. 9 Airport planes within a local area of the United States are extracted from OSM and shown in Google Earth Pro ( ©2020 Google LLC. ), where the upper-left and bottom-right coordinates of geographic range are ( 37 . 2262 ◦ N, 115 . 8819 ◦ W) and ( 32 . 6005 ◦ N, 110 . 6497 ◦ W), respectively. The zoomed image shows the accurate extent of an airport in a closed OSM way feature.
Fig. 9 Airport planes within a local area of the United States are extracted from OSM and shown in Google Earth Pro ( ©2020 Google LLC. ), where the upper-left and bottom-right coordinates of geographic range are ( 37 . 2262 ◦ N, 115 . 8819 ◦ W) and ( 32 . 6005 ◦ N, 110 . 6497 ◦ W), respectively. The zoomed image shows the accurate extent of an airport in a closed OSM way feature.CC-BY · From the PDF
Fig. 10 Illustration of the acquisition of RS scene images based on the collected geographic point, line, and area data. The points are set as the centers of scene square boxes. For line data, the center points are sampled by intervals. For plane data, scene square boxes are generated by mesh grids. The red frames indicate the generated scene square boxes which are consistent with the final scene image blocks.
Fig. 10 Illustration of the acquisition of RS scene images based on the collected geographic point, line, and area data. The points are set as the centers of scene square boxes. For line data, the center points are sampled by intervals. For plane data, scene square boxes are generated by mesh grids. The red frames indicate the generated scene square boxes which are consistent with the final scene image blocks.CC-BY · From the PDF
Image 11
CC-BY · From the PDF
Image 12
CC-BY · From the PDF
Image 13
CC-BY · From the PDF
Image 14
CC-BY · From the PDF
Image 15
CC-BY · From the PDF
Image 16
CC-BY · From the PDF
Image 17
CC-BY · From the PDF
Image 18
CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 207 references

Research works printed but not linked: 1

  1. maggiori, 0, Proc IEEE Int Geosci Remote Sens Symp, 3226

Cited by

See all 293

Metadata

Authors
Long, Y., Xia, G.-S., Li, S., Yang, W., Yang, M. Y., Zhu, X. X., Zhang, L., & Li, D.
Affiliations
  • Wuhan University (CN)
  • Chinese Academy of Sciences (CN)
  • International Institute for Geo-Information Science and Earth Observation (NL)
  • Deutsches Zentrum für Luft- und Raumfahrt e. V. (DLR) (DE)
Venue
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Published
2021
Volume
14
Pages
4205-4230
ISSN
  • 1939-1404 (ISSN-L)
  • 2151-1535
OpenAlex
W3152083889
Cited by
250
References
207
Citations by year
2026: 39 · 2025: 72 · 2024: 65 · 2023: 37 · 2022: 30 · 2021: 7
FWCI
15.59
Type
journal article
Language
en
Open access
Locations
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (publishedVersion, cc-by)
  • University of Twente Research Information (publishedVersion, cc-by)
  • elib (German Aerospace Center) (acceptedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • mediaTUM (Technical University of Munich) (submittedVersion, other-oa)
  • University of Twente Research Information (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Natural Science Foundation of China (61922065, 61771350, 41820104006, 61871299, 92038301)
  • Science and Technology Major Project of Hubei Province (2019AEA170)
  • German Federal Ministry of Education, and Research
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Remote-Sensing Image Classification
  • Social Sciences › Geography, Planning and Development › Remote Sensing and Land Use
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
benchmark dataset, remote sensing interpretation, aerial images, remote sensing scene classification, image annotation, dataset construction, bibliometric analysis, large-scale image databases

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID. https://app.cochat.ai/papers/10.1109/jstars.2021.3070368
The paper
Long, Y., Xia, G.-S., Li, S., Yang, W., Yang, M. Y., Zhu, X. X., Zhang, L., & Li, D. (2021). On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. https://doi.org/10.1109/jstars.2021.3070368

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.