A systematic review and scientific critique of methodology in modern urban heat island literature

Abstract

In the modern era of urban climatology, much emphasis has been placed on observing and documenting heat island magnitudes in cities around the world. Urban climate literature consequently boasts a remarkable accumulation of observational heat island studies. Through time, however, methodologists have raised concerns about the authenticity of these studies, especially regarding the measurement, definition and reporting of heat island magnitudes. This paper substantiates these concerns through a systematic review and scientific critique of heat island literature from the period 1950–2007. The review uses nine criteria of experimental design and communication to critically assess methodological quality in a sample of 190 heat island studies. Results of this assessment are discouraging: the mean quality score of the sample is just 50 percent, and nearly half of all urban heat island magnitudes reported in the sample are judged to be scientifically indefensible. Two areas of universal weakness in the literature sample are controlled measurement and openness of method: one‐half of the sample studies fail to sufficiently control the confounding effects of weather, relief or time on reported ‘urban’ heat island magnitudes, and three‐quarters fail to communicate basic metadata regarding instrumentation and field site characteristics. A large proportion of observational heat island literature is therefore compromised by poor scientific practice. This paper concludes with recommendations for improving method and communication in heat island studies through better scrutiny of findings and more rigorous reporting of primary research. Copyright © 2010 Royal Meteorological Society