Multiscale structural similarity for image quality assessment
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index; none of the 14 sources its claims cite is checked for support yet.
Abstract
The structural similarity image quality paradigm is based on the assumption that the human visual system is highly adapted for extracting structural information from the scene, and therefore a measure of structural similarity can provide a good approximation to perceived image quality. This paper proposes a multiscale structural similarity method, which supplies more flexibility than previous single-scale methods in incorporating the variations of viewing conditions. We develop an image synthesis method to calibrate the parameters that define the relative importance of different scales. Experimental comparisons demonstrate the effectiveness of the proposed method.
Overview
CoChat Summary
TL;DR
This paper proposes a multi-scale structural similarity (MS-SSIM) image quality method that combines contrast and structure comparisons across scales and luminance comparison at the coarsest scale. On the LIVE JPEG/JPEG2000 database, MS-SSIM outperformed PSNR, the Sarnoff model, and single-scale SSIM variants on all reported evaluation criteria.
Key findings
- MS-SSIM achieved the best reported performance: CC 0.969, ROCC 0.966, MAE 3.86, RMS 4.91, and OR 1.16%, compared with the best single-scale SSIM at M=2 (CC 0.963, ROCC 0.959, MAE 4.21, RMS 5.38, OR 2.62%) and PSNR (CC 0.905, ROCC 0.901, MAE 6.53, RMS 8.45, OR 15.7%).
- Single-scale SSIM performance varied with scale; the best single-scale result was at M=2, and single-scale SSIM tended to give higher scores as scale increased.
- The calibrated cross-scale exponents were β1=γ1=0.0448, β2=γ2=0.2856, β3=γ3=0.3001, β4=γ4=0.2363, and α5=β5=γ5=0.1333.
- The authors conclude that multi-scale SSIM provides a meaningful balance between scales and outperforms the best single-scale SSIM and state-of-the-art image quality metrics.
Methods
- The MS-SSIM system iteratively applies a low-pass filter and downsamples by 2; at each scale it computes contrast and structure comparisons, computes luminance comparison only at Scale M, and combines them using exponents normalized so that ∑γj=1.
- Cross-scale parameters were calibrated by image synthesis: 5 scales and 12 distortion levels (range 2^3 to 2^14) were used with 10 original 64×64 images, producing 600 distorted images; 8 subjects selected images of equal perceived quality across scales at a viewing distance of 32 pixels per degree of visual angle.
- Evaluation used the LIVE database with 344 JPEG and JPEG2000 compressed images (bit rate 0.028 to 3.150 bits/pixel), MOS from 13–25 subjective scores, and compared PSNR, Sarnoff JNDmetrix 8.0, single-scale SSIM for M=1–5, and MS-SSIM using CC, MAE, RMS, OR, and ROCC after logistic mapping.
Limitations
- Stated: the image-synthesis calibration approach is “still rather crude,” and the authors are working on a more systematic approach.
- Stated: CSF cannot be directly used to derive the parameters because it is typically measured at visibility threshold with simplified stimuli, whereas the goal is comparing complex structured images at visible distortion levels.
- Evident from the text: calibration used only 10 original 64×64 grayscale images and 8 subjects, including one author, and evaluation was limited to the LIVE JPEG/JPEG2000 database.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 1 Not valid · 14 Not checked · 18 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It can also be observed that the single-scale model tends to supply higher scores with the increase of scales.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Finally, for every one of the objective evaluation criteria, multi-scale SSIM model outperforms all the other models, including the best single-scale SSIM model, suggesting a meaningful balance between scales.”
- Also stated in: § 5. DISCUSSIONS · p. 4 · hedged
“Experiments show that with an appropriate parameter settings, the multi-scale method outperforms the best single-scale SSIM model as well as state-of-the-art image quality metrics.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 5. DISCUSSIONS · p. 4 · hedged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In general, the subjects agreed with each other on each image more than they agreed with themselves across different images.”
plainstates an associationn = 8- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The resulting parameters we obtained are β 1 = γ 1 = 0.0448, β 2 = γ 2 = 0.2856, β 3 = γ 3 = 0.3001, β 4 = γ 4 = 0.2363, and α 5 = β 5 = γ 5 = 0.1333, respectively.”
plainn = 8- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, the SSIM index algorithm introduced in [5] is a single-scale approach.”
What the sources say
- Not matchedcites [5] [5] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, 'Image quality assessment: From error measurement to structural similarity,' IEEE Trans. Image Processing , vol. 13, Jan. 2004. -
More than one paper fits this citation.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
10 more claims
Create a free account to see all 33 claims and every source verdict.
- It may state two separate claims as one. 8
- Probably not a substantial claim. 13
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Further, as research in quality assessment has demonstrated, incorporating multiscale information when assessing quality produces QA algorithms that perform better in terms of correlation with human perception [13], [29].”
From No-Reference Image Quality Assessment in the Spatial Domain (2012) · cites this paper as [13]
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Self-citationA universal image quality index
Wang, Z., & Bovik, A. C. (2002) · IEEE Signal Processing Letters · doi:10.1109/97.995823 · Cited by 5,757
Self-citationImage quality measures and their performance
Eskicioglu, A. M., & Fisher, P. S. (1995) · IEEE Transactions on Communications · doi:10.1109/26.477498 · Cited by 2,117 · cited 1×
“Traditional perceptual image quality assessment methods are based on a bottom-up approach which attempts to simulate the functionality of the relevant early human visual system (HVS) components.”
BackgroundA Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients
Portilla, J., & Simoncelli, E. P. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026553619983 · Cited by 1,806 · cited 2×
“These synthesis methods provide a powerful and efficient means of testing a model, and have the added benefit that the resulting images suggest improvements that might be made to the model [11].”
MethodSelf-citation(1995) · Medical Entomology and Zoology · Cited by 1,212
Pyramid-based texture analysis/synthesis
Heeger, D. J., & Bergen, J. R. (1995) · Proceedings of the 22nd annual conference on Computer graphics and interactive techniques - SIGGRAPH '95 · doi:10.1145/218380.218446 · Cited by 1,095
Video quality assessment based on structural distortion measurement
Wang, Z., Lu, L., & Bovik, A. C. (2004) · Signal Processing: Image Communication · doi:10.1016/s0923-5965(03)00076-6 · Cited by 1,024
Self-citationImage Quality Assessment: From Error Measurement to Structural Similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · Cited by 919
Self-citationTeo, P. C., & Heeger, D. J. (2002) · Proceedings of 1st International Conference on Image Processing · doi:10.1109/icip.1994.413502 · Cited by 471
Objective video quality assessment
(2003) · Handbook of Video Databases · doi:10.1201/9780203489864-45 · Cited by 359
Research works printed but not linked: 2
- 0, JNDmetrix Technology, Sarnoff Corporation
- 2000, Final report from the video quality experts group on the validation of objective models of video quality assessment, VQEG
Cited by
Items 301–400 of 6,047
Deep Learning Single-Frame and Multiframe Super-Resolution for Cardiac MRI
Masutani, E. M., Bahrami, N., & Hsiao, A. (2020) · Radiology · doi:10.1148/radiol.2020192173 · Cited by 130
“Specifically, our loss function was the sum of L1 loss and a modified form of the multiscale structural similarity index (SSIM) loss (defined as 1 minus multiscale SSIM) (27) (Appendix E1 [online]).”
MethodRate Bounds on SSIM Index of Quantized Images
Channappayya, S. S., Bovik, A. C., & Heath, R. W. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.2001400 · Cited by 133
“These new IQA algorithms include, among others, the structural similarity (SSIM) index [39], or Wang-Bovik index, and its variants [41], [ 42 ], the Visual Information Fidelity Criterion (VIF) [30] and the Visual Signal to Noise Ratio (VSNR) [4].”
MethodSelf-citationAccelerated MRI With Un-Trained Neural Networks
Darestani, M. Z., & Heckel, R. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3097596 · Cited by 173
Shen, K., Zhou, X., & Liu, Z. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3366221 · Cited by 153
“Here, we adopt the binary cross entropy (BCE) loss and structural similarity index (SSIM) loss [31] to compute the loss function.”
MethodFuture Frame Prediction Using Convolutional VRNN for Anomaly Detection
Lu, Y., Kumar, K. M. A., Nabavi, S. S., & Wang, Y. (2019) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2019.8909850 · Cited by 154
“Model Learning: For learning parameters in Conv-VRNN, we combine the least absolute deviation (L1 loss) [24], multi-scale structural similarity measurement (msssim loss) [29] and gradient difference (gdl loss) [21] to define a loss that measure the quality of the predicted frame.”
MethodMorphological Feature Visualization of Alzheimer’s Disease via Multidirectional Perception GAN
Wen, Y., Lei, B., Wang, S., Liu, Y., Feng, Z., Hu, Y. S., Shen, Y., & Ng, M. K. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3118369 · Cited by 135
Towards Benchmarking Scene Background Initialization
Maddalena, L., & Petrosino, A. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23222-5_57 · Cited by 131
“7) MultiScale Structural Similarity Index (MS-SSIM): This is the metric defined in [4], that uses structural”
MethodNo-reference image quality assessment based on log-derivative statistics of natural scenes
Zhang, Y., & Chandler, D. M. (2013) · Journal of Electronic Imaging · doi:10.1117/1.jei.22.4.043025 · Cited by 128
Statistical Modeling of Craniofacial Shape and Texture
Dai, H., Pears, N. E., Smith, W. A. P., & Duncan, C. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01260-7 · Cited by 155
“We use several performance metrics, namely: SSIM (Wang et al. 2004), MS-SSIM (Wang et al. 2003) and IW-SSIM (Wang and Li 2011) to measure the texture map quantitatively.”
MethodZhang, W., Borji, A., Wang, Z., Callet, P. L., & Liu, H.-T. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2461603 · Cited by 137
Self-citationDCT statistics model-based blind image quality assessment
Saad, M. A., Bovik, A. C., & Charrier, C. (2011) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2011.6116319 · Cited by 158
Self-citationLearning Physics-Guided Face Relighting Under Directional Light
Nestmeyer, T., Lalonde, J., Matthews, I., & Lehrmann, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00517 · Cited by 171
“Another prevailing metric of image quality assessment is structural similarity (SSIM) [48] and its multi-scale variant (MS-SSIM) [49].”
BackgroundSpatiotemporal Feature Integration and Model Fusion for Full Reference Video Quality Assessment
Bampis, C. G., Li, Z., & Bovik, A. C. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2868262 · Cited by 128
Self-citationLearned Convolutional Sparse Coding
Hillel, S., & Giryes, R. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462313 · Cited by 150
“L(x, x̂) = α(1−ms ssim(x, x̂))+(1−α)||x− x̂||1, (15) where ms ssim is the multiscale SSIM loss [21].”
Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis
Dorjsembe, Z., Pao, H.-K., Odonchimed, S., & Xiao, F. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3385504 · Cited by 248
“Besides the 3D-FID metric, we utilized Mean Squared Error (MSE), Maximum Mean Discrepancy (MMD) score [44], and Multi-Scale Structural Similarity Index Measure (MS-SSIM) [45] for quantitative evaluation.”
MethodQuality of Experience for Streaming Services: Measurements, Challenges and Insights
Bouraqia, K., Sabir, E., Sadik, M., & Ladid, L. (2020) · IEEE Access · doi:10.1109/access.2020.2965099 · Cited by 167
“approaches are Peak Signal toNoise Ratio (PSNR), Structural Similarity Metric (SSIM) [33], Multi-Scale Structural SiMilarity [34], SSIMplus [35](supports cross frame rate and cross resolution), Video Quality Model (VQM) [36], and Natural Image Quality Evaluator (NIQE) [37].”
MethodWireless Video Quality Assessment: A Study of Subjective Scores and Objective Algorithms
Moorthy, A. K., Seshadrinathan, K., Soundararajan, R., & Bovik, A. C. (2010) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2010.2041829 · Cited by 124
“SW-SSIM, VQM, and MS-SSIM seem to perform the best across distortion types, while VQM, MSSSIM, and P-SSIM seem to do well across compression rates.”
BackgroundMethodInfluentialSelf-citationVector Quantized Semantic Communication System
Fu, Q., Xie, H., Qin, Z., Slabaugh, G., & Tao, X. (2023) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2023.3255221 · Cited by 155
“In particular, we test the performance of the following methods and evaluated results using the MS-SSIM [14] • UNet-DeepJSCC: The remaining 4-layer VQ-DeepSC structures without vector quantization, which transmits images in the form of direct features transmition and compress the transmitted…”
MethodDIBR synthesized image quality assessment based on morphological wavelets
Sandic-Stankovic, D. D., Kukolj, D., & Callet, P. L. (2015) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2015.7148143 · Cited by 121
Krasula, L., Fliegel, K., Callet, P. L., & Klima, M. (2016) · International Workshop on Quality of Multimedia Experience · doi:10.1109/qomex.2016.7498936 · Cited by 111
Image-Difference Prediction: From Grayscale to Color
Lissner, I., Preiss, J., Urban, P., Lichtenauer, M. S., & Zolliker, P. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2216279 · Cited by 133
“TABLE I PARAMETERS α i AND c i OF OUR MODEL IN (17) Scale weights of the multiscale SSIM index [3] . α 1 α 2 α 3 α 4 α 5 0.0448 0.2856 0.3001 0.2363 0.”
MethodInfluentialObjective Quality Assessment for Image Retargeting Based on Structural Similarity
Fang, Y., Zeng, K., Wang, Z., Lin, W., Fang, Z., & Lin, C. (2014) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2014.2298919 · Cited by 126
Self-citationFang, Y., Yan, J.-B., Li, L.-D., Wu, J.-J., & Lin, W. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2781307 · Cited by 109
“In order to overcome the shortcomings of these methods, many advanced full reference (FR) methods which require complete reference information for image quality prediction with consideration of the characteristics of the HVS have been proposed, including structure similarity (SSIM) [6] and its variants: multi-scale SS…”
BackgroundEvent Enhanced High-Quality Image Recovery
Wang, B., He, J., Yu, L., Xia, G.-S., & Yang, W. T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_10 · Cited by 175
“For the metrics, we use PSNR and SSIM [34] for quantitative comparison while the visual effect of the reconstructed images for qualitative comparison.”
MethodAttentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network
Lao, S.-S., Gong, Y., Shi, S., Yang, S.-D., Wu, T., Wang, J., Xia, W., & Yang, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00123 · Cited by 146
Tensor Oriented No-Reference Light Field Image Quality Assessment
Zhou, W., Shi, L., Chen, Z., & Zhang, J. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2969777 · Cited by 133
“Except for SSIM, several variants of SSIM have been proposed, such as MSSSIM [11], FSIM [12] and IWSSIM [13].”
InfluentialA Fast QTMT Partition Decision Strategy for VVC Intra Prediction
Fan, Y., Chen, J., Sun, H., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3000565 · Cited by 122
“Despite the frequently adopted BDBR metric, MS-SSIM (Multiscale Structural Similarity) [28] and VMAF (Visual MultimethodAssessment Fusion) [29] are also used to further verify the performance of our method.”
MethodDeep Learning-Based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression
Liu, H., Zhang, Y., Zhang, H., et al. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2933743 · Cited by 121
“, SSIM (MSSSIM) [27], FSIM [28], GMSD [29], and VSI [30].”
Deep-learning seismic full-waveform inversion for realistic structural models
Liu, B., Yang, S., Ren, Y., Xu, X., Jiang, P., & Chen, Y. (2020) · Geophysics · doi:10.1190/geo2019-0435.1 · Cited by 130
No-Reference Point Cloud Quality Assessment via Domain Adaptation
Yang, Q., Liu, Y., Chen, S., Xu, Y., & Sun, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02050 · Cited by 133
“The motivation of H-SCNN is that multiscale features are canonical for both natural images and point clouds in terms of the full-reference IQA and PCQA [34] [39] [38].”
BackgroundMethodNaturalness-Aware Deep No-Reference Image Quality Assessment
Yan, B., Bare, B., & Tan, W. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2904879 · Cited by 131
“Among them, multi-scale SSIM (MS-SSIM) [27] and FSIM [12] are widely used in image processing community.”
BackgroundMethodRethinking Low-Light Enhancement via Transformer-GAN
Yang, S., Zhou, D., Cao, J., & Guo, Y. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3167331 · Cited by 125
Multi-exposure image fusion: A patch-wise approach
Ma, K., & Wang, Z. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7351094 · Cited by 125
“The model in [18] is based on the multi-scale structural similarity (SSIM) framework [29,30].”
MethodSelf-citationJiang, Q., Liu, Z., Gu, K., Shao, F., Zhang, X., Liu, H., & Lin, W. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3154588 · Cited by 133
“MS-SSIM [18] and IW-SSIM [19] improved SSIM from the perspectives of multi-scale mechanism and information content weighting, respectively.”
MethodThe GAN Landscape: Losses, Architectures, Regularization, and NormalizationPreprint
Kurach, K., Lučić, M., Zhai, X., Michalski, M., & Gelly, S. (2018) · arXiv (Cornell University) · Cited by 149
Improving Deep Video Compression by Resolution-Adaptive Flow Coding
Hu, Z., Chen, Z., Xu, D., Lu, G. S., Ouyang, W., & Gu, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_12 · Cited by 161
“D = d(Xt, X̂t) denotes the distortion between the input frame and the reconstructed frame, where d(·) represents the metric (mean square error or MS-SSIM [31]) for measuring the difference between two images.”
BackgroundMethodDeep One-Class Classification via Interpolated Gaussian Descriptor
Chen, Y., Tian, Y., Pang, G., & Carneiro, G. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19915 · Cited by 171
“1333 (Wang, Simoncelli, and Bovik 2003).”
MethodInfluentialMLIC: Multi-Reference Entropy Model for Learned Image Compression
Jiang, W., Yang, J., Zhai, Y., Ning, P., Gao, F., & Wang, R. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611694 · Cited by 213
OmniDet: Surround View Cameras Based Multi-Task Visual Perception Network for Autonomous Driving
Ravi Kumar, V., Yogamani, S., Rashed, H., Sitsu, G., Witt, C., Leang, I., Milz, S., & Mader, P. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3062324 · Cited by 121
Shao, F., Li, K., Lin, W., Jiang, G., Yu, M., & Dai, Q. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2436332 · Cited by 116
A game-based approximate verification of deep neural networks with provable guarantees
Wu, M., Wicker, M., Ruan, W., Huang, X., & Kwiatkowska, M. (2019) · Theoretical Computer Science · doi:10.1016/j.tcs.2019.05.046 · Cited by 127
“While we focus on Lk distances, including L1 (Manhattan distance), L2 (Euclidean distance), and L∞ (Chebyshev distance), we emphasise that the results 115 of this paper hold for any distance metric and can be adapted to image similarity distances such as SSIM [24].”
BackgroundDual-Stream Interactive Networks for No-Reference Stereoscopic Image Quality Assessment
Zhou, W., Chen, Z., & Li, W. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2902831 · Cited by 117
Virtual reality: A survey of enabling technologies and its applications in IoT
Hu, M., Luo, X., Chen, J., Lee, Y. C., Zhou, Y., & Wu, D. (2021) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102970 · Cited by 144
“To rectify this, multi-scale, structure similarity (MS-SSIM) has also been defined [97].”
MethodDSSLIC: Deep Semantic Segmentation-based Layered Image Compression
Akbari, M. A., Liang, J., & Han, J. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683541 · Cited by 141
“It is shown that when the bit rate is less than about 1.4 bpp, our scheme achieves better results than other codes in both PSNR and MS-SSIM.”
MethodInfluentialUnderwater Image Enhancement With Lightweight Cascaded Network
Jiang, N., Chen, W., Lin, Y., Zhao, T., & Lin, C. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3115442 · Cited by 123
“In view of quality assessment, a user measures the quality of an image based on its multi-scale fidelities and textures [51], which inspires us to enhance images at different scales.”
BackgroundUSOD10K: A New Benchmark Dataset for Underwater Salient Object Detection
Hong, L., Wang, X., Zhang, G., & Zhao, M. L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3266163 · Cited by 170
“Restrictions apply. where L bce , L iou , L dice , and L ssim represent BCE loss [50], IoU loss [51], Dice loss [52], and SSIM loss [53], respectively.”
Tran, Q.-K., & Song, S.-K. (2019) · Atmosphere · doi:10.3390/atmos10050244 · Cited by 127
“Besides the commonly used MAE and MSE measures, we followed Zhao et al. [25] in using SSIM and MS-SSIM measures [35–37] to implement combined loss functions.”
MethodHyperInverter: Improving StyleGAN Inversion via Hypernetwork
Dinh, T. M., Tran, A. T., Nguyen, R. M. H., & Hua, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01110 · Cited by 152
“Specifically, we use the pixel-wise L2, perceptual LPIPS [56], MS-SSIM [48], and PSNR metrics.”
BackgroundMethodNo-Reference Quality Assessment for Stereoscopic Images Based on Binocular Quality Perception
Ryu, S., & Sohn, K. (2013) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2013.2279971 · Cited by 112
ELF-VC: Efficient Learned Flexible-Rate Video Coding
Rippel, O., Anderson, A. G., Tatwawadi, K., Nair, S., Lytle, C., & Bourdev, L. D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01421 · Cited by 128
“We found that it is easy for ML codecs to perform well on MS-SSIM and VMAF by the numbers, but that perceptual quality is not commensurate with these gains.”
MethodInfluentialA Survey of Synthetic Data Augmentation Methods in Machine Vision
Mumuni, A., Mumuni, F., & Gerrar, N. K. (2024) · Machine Intelligence Research · doi:10.1007/s11633-022-1411-7 · Cited by 201
“Game engines provide more advance modeling capabilities that can be used to create large, interactive scenes and virtual environments that span whole cities.”
BackgroundFSITM: A Feature Similarity Index For Tone-Mapped Images
Nafchi, H. Z., Shahkolaei, A., Moghaddam, R. F., et al. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2381458 · Cited by 125
“Although the TMQI clearly provides better assessment for tone-mapped images than the well-known image quality assessment metrics, like SSIM [9], MS-SSIM [10], and FSIM [11], its performance is not perfect.”
MethodQiu, B., Huang, Z., Liu, X., Meng, X., You, Y., Liu, G., Yang, K., Maier, A., Ren, Q., & Lu, Y. (2020) · Biomedical Optics Express · doi:10.1364/boe.379551 · Cited by 119
“Four popular performance indices were adopted as such metrics, namely, peak signal-to-noise ratio (PSNR), SSIM [32], MS-SSIM [36], and mean squared error (MSE).”
MethodNOVA: QoE-driven optimization of DASH-based video delivery in networks
Joseph, V., & de Veciana, G. (2014) · IEEE INFOCOM 2014 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2014.6847927 · Cited by 110
“We obtained proxy subjective VQA metric for the representations based on the corresponding value of MSSSIM-Y metric ([22]).”
MethodDeep Architectures for Image Compression: A Critical Review
Mishra, D., Singh, S. K., & Singh, R. K. (2021) · Signal Processing · doi:10.1016/j.sigpro.2021.108346 · Cited by 130
Chen, H., Huang, L., Liu, T., & Özcan, A. (2022) · Light Science & Applications · doi:10.1038/s41377-022-00949-8 · Cited by 125
“The success of the reconstruction performance of FIN was also quantified using the structural similarity index (SSIM) 49 across all four FIN networks that were trained using different types of samples (see Fig.”
Display device-adapted video quality-of-experience assessment
Rehman, A., Zeng, K., & Wang, Z. (2015) · SPIE Proceedings · doi:10.1117/12.2077917 · Cited by 110
Self-citationEvaluating Quality of Screen Content Images Via Structural Variation Analysis
Gu, K., Qiao, J., Min, X., Yue, G., Lin, W., & Thalmann, D. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2771284 · Cited by 112
“In [10], [11], [12], Wang et al.”
Unsupervised video anomaly detection via normalizing flows with implicit latent features
Cho, M., Kim, T., Kim, W. J., Cho, S., & Lee, S. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108703 · Cited by 132
“We also add multi-scale SSIM Lms−ssim and gradient loss Lgrad to maintain the sharpness of the frames, which computes the difference of gradient at each pixel between the input and output frames.”
MethodNon-distortion-specific no-reference image quality assessment: A survey
Manap, R. A., & Shao, L. (2015) · Information Sciences · doi:10.1016/j.ins.2014.12.055 · Cited by 120
A Novel Image Quality Assessment With Globally and Locally Consilient Visual Quality Perception
Bae, S.-H., & Kim, M. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2545863 · Cited by 107
“…y ( j ) (12) where w(x, y) is the local weight based on visual priors with respect to a local importance (e.g., degree of image content information [9] , phase congruency [13], visual saliency index [14] , etc.) and W in (12) is a normalization factor and is the summation of all w(x, y) values…”
BackgroundMethodInfluentialFace Hallucination Using Cascaded Super-Resolution and Identity Priors
Grm, K., Scheirer, W. J., & Štruc, V. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2945835 · Cited by 125
“We therefore design a new loss function for our SR network around the structural similarity index (SSIM, [29]), and integrate it directly into our learning algorithm.”
MethodSPSIM: A Superpixel-Based Similarity Index for Full-Reference Image Quality Assessment
Sun, W., Liao, Q.-M., Xue, J.-H., & Zhou, F. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2837341 · Cited by 89
“We compare the proposed method with nine well-known FR IQA approaches, namely, PSNR, SSIM, multi-scale SSIM (MS-SSIM) [45], VIF, MAD, IW-SSIM, FSIMc, GMSD,”
MethodDeep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications
Kim, S. Y., Oh, J., & Kim, M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00321 · Cited by 109
Deep Learning for Simultaneous Seismic Image Super-Resolution and Denoising
Li, J., Wu, X., & Hu, Z. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3057857 · Cited by 167
Towards Image Understanding from Deep Compression without Decoding
Torfason, R., Mentzer, F., Agustsson, E., Tschannen, M., Timofte, R., & Gool, L. V. (2018) · Lirias · doi:10.48550/arxiv.1803.06131 · Cited by 199
“However for SSIM and MS-SSIM the joint ft. improves more than the compression ft.”
BackgroundMethodInfluentialEnd-to-End Optimized ROI Image Compression
Cai, C., Chen, L., Zhang, X., & Gao, Z. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2960869 · Cited by 128
“is the most commonly used quality metric in recent literatures on deep-learning compression [24]–[28], [31], because it is more in line with human vision system and it can measure the perceptual quality in some degree [52], [64].”
BackgroundMethodInfluentialA Comprehensive Survey and Taxonomy on Single Image Dehazing Based on Deep Learning
Gui, J., Cong, X., Cao, Y., Ren, W., Zhang, J., Zhang, J., Cao, J., & Tao, D. (2022) · ACM Computing Surveys · doi:10.1145/3576918 · Cited by 145
Subjective and Objective Quality Assessment of Compressed 4K UHD Videos for Immersive Experience
Cheon, M., & Lee, J. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2683504 · Cited by 116
“, peak signal-tonoise ratio (PSNR), structural similarity (SSIM) [29], multiscale structural similarity (MS-SSIM) [7], visual information fidelity (VIF) [9], visual signal-to-ratio (VSNR) [8], video quality model (VQM) [15], motion-based video integrity evaluation index (MOVIE) [14], etc.”
BackgroundMethodInfluentialKim, K. H., Do, W., & Park, S. (2018) · Medical Physics · doi:10.1002/mp.12945 · Cited by 107
Ma, Y., Wei, B., Feng, P., He, P., Guo, X., & Wang, G. (2020) · IEEE Access · doi:10.1109/access.2020.2986388 · Cited by 114
“Also, SMGAN [38] based on WGAN-GP combined multiscale structural loss [39] and L1 loss into the objective function, and took advantage of the information between adjacent inter-slices, produced results compared favorably with that of WGAN-VGG [35].”
ResultDigital hair removal by deep learning for skin lesion segmentation
Li, W., Raj, A. N. J., Tjahjadi, T., & Zhuang, Z. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.107994 · Cited by 116
Bai, Y., Wu, W., Yang, Z., Yu, J., Zhao, B., Liu, X., Yang, H., Mas, E., & Koshimura, S. (2021) · Remote Sensing · doi:10.3390/rs13112220 · Cited by 140
“Each loss is comprised of three parts: focal loss [49], Structural SIMilarity (SSIM) loss [50], and Intersection over Union (IoU) loss [51]:”
BackgroundCoarse-To-Fine Deep Video Coding with Hyperprior-Guided Mode Prediction
Hu, Z., Lu, G. S., Guo, J., Liu, S., Jiang, W., & Xu, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00583 · Cited by 146
“PSNR and MS-SSIM [43] are used to evaluate the video quality.”
MethodInfluentialMCL-V: A streaming video quality assessment database
Lin, J. Y., Song, R., Wu, C.-H., Liu, T., Wang, H., & Kuo, C.-C. J. (2015) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2015.02.012 · Cited by 106
“Several full-reference (FR) IQA and VQA algorithms [57–64] are evaluated against the collected MOS of the MCL-V database and reported in this subsection.”
MethodInfluentialAnomaly Detection in Videos Using Optical Flow and Convolutional Autoencoder
Duman, E., & Erdem, O. A. (2019) · IEEE Access · doi:10.1109/access.2019.2960654 · Cited by 115
“After the training stage, a reconstruction error between an optical flow volume and corresponding reconstructed optical flow volume for each eight consecutive video frames is computed according to the multi-scale structural similarity (MS-SSIM) [35] quality metric.”
MethodInfluentialNo-Reference Light Field Image Quality Assessment Based on Spatial-Angular Measurement
Shi, L., Zhou, W., Chen, Z., & Zhang, J. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2955011 · Cited by 114
“For 2D IQA, several 2D FR-IQA metrics have been proposed, for example, structural similarity (SSIM) [18], multiscale SSIM (MS-SSIM) [19], feature similarity (FSIM) [20], information content weighting SSIM (IWSSIM) [21], internal generative mechanism (IGM) [22], visual saliency-induced index (VSI) [23] and gradient mag…”
MethodDeep Learning Based End-to-End Wireless Communication Systems Without Pilots
Ye, H., Li, G. Y., & Juang, B.-H. F. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2021.3061464 · Cited by 129
Zhang, Y., Moorthy, A. K., Chandler, D. M., & Bovik, A. C. (2014) · Signal Processing: Image Communication · doi:10.1016/j.image.2014.05.004 · Cited by 102
“For the cross-validation test on LIVE, we included results of three popular FR IQA algorithms (PSNR [91], SSIM [27], and MS-SSIM [28]) whose results are accessible in the corresponding papers.”
MethodSelf-citationStreaming Video QoE Modeling and Prediction: A Long Short-Term Memory Approach
Eswara, N., Ashique, S., Panchbhai, A., Chakraborty, S., Sethuram, H. P., Kuchi, K., Kumar, A., & Channappayya, S. S. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2895223 · Cited by 124
“We investigate three VQA metrics for STSQ: 1) STRRED [21], a reduced-reference metric, 2) MS-SSIM [28], a full-reference metric, and 3) NIQE [49], a no-reference metric.”
BackgroundMethodInfluentialStructural Test Coverage Criteria for Deep Neural Networks
Sun, Y., Huang, X., Kroening, D., Sharp, J. J., Hill, M., & Ashmore, R. (2019) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3358233 · Cited by 363
“For the general case, the function for a real number d and a distance measure L p , or structural similarity distances such as SSIM [38].”
BackgroundNIQSV+: A No-Reference Synthesized View Quality Assessment Metric
Tian, S., Zhang, L., Morin, L., & Déforges, O. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2781420 · Cited by 101
“in [46].”
Betzel, F., Khatamifard, K., Suresh, H., Lilja, D. J., Sartori, J., & Karpuzcu, U. (2019) · ACM Computing Surveys · doi:10.1145/3145812 · Cited by 102
“Different metrics exist for evaluating the output quality depending on the application: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Multiscale SSIM (MS-SSIM) and so on [86, 167].”
BackgroundNTIRE 2019 Challenge on Real Image Denoising: Methods and Results
Abdelhamed, A., Timofte, R., Brown, M. S., Yu, S., Park, B., Jeong, J., Jung, S., Kim, D.-W., Chung, J. R., Liu, J., et al. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00273 · Cited by 121
“structural similarity (MS-SSIM) [34].”
Suh, S., Lee, H., Lukowicz, P., & Lee, Y. O. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.10.004 · Cited by 106
Leveraging Deep Learning Techniques to Obtain Efficacious Segmentation Results
Purohit, J., & Dave, R. (2023) · Archives of Advanced Engineering Science · doi:10.47852/bonviewaaes32021220 · Cited by 128
A survey on training challenges in generative adversarial networks for biomedical image analysis
Saad, M. M., O’Reilly, R., & Rehmani, M. H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10624-y · Cited by 170
“The occurrence of mode collapse and diversity of synthetic images is quantified by MS-SSIM [80] [72] using image similarity features while IS [81], MMD [71], and FID [74] using distance measures as discussed in Section IV.”
MethodSubjective and objective quality assessment for images with contrast change
Gu, K., Zhai, G.-T., Yang, X.-K., Zhang, W.-J., & Liu, M. (2013) · 2013 IEEE International Conference on Image Processing · doi:10.1109/icip.2013.6738079 · Cited by 101
“Mappings of the scores of the six image quality metrics, including PSNR, SSIM [10], MS-SSIM [11], IW-SSIM [12], MAD [13] and the proposed RIQMC, to subjective scores are achieved through using nonlinear regression with a fourparameter logistic function as suggested by VQEG [16]:”
MethodInfluentialScene background initialization: A taxonomy
Bouwmans, T., Maddalena, L., & Petrosino, A. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2016.12.024 · Cited by 108
Audio-Visual Multimedia Quality Assessment: A Comprehensive Survey
Akhtar, Z., & Falk, T. H. (2017) · IEEE Access · doi:10.1109/access.2017.2750918 · Cited by 113
“In [102] and [101], multiple packet loss and H.264 considering the frames in which packet loss occurs, the magnitude and angle of the motion were studied, while in [103] and [100], the visibility of packet loss via SSIM, and Patient Rule Induction Method (PRIM) and Group-of-Picture (GoP) are adopted for packet loss cl…”
MethodInfluentialProspects of Structural Similarity Index for Medical Image Analysis
Mudeng, V., Kim, M., & Choe, S.-W. (2022) · Applied Sciences · doi:10.3390/app12083754 · Cited by 101
“MS-SSIM procedures to quantify the image quality (adapted from [22]).”
MethodInfluentialFourier Transform-Based Scalable Image Quality Measure
Narwaria, M., Lin, W., McLoughlin, I. V., Emmanuel, S., & Chia, L.-T. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2197010 · Cited by 102
Thermal infrared colorization via conditional generative adversarial network
Kuang, X., Zhu, J., Sui, X., Liu, Y., Liu, C., Chen, Q., & Gu, G. (2020) · Infrared Physics & Technology · doi:10.1016/j.infrared.2020.103338 · Cited by 120
METRICS PERFORMANCE COMPARISON FOR COLOR IMAGE DATABASE
Ponomarenko, N. N., Battisti, F., Egiazarian, K., Astola, J. T., & Лукин, В. В. (2008) · Iris (Roma Tre University) · Cited by 104
Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
Fedus, W., Rosca, M., Lakshminarayanan, B., Dai, A. M., Mohamed, S., & Goodfellow, I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.08446 · Cited by 96
“We use the Inception Score [22] to measure how visually appealing CIFAR-10 samples are, the MS-SSIM metric [18, 24] to check sample diversity, and an Improved Wasserstein independent critic to assess overfitting, as well as sample quality [4].”
MethodDetecting visually similar Web pages: Application to phishing detection
Chen, T.-C., Dick, S., & Miller, J. T. (2010) · ACM Transactions on Internet Technology · doi:10.1145/1754393.1754394 · Cited by 122
“However, SSIM is known to be effective at the detection of global luminance shifts and contrast stretching [Wang et al. 2003]; in addition, a chromatic-variant of SSIM [Toet and Lucassen 2003] has been shown to be effective at detecting many chromatic, nonstructural, distortions.”
BackgroundExemplar Guided Face Image Super-Resolution Without Facial Landmarks
Dogan, B., Gu, S., & Timofte, R. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00232 · Cited by 126
“On top of that, the PSNR measure is unable to capture perceptually important differences between two images as it relies on the differences between pixel-level values at the same position [41, 42, 15].”
BackgroundZhou, W., Xu, J., Jiang, Q., & Chen, Z. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3081182 · Cited by 117
“The FR models include conventional FR IQA as well as OIQA approaches (i.e. PSNR, SSIM [29], MS-SSIM [30], FSIM [31], S-PSNR [40], WS-PSNR [41] and CPP-PSNR [42]) and a deep learning-based FR IQA method (i.e. DeepQA [36]).”
Zhang, P., Xu, W., Liu, Y., Qin, X., Niu, K., Cui, S. R., Shi, G., Qin, Z., Xu, X., Wang, F., et al. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3443193 · Cited by 155
Design of Linear Equalizers Optimized for the Structural Similarity Index
Channappayya, S. S., Bovik, A. C., Caramanis, C., & Heath, R. W. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.921328 · Cited by 98
Self-citation
Metadata
- Authors
- Wang, Z., Simoncelli, E. P., & Bovik, A. C.
- Affiliations
- Courant Institute of Mathematical Sciences (US)
- The University of Texas at Austin (US)
- Venue
- The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003
- Publisher
- IEEE
- Published
- 2004-07-08
- Pages
- 1398-1402
- ISSN
- 1058-6393 (ISSN-L)
- OpenAlex
- W1580389772
- Cited by
- 6,253
- References
- 15
- Citations by year
- 2026: 344 · 2025: 675 · 2024: 729 · 2023: 668 · 2022: 662 · 2021: 641 · 2020: 563 · 2019: 438 · 2018: 305 · 2017: 267 · 2016: 206 · 2015: 173 · 2014: 166 · 2013: 112 · 2012: 89
- FWCI
- 3.98
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Asilomar Conference on Signals, Systems and Computers (ACSSC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- structural similarity, image quality assessment, multiscale analysis, human visual system, perceptual image quality, image generation, parameter calibration, viewing conditions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.