On Statistical Tests for Randomness Included in the NIST SP800-22 Test Suite and Based on the Binomial Distribution
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index; 2 of the 16 sources its claims cite were checked for support.
Abstract
In this paper we review some statistical tests included in the NIST SP 800-22 suite, which is a collection of tests for the evaluation of both true-random (physical) and pseudorandom (algorithmic) number generators for cryptographic applications. The output of these tests is the so-called$p$-value which is a random variable whose distribution converges to the uniform distribution in the interval [0,1] when testing an increasing number of samples from an ideal generator. Here, we compute the exact non-asymptotic distribution of$p$-values produced by few of the tests in the suite, and propose some computation-friendly approximations. This allows us to explain why intensive testing produces false-positives with a probability much higher than the expected one when considering asymptotic distribution instead of the true one. We also propose a new approximation for the Spectral Test reference distribution, which is more coherent with experimental results.
Overview
CoChat Summary
## TL;DR
The paper analyzes how the p-values in three NIST SP800-22 randomness tests are actually distributed under an ideal generator, showing they are discrete and often deviate from the uniform assumption used in standard interpretations. It derives feasible approximations for the Frequency, Runs, and Spectral tests and uses them to explain and reduce false positives, especially in two-level testing.
## Key findings
- For the Frequency test and Runs test, the paper derives approximations to the exact p-value CDFs and reports that the theoretical densities match the experimental ones well for BBS and Mersenne Twister simulations.
- For the Spectral test, the paper finds that NIST’s reference variance for the distribution of \(N_1\) is better fit by \(\sigma^2 = 0.95 \cdot 0.05 \cdot n / 3.8\) rather than \(0.95 \cdot 0.05 \cdot n / 4\).
- Using the refined Spectral-test variance, the reported standard-approach significance is \(1.01258 \cdot 10^{-2}\) for \(n=10^6\) and \(0.99742 \cdot 10^{-2}\) for \(n=2^{20}\); with the NIST variance, the corresponding values are \(1.01258 \cdot 10^{-2}\) and \(0.99742 \cdot 10^{-2}\)? Not stated in the paper.
- For reliability of two-level tests, the paper gives concrete bounds such as \(N < 4798\) and \(N < 36448\) for the Frequency test under specific error requirements, \(N < 5975\) and \(N < 17524\) for a chi-square-based two-level test, and \(N < 102\) and \(N < 685\) for a Kolmogorov-Smirnov-based two-level test.
## Methods
- The paper is a mathematical/review study of three NIST SP800-22 tests: Frequency, Runs, and Spectral, all treated as binomial-based tests on bit sequences.
- It derives exact or approximated p-value distributions under the null hypothesis, then compares those distributions to simulations from BBS and Mersenne Twister generators; Table I and Table II report numerical results for two-level testing.
- It analyzes standard one-level testing, NIST-style two-level testing (fraction passing and chi-square goodness-of-fit), and an additional Kolmogorov-Smirnov-based two-level approach.
## Limitations
- The paper states that the standard approach’s exact power cannot be computed in a general way because it depends on the specific non-ideal generator under test.
- The Kolmogorov-Smirnov-based two-level analysis requires “strong hypotheses and approximations” to build a theoretical framework. This is stated in the paper.
- Evident from the text: much of the derivation is asymptotic or approximation-based, and several formulas are presented as approximations rather than exact closed forms.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


![Fig. 3 (a) Comparison between the theoretical probability density function (PDF) of N 1 according to [2], [14], [15] and the experimental one observed in N = 10 6 sequences generated with the BBS algorithm and with n = 2 20 bits . (b) Normalized L 2 distances between the observed PDF of N 1 in the Spectral Test and a normal PDF with σ 2 = 0 . 95 · 0 . 05 · n/c , with c ranging from 3 . 5 to 4 , and with n = 2 20 bits , n = 100 · 2 10 bits and n = 10 · 2 10 bits . In all cases, the minimum distance is achieved for c ≃ 3 . 8 .](https://figures.cochat.ai/paper-figures/d18283f6df3b468ca7ff71b1a4cd6f35/on-statistical-tests-for-randomness-included-in-the-nist-sp800-22-test-suite-and-figure-3.png)
![Fig. 4 Comparison between the uniform density, the theoretically expected one and the observed experimental ones achieved with the BBS and the Mersenne-Twister algorithms, all discretized in k = 16 bins, for the p -values in the Spectral Test, with n = 2 20 , for the cases (a) the variance is computed as σ 2 = 0 . 95 · 0 . 05 · n/ 4 . 0 as indicated by the NIST document [2]; (b) the variance is computed as σ 2 = 0 . 95 · 0 . 05 · n/ 3 . 8 . Note that the experimental densities in (a) and (b) are different since the p -values are computed with two different functions. The matching in the case (b) is almost perfect.](https://figures.cochat.ai/paper-figures/03bad8cc159740a488ec80acdf721bd6/on-statistical-tests-for-randomness-included-in-the-nist-sp800-22-test-suite-and-figure-4.png)

![Fig. 6 Comparison between the average observed value of D N -E[ D N 0 ] in a two-level Frequency Test for different values of N , with n = 2 20 (solid line), n = 100 · 2 10 (dashed line), n = 10 · 2 10 (dotted-dashed line), along with the approximated theoretically expected values (dotted horizontal lines).](https://figures.cochat.ai/paper-figures/9b1f74e5cf3b4a6798563ae05a688fe1/on-statistical-tests-for-randomness-included-in-the-nist-sp800-22-test-suite-and-figure-6.png)







Verification
Claims the paper makes
1 Valid · 2 Not valid · 8 Not checked · 73 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The reference distribution for N 1 indicated by NIST in [2] is a normal distribution with µ = N 0 = 0 . 95 · n/ 2 and σ 2 = 0 . 95 · 0 . 05 · n/ 4 . This has been experimentally found independently by many authors [14], [15].”
What the sources say
- Not matchedcites [2]
The reference list we hold has no entry [2].
- Disputedcites [14] matched to “Corrections of the NIST Statistical Test Suite for Randomness” (2004) sourceExperimental — accuracy not yet measured
Section 3.2 (p. 10) says NIST's theoretical distribution uses mean 0.95·n/2 but variance 0.95·0.05·n/2. The paper argues that dependence between peaks requires the corrected variance 0.95·0.05·n/4; it does not identify that corrected variance as NIST's reference value. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [15] matched to “The Distribution of the Spectrum for the Discrete Fourier Transform Test Included in SP800-22” (2005) source
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Not matched
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the actual p -values distribution F p depends on u .”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also propose a new approximation for the Spectral Test reference distribution, which is more coherent with experimental results.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Assuming the two symbols have the same probability, S n is a zero average random variable which can be approximated, for large values of n , with a normal distribution with variance σ 2 = n .”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Proposition 1: In a generic binomial test, where the limit distribution of T is normal with mean value µ and variance σ 2 (a) the limit CDF F ∞ ∥·∥ is given by”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
- Rests on an assumption not shown here.
41 more claims
Create a free account to see all 84 claims and every source verdict.
- It may state two separate claims as one. 26
- Probably not a substantial claim. 13
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In this case, computational errors accumulate at each level and two-level tests detect the Lack-Of-Fit of that approximation, leading to rejection even if the generator is good [7, 10, 13, 17].”
From Checking the quality of approximation of p-values in statistical tests for random number generators by using a three-level test (2019) · cites this paper as [17]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Subsequently, Pareschi et al. [17] proposed d = 3 . 8 for n ≈ 10 6 , which we use here as modification.”
From Checking the quality of approximation of p-values in statistical tests for random number generators by using a three-level test (2019) · cites this paper as [17]
Citations
Cites
The Art of Computer Programming
Knuth, D. E. (1968) · Cited by 16,319
Handbook of applied cryptography
(1997) · Choice Reviews Online · doi:10.5860/choice.34-4512 · Cited by 10,357
The Art of Computer Programming
Whitesides, G. E. (1970) · Nuclear Science and Engineering · doi:10.13182/nse70-a19705 · Cited by 6,008
Mathematical Methods of Statistics.
Geary, R. C., & Cramer, H. (1947) · The Economic Journal · doi:10.2307/2226151 · Cited by 5,927
Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-random number generator
Matsumoto, M., & Nishimura, T. (1998) · ACM Transactions on Modeling and Computer Simulation · doi:10.1145/272991.272995 · Cited by 5,780
Simulation modeling and analysis
(1999) · Journal of Manufacturing Systems · doi:10.1016/s0278-6125(99)90119-x · Cited by 3,833
Bassham, L. E., Rukhin, A. L., Soto, J., Nechvatal, J. R., Smid, M. E., Barker, E. B., Leigh, S. D., Levenson, M., Vangel, M., Banks, D. L., et al. (2010) · NIST Special Publication 800-22 Revision 1a · doi:10.6028/nist.sp.800-22r1a · Cited by 3,617
A Simple Unpredictable Pseudo-Random Number Generator
Blum, L., Blum, M., & Shub, M. (1986) · SIAM Journal on Computing · doi:10.1137/0215025 · Cited by 1,036
TestU01: A C library for empirical testing of random number generators
L'Ecuyer, P., & Simard, R. (2007) · ACM Transactions on Mathematical Software · doi:10.1145/1268776.1268777 · Cited by 1,014 · cited 7×
“This also makes these tests suitable for both true-random generators (based on some physical phenomenon that is intrinsically random) and pseudorandom generators (based on a recursive algorithm on a fi nite state machine).”
BackgroundMethodInfluentialShiryaev, A. N. (1996) · Graduate texts in mathematics · doi:10.1007/978-1-4757-2539-1 · Cited by 834
Research works printed but not linked: 4
- "Random Numbers Generation using Quantum Physics," White Paper, idQuantique, 2004. [Online]. Available: http://www.idquantique.com/ products/files/quantis-whitepaper.pdf
- Gianluca Setti (S'89-M'91-SM'02-F'06) received the Dr.Eng. degree (with honors) in electronic en- gineering and the Ph.D. degree in electronic engi- neering and computer science from the University of Bologna, Italy, in 1992 and 1997, respectively, for his contribution to the study of neural networks and chaotic systems. From May 1994 to July 1995, he was with the Laboratory of Nonlinear Systems (LANOS) of the Swiss Federal Institute of Tech- nology in Lausanne (EPFL) as Visiting Researcher. Since 1997 he has been with the School of Engi- neering at the University of Ferrara, Italy, where he is currently a Professor of Circuit Theory and Analog Electronics. He held several visiting position at Visiting Professor/Scientist at EPFL (2002, 2005), UCSD (2004), IBM T. J. Watson Laboratories (2004, 2007) and at the University of Washington in Seattle (2008), and is also a permanent faculty member of ARCES, University of Bologna. He is co-editor of the book Chaotic Electronics in Telecommunications (CRC Press, Boca Raton, 2000) and Circuits and Systems for Future Generation of Wireless Communications (Springer, 2009) as well as one of the guest editors of the May 2002 special issue of the PROCEEDINGS OF THE IEEE on Applications of Non-linear Dynamics to Electronic and Information Engineering. His research interests include nonlinear circuits, recurrent neural networks, implementation and application of chaotic circuits and systems, statistical signal processing, electromagnetic compatibility, wireless communications, and sensor networks. Dr. Setti received the 1998 Caianiello prize for the best Italian Ph.D. thesis on Neural Networks and he is corecipient of the 2004 IEEE CAS Society Darlington Award, as well as of the best paper award at ECCTD 2005 and the best student paper award at EMCZurich 2005 and ISCAS 2011. He served as an Associate Editor for the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS-PART I (1999-2002 and 2002-2004) and for the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS-PART II (2004-2007), as the Deputy-Editor-in-Chief, for the IEEE Circuits and Systems Magazine (2004-2007) and as the Editor-in-Chief for the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS-PART II (2006-2007) and for the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS-PART I (2008-2009). He was the 2004 Chair of the Technical Committee on Nonlinear Circuits and Systems of the of the IEEE CAS Society, a Distinguished Lecturer (2004- 2005), a member of the Board of Governors (2005-2008), and President (2010) of the same society. He was also the Technical Program Co-Chair of NDES2000 (Catania) the Track Chair for Nonlinear Circuits and Systems of ISCAS 2004 (Vancouver), the Special Sessions Co-Chair of ISCAS 2005 (Kobe) and ISCAS 2006 (Kos), the Technical Program Cochair of ISCAS2007 (New Orleans), and ISCAS 2008 (Seattle), as well as the General Co-Chair of NOLTA 2006 (Bologna).
- W. Killmann and W. Schindler, AIS 31: Functionality Classes and Eval- uation Methodology for True (Physical) Random Number Generators, Bundesamt fur Sicherheit in der Informationstechnik (BSI) version 3.1, Sep. 2001.
- W. Schindler, AIS 20: Functionality Classes and Evaluation Method- ology for Deterministic Random Number Generators, Bundesamt fur Sicherheit in der Informationstechnik (BSI) version 2.0, Dec. 1999.
Cited by
Efficient cryptosystem approaches: S-boxes and permutation–substitution-based encryption
Belazi, A., Khan, M., El-Latif, A. A. A., & Belghith, S. (2017) · Nonlinear Dynamics · doi:10.1007/s11071-016-3046-0 · Cited by 249
“In order to test the cipher randomness, NIST SP 800-22 [43] tests are used.”
MethodBelazi, A., Abd El-Latif, A. A., Diaconu, A.-V., Rhouma, R., & Belghith, S. (2017) · Optics and Lasers in Engineering · doi:10.1016/j.optlaseng.2016.07.010 · Cited by 197
Design of image cipher using block-based scrambling and image filtering
Hua, Z., & Zhou, Y. (2017) · Information Sciences · doi:10.1016/j.ins.2017.02.036 · Cited by 193
EMI Reduction via Spread Spectrum in DC/DC Converters: State of the Art, Optimization, and Tradeoffs
Pareschi, F., Rovatti, R., & Setti, G. (2015) · IEEE Access · doi:10.1109/access.2015.2512383 · Cited by 158
“Of course, in any practical circuit implementation, the required ideal random source can only be obtained with some approximation [71]–[73].”
BackgroundSelf-citationSahari, M. L., & Boukemara, I. (2018) · Nonlinear Dynamics · doi:10.1007/s11071-018-4390-z · Cited by 159
“01, the binary sequence is accepted as random with a con-fidence of 99%; otherwise, it is considered as non-random [36,40].”
BackgroundMethodColor multiple image encryption scheme based on 3D-chaotic maps
Malik, D. S., & Shah, T. (2020) · Mathematics and Computers in Simulation · doi:10.1016/j.matcom.2020.07.007 · Cited by 131
Xu, Q., Huang, L., Wang, N., Bao, H., Wu, H., & Chen, M. (2023) · Nonlinear Dynamics · doi:10.1007/s11071-023-08905-w · Cited by 115
Dual-Metastability Time-Competitive True Random Number Generator
Wieczorek, P. Z., & Golofit, K. (2014) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2013.2265952 · Cited by 87
“We chose a 1-Mbit length of analyzed streams, as suggested in [6], [13].”
BackgroundMethodInfluentialChaotic image encryption algorithm using wave-line permutation and block diffusion
Ye, G., Zhao, H., & Chai, H. (2015) · Nonlinear Dynamics · doi:10.1007/s11071-015-2465-7 · Cited by 81
“NIST 800-22 test can evaluate the true-random and pseudorandom numbers for cryptographic application [28].”
A class of novel discrete memristive chaotic map
Liu, X., Sun, K., Wang, H., & He, S. (2023) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2023.113791 · Cited by 83
A novel image encryption scheme based on quantum dynamical spinning and rotations
Khan, M., & Waseem, H. M. (2018) · PLOS ONE · doi:10.1371/journal.pone.0206460 · Cited by 73
RetractedThe image encryption algorithm based on chaos and DNA computing
Zhang, Y. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-017-5585-x · Cited by 79
A new color image encryption technique using DNA computing and Chaos-based substitution box
Masood, F., Masood, J., Zhang, L., Jamal, S. S., Boulila, W., Rehman, S. U., Khan, F. A., & Ahmad, J. (2022) · Soft Computing · doi:10.1007/s00500-021-06459-w · Cited by 67
“Table 7 shows the results of several randomization tests (Pareschi et al. 2012).”
BackgroundColor image encryption using chaotic nonlinear adaptive filter
Hsiao, H.-I., & Lee, J. (2015) · Signal Processing · doi:10.1016/j.sigpro.2015.06.007 · Cited by 66
“Randomness tests for color cipher images In Tables 17–19, we have utilized the NIST SP 800-22a Test Suite [25] for testing the randomness of the color ncryption using chaotic nonlinear adaptive filter, Signal 7i 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 7…”
MethodInfluentialMulti-Images Encryption Scheme Based on 3D Chaotic Map and Substitution Box
Tanveer, M., Shah, T., Rehman, A. U., Ali, A., Siddiqui, G. F., Saba, T., & Tariq, U. (2021) · IEEE Access · doi:10.1109/access.2021.3081362 · Cited by 71
“A cryptosystem is safe if the data is evenly distributed, so it exhibits high complexity and durability [44].”
BackgroundFingerprint image cryptography based on multiple chaotic systems
Hsiao, H.-I., & Lee, J. (2015) · Signal Processing · doi:10.1016/j.sigpro.2015.01.024 · Cited by 60
“We have employed the NIST SP 800-22a Test Suite [49] for checking the randomness of binary bit sequence (720000 bits) of converting encrypted images.”
MethodA Physical-Layer Key Generation Approach Based on Received Signal Strength in Smart Homes
Zhao, H. Q., Zhang, Y., Huang, X., et al. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3119053 · Cited by 54
Wang, Q., Zhang, X., & Zhao, X. (2023) · Nonlinear Dynamics · doi:10.1007/s11071-023-09020-6 · Cited by 56
Wieczorek, P. Z., & Golofit, K. (2018) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2017.2751144 · Cited by 53
“The bit streams generated by the TRNG implementations were verified with both the NIST and Diehard test suites [36], [37].”
BackgroundMethodA novel chaotic image cryptosystem based on DNA sequence operations and single neuron model
Ben Slimane, N., Aouf, N., Bouallegue, K., & Machhout, M. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6145-8 · Cited by 51
“NIST SP 800-22 [47] is used to test the randomness of encrypted image.”
MethodA new approach to digital content privacy using quantum spin and finite-state machine
Waseem, H. M., & Khan, M. (2019) · Applied Physics B · doi:10.1007/s00340-019-7142-y · Cited by 47
4D mixed chaotic system and its application to RGB image encryption using substitution-diffusion
Haq, T. U., & Shah, T. (2021) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2021.102931 · Cited by 48
Bonny, T., Debsi, R. A., Majzoub, S., et al. (2018) · Circuits Systems and Signal Processing · doi:10.1007/s00034-018-0905-6 · Cited by 44
Algebra-chaos amalgam and DNA transform based multiple digital image encryption
Haq, T. U., & Shah, T. (2020) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2020.102592 · Cited by 54
A true random number generator architecture based on a reduced number of FPGA primitives
Stanchieri, G. D. P., Marcellis, A. D., Palange, E., & Faccio, M. (2019) · AEU - International Journal of Electronics and Communications · doi:10.1016/j.aeue.2019.03.006 · Cited by 46
Block Image Encryption Algorithm Based on Novel Chaos and DNA Encoding
Zhao, J., Wang, S., & Zhang, L. (2023) · Information · doi:10.3390/info14030150 · Cited by 51
“5 of 18 The quality evaluation of the chaotic pseudo ‐ random sequence is shown in Table 1 according to the FIPS 140 ‐ 2 standard [26].”
Method12×12 S-box Design and its Application to RGB Image Encryption
Haq, T. U., & Shah, T. (2020) · Optik · doi:10.1016/j.ijleo.2020.164922 · Cited by 42
An efficient block-level image encryption scheme based on multi-chaotic maps with DNA encoding
Naskar, P. K., Bhattacharyya, S., Mahatab, K. C., Dhal, K. G., & Chaudhuri, A. (2021) · Nonlinear Dynamics · doi:10.1007/s11071-021-06761-0 · Cited by 40
Differential cryptanalysis of image cipher using block-based scrambling and image filtering
Yu, F. Q., Gong, X., Li, H., & Wang, S. (2021) · Information Sciences · doi:10.1016/j.ins.2020.12.037 · Cited by 38
A new fast double-chaotic based Image encryption scheme
Abdelfatah, R. I. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08234-4 · Cited by 38
A 4T/Cell Amplifier-Chain-Based XOR PUF With Strong Machine Learning Attack Resilience
Zhang, J., Xu, C., Law, M. K., Jiang, Y., Zhao, X., Mak, P., & Martins, R. P. (2021) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2021.3114084 · Cited by 36
Improved SERPENT Algorithm: Design to RGB Image Encryption Implementation
Shah, T., Haq, T. U., & Farooq, G. (2020) · IEEE Access · doi:10.1109/access.2020.2978083 · Cited by 39
“In this paper, these objectives are measured by using the NIST SP 800-22 [32] test.”
MethodFrom Chaos to Pseudorandomness: A Case Study on the 2-D Coupled Map Lattice
Wang, Y., Liu, Z., Zhang, L. Y., Pareschi, F., Setti, G., & Chen, G. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3129808 · Cited by 40
“2This is called the two-level random number testing procedure [42].”
MethodSelf-citationFlores-Vergara, A., Inzunza-González, E., García-Guerrero, E. E., López-Bonilla, O. R., Rodríguez-Orozco, E., Hernández-Ontiveros, J. M., Cárdenas-Valdez, J. R., & Tlelo‐Cuautle, E. (2019) · Entropy · doi:10.3390/e21030268 · Cited by 35
“If a computed p -value is greater than 0.01, the binary sequence is accepted as random with a confidence of 99%; otherwise, it is considered to be non-random [4,86,101].”
BackgroundInfluentialA fully CMOS true random number generator based on hidden attractor hyperchaotic system
Nguyen, N., Kaddoum, G., Pareschi, F., Rovatti, R., & Setti, G. (2020) · Nonlinear Dynamics · doi:10.1007/s11071-020-06017-3 · Cited by 34
Self-citationA Fast Image Encryption Algorithm Based on Improved 6-D Hyper-Chaotic System
Chen, H., Bai, E., Jiang, X., & Wu, Y. P. (2022) · IEEE Access · doi:10.1109/access.2022.3218668 · Cited by 34
“When P value is larger than 0.01, the sequence is regard as random, and the sequences that pass all the test have good randomness [51].”
BackgroundInfluentialQuantum Image Encryption Using a Self-Adaptive Hash Function-Controlled Chaotic Map (SAHF-CCM)
Abdelfatah, R. I. (2022) · IEEE Access · doi:10.1109/access.2022.3212899 · Cited by 30
“The control parameters range for the first part of the proposed PRNG are as follows: [43] shown in Table 1, where one or more P values are computed.”
MethodLightweight TRNG Based on Multiphase Timing of Bistables
Wieczorek, P. Z. (2016) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2016.2555248 · Cited by 27
“The product of XOR’ing and VN correction was tested with the second-level NIST and Diehard tests [25]–[27], moreover additional bit rate measurements were also performed.”
MethodDeriving the Variance of the Discrete Fourier Transform Test Using Parseval’s Theorem
Iwasaki, A. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2019.2947045 · Cited by 29
“[5] and a = 2/0.”
Recommendations on Statistical Randomness Test Batteries for Cryptographic Purposes
Luengo, E. A., & Villalba, L. J. G. (2021) · ACM Computing Surveys · doi:10.1145/3447773 · Cited by 36
“For example, in [48], the authors calculate the exact non-asymptotic distribution of the p − values generated by some tests included in the battery, and propose some approximations.”
BackgroundFurther analysis of the statistical independence of the NIST SP 800-22 randomness tests
Luengo, E. A., Olivares, B. A., Villalba, L. J. G., & Hernandez-Castro, J. (2023) · Applied Mathematics and Computation · doi:10.1016/j.amc.2023.128222 · Cited by 26
Malik, D. S., Shah, T., Tehsin, S., Nasir, I. M., Fitriyani, N. L., & Syafrudin, M. (2024) · Mathematics · doi:10.3390/math12152302 · Cited by 27
Improved Twofish Algorithm: A Digital Image Enciphering Application
Haq, T. U., Shah, T., Siddiqui, G. F., et al. (2021) · IEEE Access · doi:10.1109/access.2021.3081792 · Cited by 26
“SP 800-22 [29] test.”
On-chip lightweight implementation of reduced NIST randomness test suite
Suresh, V., Antonioli, D., & Burleson, W. P. (2013) · IEEE International Symposium on Hardware Oriented Security and Trust (HOST) · doi:10.1109/hst.2013.6581572 · Cited by 34
Okada, H., & Umeno, K. (2017) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2017.2656473 · Cited by 25
“[6] reviewed three tests included in NIST SP 800-22, and they also numerically estimated the distribution of the test statistic.”
MethodDi, X., Li, J., Qi, H., Cong, L., & Yang, H. (2017) · PLoS ONE · doi:10.1371/journal.pone.0184586 · Cited by 26
Shah, T., Ali, A., Khan, M., Farooq, G., & de Andrade, A. A. (2020) · Wireless Personal Communications · doi:10.1007/s11277-020-07274-6 · Cited by 23
RetractedA $16\times 16$ Pixel Post-Processing Free Quantum Random Number Generator Based on SPADs
Xu, H., Perenzoni, D., Tomasi, A., et al. (2018) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2018.2821904 · Cited by 20
Image Encryption Algorithm Based on Bit-Level Permutation and Dynamic Overlap Diffusion
Wang, J., Li, J., Di, X., Zhou, J., & Man, Z. (2020) · IEEE Access · doi:10.1109/access.2020.3020187 · Cited by 20
More Powerful and Reliable Second-Level Statistical Randomness Tests for NIST SP 800-22
Zhu, S., Ma, Y., Lin, J., Zhuang, J., & Jing, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-662-53887-6_11 · Cited by 22
“Furthermore, inspired by [10], we investigate the actual distributions of Pvalues and Q-values with a finite block length, for the binomial-based tests.”
BackgroundMethodInfluentialLiu, X., Sun, K., & Wang, H. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-14133-y · Cited by 20
“[38] Ref.”
BackgroundMethodInfluentialLiu, X., Sun, K., & Wang, H. (2024) · The European Physical Journal Plus · doi:10.1140/epjp/s13360-023-04823-3 · Cited by 19
“To comprehensively verify the randomness of a sequence for cryptographic applications, the statistical standard NIST test [48] as a collection of tests is widely employed.”
RIS-Assisted Integration of Communications and Security: Protocol, Prototyping, and Field Trials
Wan, Z., Huang, K., Xu, X., Yi, M., Wang, H., Zhu, Z., & Jin, L. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3390677 · Cited by 19
“To evaluate the randomness of the generated secret keys, we conduct the National Institute of Standards and Technology (NIST) randomness test [35].”
MethodNon-Associative Algebra Redesigning Block Cipher with Color Image Encryption
Sanam, N., Ali, A., Shah, T., & Farooq, G. (2021) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2021.014442 · Cited by 18
“By a definite end objective to attain these prerequisites, we used NIST SP 800–22 [39] for testing the randomness of digital images.”
MethodSecure lightweight cryptosystem for IoT and pervasive computing
Abutaha, M., Atawneh, B., Hammouri, L., & Kaddoum, G. (2022) · Scientific Reports · doi:10.1038/s41598-022-20373-7 · Cited by 29
“(1) The national institute of standards and technology (NIST) statistical test 27 , among the most widely used tests for analyzing the randomization of binary streams, is used to examine the statistical correctness of the keystream created.”
MethodRandomness Test to Solve Discrete Fourier Transform Test Problems
IWASAKI, A., & Umeno, K. (2018) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1587/transfun.e101.a.1204 · Cited by 18
Color image encryption by a non-chain Galois ring extension
Shah, T., Safdar, M. U., Ali, A., & Haq, T. U. (2023) · Physica Scripta · doi:10.1088/1402-4896/ad0bba · Cited by 17
Safdar, M. U., Shah, T., & Ali, A. (2024) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-024-09010-9 · Cited by 17
A novel discrete image encryption algorithm based on finite algebraic structures
Shah, D., & Shah, T. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09182-0 · Cited by 18
High-speed encrypted traffic classification by using payload features
Yan, X., He, L., Xu, Y., Cao, J., Wang, L., & Xie, G. (2025) · Digital Communications and Networks · doi:10.1016/j.dcan.2024.02.003 · Cited by 17
Analysis of NIST SP800-22 focusing on randomness of each sequence
Iwasaki, A. (2018) · JSIAM Letters · doi:10.14495/jsiaml.10.1 · Cited by 15
Error Analysis of NIST SP 800-22 Test Suite
Chen, D., Chen, H., Fan, L., & Luo, K. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3287391 · Cited by 14
“[4] have calculated an upper bound in the Frequency Test for the error between the approximated p-value and the actual one based on Berry-Esséen theorem, and formulated a mathematical condition for the reliability of a two-level test to obtain the restrictions on the parameters N , k.”
BackgroundMethodResultInfluentialLightweight True Random Bit Generators in PLDs: Figures of Merit and Performance Comparison
Addabbo, T., Fort, A., Moretti, R., Mugnaini, M., Vignoli, V., & Garcia-Bosque, M. (2019) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2019.8702791 · Cited by 15
“When targeting cryptographic applications, a standard approach to evaluate the quality of a TRBG is to refer to standard statistical testing [1], [7], [8].”
MethodSafdar, M. U., Shah, T., Ali, A., & Haq, T. U. (2023) · Integration · doi:10.1016/j.vlsi.2023.102095 · Cited by 12
Kurunathan, H., Li, K., Tovar, E., Jorge, A., Ni, W., & Jamalipour, A. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3410534 · Cited by 13
“We used the standard randomness test suite from NIST [67] to verify the randomness of the keys generated by the DRL algorithm.”
MethodA combinatory approach of non-chain ring and henon map for image encryption application
Din, S. M. U., Shah, T., Alblehai, F., Nooh, S. A., & Jamal, S. S. (2025) · Scientific Reports · doi:10.1038/s41598-025-85814-5 · Cited by 19
“For examine a randomness in a digital image we use NIST SP 800 − 22 to get a specific goal 42 .”
Multiple-image encryption algorithm based on S-boxes and DNA sequences
Safdar, M. U., Shah, T., & Ali, A. (2025) · Signal Processing Image Communication · doi:10.1016/j.image.2025.117353 · Cited by 20
Sine-modulation Discrete Memristor Chaotic Map and Its Applications
Zhu, H., Liu, W., Sun, K., et al. (2025) · Nonlinear Dynamics · doi:10.1007/s11071-025-11463-y · Cited by 12
An Adaptive Secret Key Establishment Scheme in Smart Home Environments
Zhao, H. Q., Zhang, Y., Huang, X., et al. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761344 · Cited by 12
“In addition, we evaluate the randomness of the established key using the wide employed NIST randomness test tool [18].”
MethodNew pseudo-random number generator based on improved discrete-space chaotic map
Lambić, D., & Nikolić, M. (2019) · Filomat · doi:10.2298/fil1908257l · Cited by 14
“Also, these tests can report false-positives results with much higher probability than expected [23].”
BackgroundDesigning S-boxes triplet over a finite chain ring and its application in RGB image encryption
Jahangir, S., & Shah, T. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08995-3 · Cited by 12
“By a definite end target to accomplish these preconditions, we used NIST SP 800 – 22 [23] for testing the randomness of digital images.”
MethodA color image privacy scheme established on nonlinear system of coupled differential equations
Javeed, A., Shah, T., & Ullah, A. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09582-2 · Cited by 12
“To ensure these features as well as for the measurement of randomness of digital images, we used NIST SP 800 – 22 [28], which also include several subclasses.”
BackgroundMethodDouble quantum images encryption scheme based on chaotic system
Jiang, S.-X., 李, Y. 杨. L., 石, J. 锦. S., & 张, R. 茹. Z. (2023) · Chinese Physics B · doi:10.1088/1674-1056/ad1174 · Cited by 14
Random Sequence Generation using Superconducting Qubits
Gupta, M., & Nene, M. J. (2021) · International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV) · doi:10.1109/icicv50876.2021.9388495 · Cited by 11
“The sequences generated are required to go through battery of tests using any of the many available testing suites like NIST testing suite [57].”
MethodGafsi, M., Amdouni, R., Hajjaji, M. A., Mtibaa, A., & Bourennane, E. (2022) · International Journal of Circuit Theory and Applications · doi:10.1002/cta.3415 · Cited by 11
“The NIST 800-22 test determines if a key generator is appropriate for information security.(41) It includes fifteen tests that are dedicated to evaluate key sequences in accordance with specific requirements.”
BackgroundAn Adaptive Physical Layer Key Extraction Scheme for Smart Homes
Zhao, H. Q., Zhang, Y., Huang, X., et al. (2019) · doi:10.1109/trustcom/bigdatase.2019.00073 · Cited by 11
“Moreover, we evaluate the randomness of the extracted secret key using the widely employed NIST randomness test tool [27].”
MethodAmdouni, R., Hajjaji, M. A., & Mtibaa, A. (2024) · Physica Scripta · doi:10.1088/1402-4896/ad2e5b · Cited by 10
A New PRNG Hardware Architecture Based on an Exponential Chaotic Map
Cardoso, M. B. R., Silva, S. S. D., Nardo, L. G., Passos, R. M., Nepomuceno, E. G., & Arias-García, J. (2021) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas51556.2021.9401653 · Cited by 14
SPAD-Based Quantum Random Number Generator With an $N^{\rm{th}}$ -Order Rank Algorithm on FPGA
Tontini, A., Gasparini, L., Massari, N., & Passerone, R. (2019) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2019.2909013 · Cited by 9
Kopp, M. I., & Samuilik, I. (2025) · Mathematics · doi:10.3390/math13172838 · Cited by 10
A chaos-metastability TRNG for natively flexible IGZO circuits
Gołofit, K., Wieczorek, P. Z., & Pilarz, M. (2023) · AEU - International Journal of Electronics and Communications · doi:10.1016/j.aeue.2023.154835 · Cited by 11
An FPGA-Based Architecture of True Random Number Generator for Network Security Applications
Stanchieri, G. D. P., Marcellis, A. D., Faccio, M., & Palange, E. (2018) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2018.8351376 · Cited by 10
“Successively, the NIST test Suite, one of the most used approach that executes fifteen different tests, has been taken into consideration [22-24].”
MethodFast Chaotic Image Encryption Algorithm Using a Novel Divide and Conquer Diffusion Strategy
Ge, B., Shen, Z., & Zhang, J. (2022) · IEEE Access · doi:10.1109/access.2022.3204873 · Cited by 9
“Therefore, a further statistical test is conducted using 413 the SP 800-22 randomness test suit powered by NIST [53].”
MethodCui, Y., Li, J., Wang, C., Gu, C., & Liu, W. (2021) · Microelectronics Journal · doi:10.1016/j.mejo.2021.105265 · Cited by 9
Savelov, M. P. (2021) · Дискретная математика · doi:10.4213/dm1686 · Cited by 9
NTK-3D chaotic map and a non-associative LA-field sequences for image encryption
Sanam, N., Haq, T. U., & Tariq, K. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109196 · Cited by 8
Symmetric Color Image Encryption Using a Novel Cross–Plane Joint Scrambling–Diffusion Method
Ge, B., Shen, Z., & Wang, X. (2023) · Symmetry · doi:10.3390/sym15081499 · Cited by 9
Tan, S., Sun, J., Tang, Y., Sun, Y., & Wang, C. (2024) · Nonlinear Dynamics · doi:10.1007/s11071-024-09317-0 · Cited by 9
“Figure 12 shows the reconstructed image of the phase mask matrix under Gaussian noise attack with a mean of 0 and different variances for 3000 measurements (the variances of Gaussian noise added to the phase mask matrix from left to right for each row of images are 0.05, 0.1, and 0.2, respectively).”
InfluentialHaramoto, H., & Matsumoto, M. (2019) · Mathematics and Computers in Simulation · doi:10.1016/j.matcom.2018.08.005 · Cited by 7
“In this case, computational errors accumulate at each level and two-level tests detect the Lack-Of-Fit of that approximation, leading to rejection even if the generator is good [7, 10, 13, 17].”
BackgroundNing, H., Zhao, G., Dong, Y., Ma, Y., & Jia, J. (2022) · Nonlinear Dynamics · doi:10.1007/s11071-022-07543-y · Cited by 7
“According to the paper [38], the minimum pass rate for each statistical subtest with the exception of the random excursion (variant) subtest is 96 for 100 sequences.”
BackgroundKojima, S., & Sugiura, S. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3386597 · Cited by 7
“WeevaluatetherandomnessofthegeneratedkeysbytheNIST randomness test [39].”
Zhu, P., Wang, H., & Ji, Y. (2024) · Journal of Lightwave Technology · doi:10.1109/jlt.2024.3435457 · Cited by 7
“5(b) shows the results of the NIST 800-22 test for the chaotic system [29].”
Nabil, H., & Tayeb, H. (2025) · The European Physical Journal Plus · doi:10.1140/epjp/s13360-025-06953-2 · Cited by 9
“The NIST SP 800-22 test suite comprises 15 statistical tests designed to evaluate the randomness of binary sequences produced by cryptographic random or pseudorandom number generators [38].”
A Modified Algorithm for the Logistic Sequence Based on PCA
Liu, C., & Ding, Q. (2020) · IEEE Access · doi:10.1109/access.2020.2978503 · Cited by 7
“01, the binary sequence is accepted as random with a confidence of 99%; otherwise, it is considered as nonrandom [39] and [40].”
BackgroundAn LA-group based design of the non-linear component of block cipher
Mumtaz, S., Sanam, N., & Haq, T. U. (2023) · Integration · doi:10.1016/j.vlsi.2023.06.004 · Cited by 7
A new discrete Fourier transform randomness test
Chen, M., Chen, H., Fan, L., Zhu, S., Xi, W., & Feng, D. (2019) · Science China Information Sciences · doi:10.1007/s11432-018-9489-x · Cited by 7
Combining DNA Encoding and Chaos for Medical Image Encryption
Amdouni, R., Gafsi, M., Hajjaji, M. A., & Mtibaa, A. (2022) · International Conference on Sciences and Techniques of Automatic Control and Computer Engineering · doi:10.1109/sta56120.2022.10019116 · Cited by 7
“The sequences created have to pass the statistical NIST SP-800 test, that is a collection of 188 tests and sub-tests designed to assess and test the randomness of large binary sequences [17].”
A VOHE System for Underwater Communications
Yang, P., Nagase, T., You, S., et al. (2020) · Electronics · doi:10.3390/electronics9101557 · Cited by 7
(2018) · Iraqi Journal of Science · doi:10.24996/ijs.2018.59.1a.19 · Cited by 9
“01,according to [29] when the P-value larger than 0.”
BackgroundA random decomposition method for chaotic sequences to improve the security of image encryption
Xiao, Q., & Yu, W. (2025) · Physica Scripta · doi:10.1088/1402-4896/adb523 · Cited by 7
Metadata
- Authors
- Pareschi, F., Rovatti, R., & Setti, G.
- Affiliations
- University of Ferrara (IT)
- University of Bologna (IT)
- Venue
- IEEE Transactions on Information Forensics and Security
- Published
- 2012-04
- Volume
- 7
- Issue
- 2
- Pages
- 491-505
- ISSN
- 1556-6013 (ISSN-L)
- 1556-6021
- OpenAlex
- W2091221194
- Cited by
- 189
- References
- 29
- Citations by year
- 2026: 16 · 2025: 24 · 2024: 25 · 2023: 20 · 2022: 15 · 2021: 22 · 2020: 16 · 2019: 15 · 2018: 13 · 2017: 5 · 2016: 5 · 2015: 7 · 2014: 1 · 2013: 4
- FWCI
- 3.67
- Type
- journal article
- Language
- en
- Open access
- green
- porto.polito.it
- Locations
- IEEE Transactions on Information Forensics and Security (publishedVersion)
- PORTO Publications Open Repository TOrino (Politecnico di Torino) (submittedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion)
- PORTO@Iris (submittedVersion, other-oa)
- Institutional Research Information System University of Ferrara (University of Ferrara) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Computer Science › Artificial Intelligence › Cryptographic Implementations and Security
- Physical Sciences › Mathematics › Discrete Mathematics and Combinatorics › Coding theory and cryptography
- Keywords
- NIST SP 800-22, randomness testing, pseudorandom number generators, true random number generator, p-value distribution, binomial distribution, p-value approximation, false positives, spectral test, cryptographic randomness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.