The DOI matches its Crossref record.
Markov Chains: From Theory to Implementation and Experimentation
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In addition, the Markov model [159, 296] (elaborated 50 years earlier) was leveraged to construct a process based on discrete-time state transitions and action rewards, named Markov Decision Process (MDP), which formalizes sequential decision-making problems in a fully observable, controlled environment [46].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [159]
Citations
Cited by
Sorted by citations
Items 301–400 of 413
Unifying evolutionary dynamics: a set theory exploration of symmetry and interactionPreprint
Berkemeier, F., & Page, K. M. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.09.27.559729 · Cited by 1
“A counting metric that counts the number of individuals displaying various phenotypes is used to characterise the population as a whole, evolving according to a Markov process (Gagniuc, 2017).”
MethodGnoyke, P., Schulze, S., & Krüger, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4693779 · Cited by 1
Simulation studies of characteristics of Full Availability Group with generalized traffic classes
Kaliszan, A. (2018) · International Symposium on Communication Systems, Networks and Digital Signal Processing (CSNDSP) · doi:10.1109/csndsp.2018.8471806
“The other reason is connected with the fact that the processes in with the time intervals between events are described using exponential distribution are Markovian processes[3].”
BackgroundAnalysis on Convergence of Stochastic Processes in Cloud Computing Models
Nie, J., Tang, H., & Wei, J. (2018) · International Conference on Computational Intelligence and Security (CIS) · doi:10.1109/cis2018.2018.00024
“Before talking about the convergence of stochastic matrices [10], we introduce some useful definations.”
MethodXia, Y. J., Gong, Y., Zhang, X., et al. (2019) · Advances in Science Technology and Engineering Systems Journal · doi:10.25046/aj040244
Zhang, J., Konduri, K. C., Chen, R., et al. (2019) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/0361198119847476
A Fusion Model of Social Influence and Group Decision
Jin, J. (2019) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc.2019.8833301
“Proposition 1 [19] matrix P is an irreducible Markov matrix.”
BackgroundGreedy Queens on an Infinite Chessboard
Paulsen, W. H. (2019) · College Mathematics Journal · doi:10.1080/07468342.2019.1632671
Designing metrics to evaluate the help center of Baidu cloud
Gao, Z., Gao, Y., & Xu, J. (2019) · ACM International Conference on the Design of Communication (SIGDOC) · doi:10.1145/3328020.3353936
“Markov model, borrowed from probability theory, is a stochastics model used to model randomly changing systems [14].”
MethodLee, J., & Choi, J. (2020) · New & Renewable Energy · doi:10.7849/ksnre.2020.2042
Concurrent Hydroclimatic Hazards from Catchment to Global ScalesPreprint
De-Luca, P. (2020) · doi:10.31237/osf.io/upknr
Hemline Index Theory: empirical analysis with Google data
Škrinjarić, T. (2020) · International Journal of Fashion Design Technology and Education · doi:10.1080/17543266.2020.1799079
Operational Analysis and Basic Queueing Models
Kounev, S., Lange, K.-D., & von Kistowski, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-41705-5_7
Pretschner, A., Pabel, S., Haas, M., Monika Heiner, A., & Marwan, W. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.09.16.299578
“2020; Werthmann and Marwan 2017), models gene expression trajectories as Markov chains (Gagniuc 2017), which assumes that each subsequent state only depends on the current state of a cell and not on its previous states, i.”
BackgroundResultPoéticas composicionais em peças estreadas na 21a temporada do Grupo Prelúdio 21
Schubert, A., Gentil-Nunes, P., Alves, J. O., et al. (2020) · Orfeu · doi:10.5965/2525530405012020437
Generation of search queries with neural networks
Gusenkov, A., & Sittikova, A. R. (2020) · Naučnyj servis v seti internet · doi:10.20948/abrau-2020-52
Probabilistic Analysis on a Political Career
Ren, Y. (2020) · International Conference on Computer Vision, Image and Deep Learning (CVIDL) · doi:10.1109/cvidl51233.2020.00-19
Dynamic Data Mining Based on the Stability of Dynamic Models
Chebi, H. (2020) · Advances in computational intelligence and robotics book series · doi:10.4018/978-1-7998-4703-8.ch005
A Simple Mathematical Model of Politics (II)Preprint
Huang, J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4444179
“The stochastic matrix was first developed by Andrey Markov in 1906 so it’s also called a Markov matrix [2].”
MethodHierarchical Image Object Search Based on Deep Reinforcement Learning
Zhang, W., Yao, H., & Tan, Y. (2020) · International Journal of Advanced Network Monitoring and Controls · doi:10.21307/ijanmc-2020-029
A Semantically Sensitive Privacy Protection Method for Trajectory Publishing
Shao, Z., Feng, B., & Li, X. (2021) · Journal of Computer and Communications · doi:10.4236/jcc.2021.94003
Bio-Electromagnetics without Fields: The Effect of the Vector Potential
Szász, A. (2021) · Open Journal of Biophysics · doi:10.4236/ojbiphy.2021.112007
“It is a Markovian process [46], determines the development in subsequent series of values.”
Influence of Node’s Reliability Indicators on the Wireless Sensor Network Operability
Novikov, K., & Polesskiy, S. (2021) · International Seminar on Electron Devices Design and Production (SED) · doi:10.1109/sed51197.2021.9444492
“Thus, the Markov model of a set of independent object groups will have the following form [4] (Fig.”
BackgroundDoubly stochastic matrices and the quantum channels
Das, H. K., & Ahmed, M. K. (2021) · Journal of Applied Mathematics Statistics and Informatics · doi:10.2478/jamsi-2021-0005
Network manipulation algorithm based on inexact alternating minimization
Müller, D., & Shikhman, V. (2022) · Computational Management Science · doi:10.1007/s10287-022-00429-9
“For a finite state space, a transition matrix describes the structure of changes on the chain’s states, see e.g. Gagniuc (2017).”
BackgroundProduct Category Recommendation System Using Markov Model
Sivakriskul, K., & Phienthrakul, T. (2021) · Lecture notes in networks and systems · doi:10.1007/978-981-16-0882-7_60
Using Transition Learning to Enhance Mobile-Controlled Handover In Decentralized Future Networks
Platt, S., Demirel, B., & Oliver, M. (2021) · IEEE 5G World Forum (5GWF) · doi:10.1109/5gwf52925.2021.00081
“Having all grid positions being equidistant and with an eigenvalue of 1, the sum of probability of the UE transitioning into any given position in the state space converges to 1 after the 2000 walk trial (5) [35].”
BackgroundAn, T. (2021) · IEEE International Conference on Frontiers Technology of Information and Computer (ICFTIC) · doi:10.1109/icftic54370.2021.9647091
Markov Chains: The Machine ( I )
Gagniuc, P. A. (2021) · doi:10.1002/9781119698005.ch13
Self-citationParameshwarappa, P., Chen, Z., & Koru, G. (2021) · IGI Global eBooks · doi:10.4018/978-1-7998-8954-0.ch032
Gagniuc, P. A. (2021) · doi:10.1002/9781119698005.ch4
Self-citationGagniuc, P. A. (2021) · Wiley eBooks · doi:10.1002/9781119698005.ref
Self-citationContext-Aware Graph Convolutional Network for Dynamic Origin-Destination Prediction
Nathaniel, J., & Zheng, B. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671752
Cormack, A. N. (2022) · doi:10.1002/9781118939079.ch1
VIX Implied Volatility as a Time-Invariant, Stationary Assessor of Market Nervousness/Uncertainty
Ronn, E. I. (2022) · Review of Pacific Basin Financial Markets and Policies · doi:10.1142/s0219091522500205
Nowcasting Extreme Weather with Machine Learning Techniques Applied to Different Input DatasetsPreprint
Biondi, R., Chkeir, S., Anesiadou, A., et al. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4144317
A Contextual and Content Features-Based Device Behavioral Fingerprinting Method in Smart Grid
Jin, Z., Li, N., Liu, C., et al. (2021) · doi:10.1109/hpcc-dss-smartcity-dependsys53884.2021.00079
“Several techniques extract device behavioral fingerprints from the network traffic, including statistical-based method [2], [3], [4], and knowledge-based method [5], [6], [7].”
MethodSpectral Non-Stationarity in Road Vehicle Vibrations
Hári, L. R. (2022) · Periodica Polytechnica Transportation Engineering · doi:10.3311/pptr.18318
“A stochastic process is said to be stationary in the strong sense if its unconditional joint probability distribution does not change when shifted in time (Gagniuc, 2017).”
BackgroundJang, J., Song, N. C., Shim, J., Lee, G., Choi, J.-Y., Prasad, R., & Choi, S. (2023) · AIAA SciTech Forum · doi:10.2514/6.2023-2659
Cayetano, C., Sullivan, E., Miller, P. I., et al. (2023) · doi:10.14324/111.444/000157.v2
Accelerating Evolution Through Gene Masking and Distributed Search
Shahrzad, H., & Miikkulainen, R. P. (2023) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3583131.3590508
Analysis of the Attractor-Based Search System
Li, W. (2023) · Synthesis lectures on operations research and applications · doi:10.1007/978-3-031-35719-0_4
Eco-Approach Towards an Actuated Signalized Intersection: Aware of Passing OpportunitiesPreprint
Hu, J. S., Li, S., & Wang, H. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4581026
Identifying Critical Congested Roads Based on Traffic Flow-Aware Road Network Embedding
Zhao, J., Cheng, P., Ge, Q., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46661-8_40
Markov Processes for the Management of a Microgrid
Vergine, S., Álvarez-Arroyo, C., D’Amico, G., et al. (2023) · Contributions to statistics · doi:10.1007/978-3-031-40209-8_13
End-to-End Benchmarking of Chiplet-Based In-Memory Computing
Krishnan, G., V, Mandal, S. K., Goksoy, A. A., Wang, Z., Chakrabarti, C., Seo, J.-S., Ogras, U. Y., & Cao, Y. K. (2023) · Artificial intelligence · doi:10.5772/intechopen.111926
Narratives of Italian Transatlantic (re)migration, 1897–1936
Viola, L. (2023) · Frontiers in Sociology · doi:10.3389/fsoc.2023.1239585
“Ngram language models are probabilistic models –known as Markov chain (Gagniuc, 2017) – for predicting the next item in a sequence of elements in which the probability of each item depends on the proceeding one.”
BackgroundMore Quality Less Trust? On the Detrimental Effect of Quality Via the Loss of TrustPreprint
Dreyfuss, M., Giat, Y., & Manes, E. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4657196
Feasibility Evaluation of Oversize Load Transportation Using Conditional Rewarded Deep Q-Networks
Son, H., Kim, J., Jung, H., et al. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3339143
“The reason for stacking images is related to Markov system modeling [20].”
BackgroundPark, J., Shim, J., Lee, G., & Choi, S. (2024) · AIAA SciTech Forum · doi:10.2514/6.2024-1711
Zhang, L., Wang, J., & Zhu, Y. (2023) · International Conference on New Trends in Computational Intelligence (NTCI) · doi:10.1109/ntci60157.2023.10403681
Optimizing with Attractor: A Tutorial
Li, W. (2024) · ACM Computing Surveys · doi:10.1145/3648354
“The essential idea of Markov-chain Monte Carlo simulation is using randomness to solve problems that might be deterministic in principle [ 41 , 54 , 65 , 68 , 80 ].”
BackgroundKorvin, G. (2024) · Earth and environmental sciences library · doi:10.1007/978-3-031-46700-4_8
Миронов, О. (2024) · Наукові перспективи (Naukovì perspektivi) · doi:10.52058/2708-7530-2024-4(46)-689-700
Моделирование процесса эксплуатации систем противо-пожарной защиты с использованием марковских цепей
Танклевский, Л. Т., Таранцев, А. А., & Balabanov, I. (2024) · Pozharovzryvobezopasnost/Fire and Explosion Safety · doi:10.22227/0869-7493.2024.33.02.59-67
Kramer, M. P., Viana, J., Mueller, R. A. E., et al. (2024) · Springer eBooks · doi:10.1007/978-3-031-45742-5_5
Zhu, Y., Schmidt, S., Zhao, L., Jiang, C., Micco, P. D., Mitsopoulos, C., Futreal, P. A., & Al‐Lazikani, B. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.06.04.597409
Modeling Functional Architectures Using Markov Chains
Domerçant, J. C. (2024) · 2020 IEEE International Systems Conference (SysCon) · doi:10.1109/syscon61195.2024.10553407
Xu, C., Wang, Y., Duan, Y., & Zhang, C. (2024) · Journal of Systems Engineering and Electronics · doi:10.23919/jsee.2024.000066
Ibrahim, P. O., Sternberg, H., & Ojigi, L. M. (2024) · Applied Geomatics · doi:10.1007/s12518-024-00564-9
“The Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event (Gagniuc 2017).”
Rajendiran, S., Galdos, F. X., Lee, C. A., et al. (2024) · Systems and Control Transactions · doi:10.69997/sct.152564
Kotb, M., & Hosny, O. A. (2024) · Lecture notes in civil engineering · doi:10.1007/978-3-031-60419-5_3
Strengthening communication and self-management
Boonstra, M. D. (2024) · University of Groningen research database (University of Groningen / Centre for Information Technology) · doi:10.33612/diss.1073766916
Evaluation of sales areas in supermarket by Markov chain model
Sano, N., Yada, K., Okamoto, S., et al. (2024) · Multidisciplinary International Social Networks Conference (MISNC) · doi:10.1145/3675669.3675676
Improving Power Grid Operational Resilience During A Tornado Disaster
Nazari, M., Rezaei, N., & Bevrani, H. (2024) · doi:10.1109/icee63041.2024.10667739
Jin, L., Wei, L., & Lv, X. (2024) · Springer eBooks · doi:10.1007/978-3-031-68594-1_7
Otunuga, O. M. (2024) · Springer proceedings in mathematics & statistics · doi:10.1007/978-3-031-69710-4_6
Dreyfuss, M., Giat, Y., & Manes, E. (2024) · International Journal of Financial Studies · doi:10.3390/ijfs12040123
Razavi, F., & Ramazani, S. (2025) · Molecular Physics · doi:10.1080/00268976.2025.2486710
“In Markov chain the probability of each event is based on the previous event (In the other words the probability of each event depends only on the state attained in the previous event) [60].”
Optimal Digital Twinning of Random Systems with Twinning Rate Constraints
Tunç, Ç. (2024) · IEEE International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD) · doi:10.1109/camad62243.2024.10942782
“Taking the expected value of both sides of (3) yields: 2024 IEEE 29th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD) P kk ( t ) denotes the probability that the system remains in state k t time units after initialled sampled at time t 0 = 0 .”
Experimental Classification of Elementary Cellular Automata Using Markov Chains
Seck-Tuoh-Mora, J. C., Hernández-Romero, N., Medina-Marín, J., et al. (2025) · Emergence, complexity and computation · doi:10.1007/978-3-031-78757-7_9
A Bayesian and Markov Chain Approach to Short-Term Personal Watercraft Trajectory ForecastingPreprint
Žužić, L., & Lerga, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5156719
Ding, J. M., Choo, W., & Yap, N. K. (2025) · International Journal of ADVANCED AND APPLIED SCIENCES · doi:10.21833/ijaas.2025.03.015
A Versatile Hardware Solution for Generating Power Law Noise With Tunable Long-Term Dependency
Yuan, X., Luo, Y., Chai, Z., et al. (2025) · IEEE Electron Device Letters · doi:10.1109/led.2025.3577161
“On the other hand, it is important to note that a two-level RTN can be represented by a two-state Markov chain (MC) [16], [17].”
Call Detail Record Generator for Modeling Real-World Telecommunication Scenarios
Rafało, M. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-94193-1_22
Balint, I. B., & ERDODI, B. T. (2025) · Minerva Endocrinology · doi:10.23736/s2724-6507.25.04133-8
Bayesian Regression-Based Power Loss Probabilistic Estimation of Dust-Accumulated PV Systems
Li, C., Chen, X., Shen, Y. M., Zhang, K., Fang, S., & Wei, H. (2025) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc65474.2025.11090541
Carnes, N. C., Biggs, A. T., Hamilton, J. A., et al. (2025) · The Journal of Defense Modeling and Simulation Applications Methodology Technology · doi:10.1177/15485129251358280
Polychroniou, P. V., Tsagkanou, M., Vasilagkos, T., & Manousakis, E. (2025) · Journal of Hospitality and Tourism Horizons · doi:10.1108/jhth-02-2025-0021
RefineSeg: Dual Coarse-to-Fine Learning for Medical Image Segmentation
Du, A., Aung, N., Arvanitis, T. N., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05325-1_44
Grote, A., Hariharan, A., & Weinhardt, C. (2025) · Frontiers in Big Data · doi:10.3389/fdata.2025.1604887
“Similarly, Markov Chains create a memoryless transition matrix, abstracting away specific sequence occurrences.”
InfluentialReliability Analysis of a Smart Indoor Garden Developed Using IoT Technologies
Misaroş, M., Stan, O. P., Enyedi, S., et al. (2025) · International Conference on System Theory, Control and Computing (ICSTCC) · doi:10.1109/icstcc66753.2025.11240333
Truong, V.-D., Brace, W., Benzoni, G., et al. (2025) · Research in Engineering Design · doi:10.1007/s00163-025-00466-w
“The following sections describe these steps in detail (the detailed method for each phase is depicted in Fig.”
Triché, L. R., Futier, E., Carvalho, M. D., Piñol‐Domenech, N., Bodet-Contentin, L., Jabaudon, M., & Pereira, B. (2023) · Figshare · doi:10.6084/m9.figshare.23819841
Predicting the Impact of a Vulnerability with Markov Chains
Dimitrov, W. A., Ostrovska, T., Ruzhenov, V., et al. (2025) · doi:10.1109/bdkcse67969.2025.11300526
Emerging Semantic Segmentation from Positive and Negative Coarse Label Learning
Zhang, L., Wu, F., Thirunavukarasu, A., Bronik, K., Nichols, T., & Papież, B. W. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-09513-8_23
Wang, J., Kolog, S. D., & Alhassan, E. A. (2026) · The European Physical Journal Plus · doi:10.1140/epjp/s13360-026-07300-9
Computation-bandwidth-memory trade-offs: a unified paradigm for AI infrastructure
Fan, Y., Weng, Q., & Li, X. (2026) · Vicinagearth. · doi:10.1007/s44336-026-00041-4 · Cited by 4
Forecasting step-like reservoir landslide using a physical-mechanical-numerical framework
Zhang, J., Zhang, S., Tang, H., Liang, K., & Li, Q. (2026) · Journal of Rock Mechanics and Geotechnical Engineering · doi:10.1016/j.jrmge.2025.10.040 · Cited by 1
Evolving Markov Chains: Online Mode Discovery and Recognition From Data Streams
Coşkun, K., Tümer, B., Hiller, B. C., et al. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3660046
“The simplest offline way to build the stochastic matrix of a Markov chain from a sequence of observations is to calculate the transition frequencies and normalize them by the total number of transitions originating from the corresponding state [13].”
Zubair, M., Kabir, A., & Nabi, J. (2025) · Advances in Science and Engineering Technology International Conferences (ASET) · doi:10.1109/aset66891.2025.11428124
Conditioning on posterior samples for flexible frequentist goodness-of-fit testing
Bhaduri, R., Bhattacharyya, A., Barber, R. F., et al. (2026) · Biometrika · doi:10.1093/biomet/asag019
“These techniques, and the extent to which they approximate i.i.d. sampling from the posterior sampling, are well-studied in the Bayesian literature (Gelman et al., 1995; Gagniuc, 2017; Barber, 2012), so we do not explore their theoretical properties further here.”
Metadata
- Authors
- Gagniuc, P. A.
- Corresponding author
- Gagniuc, Paul A
- ORCID
- Gagniuc, Paul A
- Venue
- Wiley eBooks
- Publisher
- Wiley
- Published
- 2017-06-22
- OpenAlex
- W4235646468
- Cited by
- 415
- References
- 0
- Citations by year
- 2026: 19 · 2025: 37 · 2024: 64 · 2023: 60 · 2022: 54 · 2021: 68 · 2020: 58 · 2019: 38 · 2018: 17
- FWCI
- 7.99
- Type
- book
- Language
- en
- Open access
- closed
- Locations
- Wiley eBooks (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Mathematics › Mathematical Physics › Stochastic processes and statistical mechanics
- Keywords
- Markov chains, stochastic processes, transition probabilities, Markov property
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review