Markov Chains: From Theory to Implementation and Experimentation
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Markov model was leveraged to construct a process based on discrete-time state transitions and action rewards, named Markov Decision Process (MDP), which formalizes sequential decision-making problems in a fully observable, controlled environment.
“In addition, the Markov model [159, 296] (elaborated 50 years earlier) was leveraged to construct a process based on discrete-time state transitions and action rewards, named Markov Decision Process (MDP), which formalizes sequential decision-making problems in a fully observable, controlled environment [46].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [159]
Cited by
Sorted by citations
Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036 · see citing works
“In addition, the Markov model [159, 296] (elaborated 50 years earlier) was leveraged to construct a process based on discrete-time state transitions and action rewards, named Markov Decision Process (MDP), which formalizes sequential decision-making problems in a fully observable, controlled environment [46].”
MethodXiao, Z., Fu, X., Zhang, L., & Goh, R. S. M. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2908191 · Cited by 205 · see citing works
“Markov model has been applied for the prediction purpose in many fields [178].”
MethodLearning with Biased Complementary Labels
Yu, X., Liu, T., Gong, M., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_5 · Cited by 186 · see citing works
“Note that transition matrix is also widely exploited in Markov chains [7] and has many applications in machine learning, such as learning with label noise [21, 26, 22].”
BackgroundBABEL: Bodies, Action and Behavior with English Labels
Punnakkal, A. R., Chandrasekaran, A., Athanasiou, N., Quirós-Ramírez, M. A., & Black, M. J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00078 · Cited by 191 · see citing works
“Concretely, we compute P ( a i | a i − 1 , a i − 2 , a i − 3 ) , an order 3 Markov Chain [9], and observe in Table 2 that random walks along this chain generate plausible action sequences for human movement.”
Sampling methods for solving Bayesian model updating problems: A tutorial
Lye, A., Cicirello, A., & Patelli, E. (2021) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.107760 · Cited by 181 · see citing works
“;N whereby the value of hiþ1 depends only on the previous value hi [98].”
BackgroundDeep Learning for Source Code Modeling and Generation
Le, T. H. M., Chen, H., & Babar, M. A. (2020) · ACM Computing Surveys · doi:10.1145/3383458 · Cited by 175 · see citing works
“Different from the statistical construction of language models [22, 77], where the probability distribution of transition states of a Markov model [68] is learned, grammar inference determines an explicit representation of a programming language.”
BackgroundBayesian learning for neural networks: an algorithmic survey
Magris, M., & Iosifidis, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10443-1 · Cited by 166 · see citing works
“An accessible introduction to Markov Chains can be found in [Gagniuc, 2017], for a dedicated monograph on MCMC methods oriented toward Bayesian statistics and applications see, e.g., [Gamerman and Lopes, 2006].”
MethodSánchez, P. M. S., Valero, J. M. J., Celdrán, A. H., Bovet, G., Pérez, M. G., & Pérez, G. M. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3064259 · Cited by 149 · see citing works
“Finite state machines , such as Markov Models [104] and n-gram models [105], describe the sequential logic followed by a certain entity and predict its future status based on the previous ones.”
BackgroundA Review of Rule Learning-Based Intrusion Detection Systems and Their Prospects in Smart Grids
Liu, Q., Hagenmeyer, V., & Keller, H. B. (2021) · IEEE Access · doi:10.1109/access.2021.3071263 · Cited by 138 · see citing works
“In Markov chains, the conditional probability of choosing a next system state only depends on the current system state, which is known as Markov property [84].”
BackgroundYang, C., Wang, H., Yang, J., Yao, H., He, T., Bai, J., Guang, T., Cheng, H., Yan, J., & Qu, L. (2022) · Advanced Materials · doi:10.1002/adma.202205249 · Cited by 110 · see citing works
Future Hydrogen Markets for Transportation and Industry: The Impact of CO2 Taxes
Cerniauskas, S., Grube, T., Praktiknjo, A., Stolten, D., & Robinius, M. (2019) · Energies · doi:10.3390/en12244707 · Cited by 103 · see citing works
Walrasian Equilibrium-Based Multiobjective Optimization for Task Allocation in Mobile Crowdsourcing
Wang, Y., Cai, Z., Zhan, Z., et al. (2020) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2020.2995760 · Cited by 93 · see citing works
Farahani, A., Wallbaum, H., & Dalenbäck, J.-O. (2018) · Construction Management and Economics · doi:10.1080/01446193.2018.1512750 · Cited by 81 · see citing works
“…conditions on the life expectancy of building components, Schroeder’s deterioration function is modified using the main property of a discrete-time Markov chain process (deterioration); i.e. the condition of a system in the future depends on the current condition state of the system (Gagniuc 2017).”
MethodLocalization, Mapping, Navigation, and Inspection Methods in In-Pipe Robots: A Review
Kazeminasab, S., Sadeghi, N., Janfaza, V., et al. (2021) · IEEE Access · doi:10.1109/access.2021.3130233 · Cited by 79 · see citing works
Automated Stoichiometry Analysis of Single-Molecule Fluorescence Imaging Traces via Deep Learning
Xu, J., Qin, G., Luo, F., Wang, L., Zhao, R., Li, N., Yuan, J., & Fang, X. (2019) · Journal of the American Chemical Society · doi:10.1021/jacs.9b00688 · Cited by 77 · see citing works
Future climates: Markov blankets and active inference in the biosphere
Rubin, S., Parr, T., Costa, L. D., & Friston, K. J. (2020) · Journal of The Royal Society Interface · doi:10.1098/rsif.2020.0503 · Cited by 61 · see citing works
Astromers: Nuclear Isomers in Astrophysics*
Misch, G. W., Ghorui, S. K., Banerjee, P., Sun, Y. R., & Mumpower, M. R. (2020) · The Astrophysical Journal Supplement Series · doi:10.3847/1538-4365/abc41d · Cited by 64 · see citing works
“Our graph represents an irreducible Markov chain (every state is reachable from every other state by at least one path), so all states are members of a single communicating class (Gagniuc 2017).”
BackgroundGenomics and data science: an application within an umbrella
Navarro, F. C. P., Mohsen, H., Yan, C., Li, S., Gu, M., Meyerson, W. U., & Gerstein, M. (2019) · Genome biology · doi:10.1186/s13059-019-1724-1 · Cited by 62 · see citing works
Structural-functional brain network coupling predicts human cognitive ability
Popp, J. L., Thiele, J. A., Faskowitz, J., Seguin, C., Sporns, O., & Hilger, K. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120563 · Cited by 63 · see citing works
Collaborative Neural Solution for Time-Varying Nonconvex Optimization With Noise Rejection
Wei, L., & Jin, L. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3369482 · Cited by 58 · see citing works
“Lemma 1: [34] Consider that a finite Markov chain as in Definition 3.”
BackgroundAn interdisciplinary comparison of sequence modeling methods for next-element prediction
Tax, N., Teinemaa, I., & van Zelst, S. V. (2018) · Journal of Software and Systems Modeling · doi:10.1007/s10270-020-00789-3 · Cited by 64 · see citing works
The dynamics of cyclones in the twentyfirst century: the Eastern Mediterranean as an example
Hochman, A., Alpert, P., Kunin, P., Rostkier‐Edelstein, D., Harpaz, T., Saaroni, H., & Messori, G. (2019) · Climate Dynamics · doi:10.1007/s00382-019-05017-3 · Cited by 53 · see citing works
“The dynamics of Cyprus Lows are further studied using transition probability matrices, i.e., Markov chains (Gagniuc 2017).”
MethodMusah, M. (2022) · Environmental Science and Pollution Research · doi:10.1007/s11356-022-19994-2 · Cited by 50 · see citing works
“The variables being stationary after their first differentiation imply a shift in time did not cause a change in the shape of their distributions (Gagniuc 2017).”
RetractedRincón, L. F., Moscoso, Y. F. M., Hamami, A. E. A., Matos, J. C., & Bastidas‐Arteaga, E. (2024) · Buildings · doi:10.3390/buildings14030562 · Cited by 52 · see citing works
Trust as a determinant of social welfare in the digital economy
Shah, S. S., & Shah, S. A. H. (2024) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01238-5 · Cited by 48 · see citing works
Transfer Learning for Clinical Sleep Pose Detection Using a Single 2D IR Camera
Mohammadi, S. M., Enshaeifar, S., Hilton, A., et al. (2020) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2020.3048121 · Cited by 48 · see citing works
“The Markov chain is a stochastic process x = { x 1 , x 2 , · · · , x n } that undergoes the transition from one to another state wherein for this work the states are presented as sleep poses and empty bed: S = { s s , s l , s r , s p , s e } (where x i ∈ S ) [34], [35].”
BackgroundLanger-Osuna, J. M., Munson, J., Gargroetzi, E. C., et al. (2020) · Educational Studies in Mathematics · doi:10.1007/s10649-020-09962-3 · Cited by 47 · see citing works
“We chose to represent the data as Markov chains, a probabilistic model where the likelihood of a future state is calculated as depending on the immediately previous state (Gagniuc, 2017).”
MethodThe Surprising Effectiveness of Linear Models for Visual Foresight in Object Pile Manipulation
Suh, H. J. T., & Tedrake, R. (2021) · Springer proceedings in advanced robotics · doi:10.1007/978-3-030-66723-8_21 · Cited by 43 · see citing works
Dibaj, M., Javadi, A. A., Akrami, M., Ke, K., Farmani, R., Tan, Y., & Chen, A. (2020) · Hydrogeology Journal · doi:10.1007/s10040-020-02172-4 · Cited by 41 · see citing works
“Due to the lack of observed salinity data, a steadystate model was first developed in order to provide initial salinity data (Gagniuc 2017).”
MethodA generalised framework for detailed classification of swimming paths inside the Morris Water Maze
Vouros, A., Gehring, T. V., Szydłowska, K., Janusz, A., Tu, Z., Croucher, M., Lukasiuk, K., Konopka, W., Sandi, C., & Vasilaki, E. (2018) · Scientific Reports · doi:10.1038/s41598-018-33456-1 · Cited by 40 · see citing works
A generalised framework for detailed classification of swimming paths inside the Morris Water Maze
Vouros, A., Gehring, T. V., Szydlowska, K., Janusz, A., Tu, Z., Croucher, M., Lukasiuk, K., Konopka, W., Sandi, C., & Vasilaki, E. (2018) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 38 · see citing works
Voice modulatory cues to structure across languages and species
Matzinger, T., & Fitch, W. (2021) · Philosophical Transactions of the Royal Society of London. Biological Sciences · doi:10.1098/rstb.2020.0393 · Cited by 13 · see citing works
“Using transitional probabilities to infer characteristics of a signal appears to be a very general behaviour since in basically any domain of action, including animal vocalizations, certain events are more likely to follow each other than others [98,99].”
BackgroundLux, B. K., Andrews‐Hanna, J. R., Han, J.-H., Lee, E., & Woo, C. (2022) · Science Advances · doi:10.1126/sciadv.abn8616 · Cited by 36 · see citing works
Schlaich, A., Barrat, J., & Coasne, B. (2025) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00406 · Cited by 36 · see citing works
Viana, C. M., & Rocha, J. (2020) · Sustainability · doi:10.3390/su12104332 · Cited by 36 · see citing works
An Adaptive Network Security System for IoT-Enabled Maritime Transportation
Gyamfi, E., Ansere, J. A., Kamal, M., Tariq, M., & Jurcut, A. D. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3159450 · Cited by 36 · see citing works
“According to [24], we can express Markov property as the dependent probability of IoT devices selecting a MEC server for offloading their task when there is a new attack (and”
MethodAlsharif, M. A., Alzandi, A. A., Shrahily, R. Y., et al. (2022) · Forests · doi:10.3390/f13101530 · Cited by 34 · see citing works
The Health Status Transition and Medical Expenditure Evaluation of Elderly Population in China
Wang, L., Tang, Y., Roshanmehr, F., Bai, X., Taghizadeh‐Hesary, F., & Hesary, F. T. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18136907 · Cited by 32 · see citing works
Musah, M. (2022) · Environment Development and Sustainability · doi:10.1007/s10668-022-02521-1 · Cited by 31 · see citing works
“This implies shifts in time had no effect on the distributions of the series (Gagniuc, 2017).”
BackgroundApplication-Layer DDoS Defense with Reinforcement Learning
Feng, Y., Li, J., & Nguyen, T. H. (2020) · IEEE/ACM International Symposium on Quality of Service (IWQoS) · doi:10.1109/iwqos49365.2020.9213026 · Cited by 30 · see citing works
“3) Markov-based methods: A Markov model is a stochastic model used to model randomly changing events in probability theory [10].”
MethodYu, P., Tang, J., Xia, Z., et al. (2023) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2022.3233421 · Cited by 30 · see citing works
“Markov model is a stochastic model for probabilistic systems [42], in which it is supposed that the future states only depend on the current states.”
BackgroundMusic Sequence Prediction with Mixture Hidden Markov Models
Li, T., Choi, M., Fu, K., & Lin, L. (2019) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata47090.2019.9005695 · Cited by 28 · see citing works
“Markovian property is an assumption on which the future state depends only on a current state which can be written as [3]”
BackgroundA unicellular walker controlled by a microtubule-based finite-state machine
Larson, B. T., Garbus, J., Pollack, J. B., & Marshall, W. F. (2022) · Current Biology · doi:10.1016/j.cub.2022.07.034 · Cited by 30 · see citing works
Ben-Marzouk, M., Pélissier, S., Clerc, G., et al. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3077671 · Cited by 28 · see citing works
Deriving Economic Potential and GHG Emissions of Steel Mill Gas for Chemical Industry
Collis, J., Strunge, T., Steubing, B. R. P., Zimmermann, A., & Schomäcker, R. (2021) · Frontiers in Energy Research · doi:10.3389/fenrg.2021.642162 · Cited by 30 · see citing works
“A discrete-time Markov chain is implemented to simulate flaring patterns across a range of potential steel plants (Mcbratney and Everitt, 2002; Towers, 2016; Gagniuc, 2017).”
BackgroundCommunication and Energy Efficient Decentralized Learning Over D2D Networks
Liu, S., Yu, G., Wen, D., Chen, X., Bennis, M., & Chen, H. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3271854 · Cited by 28 · see citing works
“5: end for • X is a doubly stochastic matrix, which satisfies X1 = 1 and 1 T X = 1 T , where 1 = [1] N × 1 and X are nonnegative real numbers, as defined in [29] and [30].”
BackgroundWang, L., Chen, A., Zhang, Y., et al. (2020) · Diagnostics · doi:10.3390/diagnostics10040217 · Cited by 27 · see citing works
Bahati, H. K., Ogenrwoth, A., & Sempewo, J. I. (2021) · Journal of Water and Climate Change · doi:10.2166/wcc.2021.273 · Cited by 27 · see citing works
Muñoz, R. N., Leung, A., Zecevik, A., Pollock, F. A., Cohen, D., van Swinderen, B., Tsuchiya, N., & Modi, K. K. (2020) · Physical Review Research · doi:10.1103/physrevresearch.2.023219 · Cited by 23 · see citing works
An Approach for a Next‐Word Prediction for Ukrainian Language
Shakhovska, K., Dumyn, I., Kryvinska, N., et al. (2021) · Wireless Communications and Mobile Computing · doi:10.1155/2021/5886119 · Cited by 25 · see citing works
Wang, L., Laird-Fick, H. S., Parker, C. J., et al. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02633-8 · Cited by 25 · see citing works
Al-Shaar, W., Gérard, J. A., Nehme, N., et al. (2020) · Modeling Earth Systems and Environment · doi:10.1007/s40808-020-00971-y · Cited by 24 · see citing works
“1 3 More info about the Markov Chain mathematical equations are explained in the following references: (Hamad et al. 2018; Iacono et al. 2017; Rozario et al. 2017; Gagniuc 2017; Parsa et al. 2016; Han et al. 2015; Subedi et al. 2013; Takada et al. 2010; Grinstead and Snell 2006; Ching and Ng 2006; Levinson and Chen 20…”
BackgroundA Systematic Review of Simulation Models to Track and Address the Opioid Crisis
Cerdá, M., Jalali, M. S., Hamilton, A. D., DiGennaro, C., Hyder, A., Santaella-Tenorio, J., Kaur, N., Wang, C., & Keyes, K. M. (2021) · Epidemiologic Reviews · doi:10.1093/epirev/mxab013 · Cited by 24 · see citing works
“They include statistical simulation approaches such as Markov models, which are stochastic models used to model transitions across states, assuming that future states only depend on the current state (18).”
Maurya, N. K., Rafi, S., & Shamoo, S. (2022) · GeoJournal · doi:10.1007/s10708-022-10593-9 · Cited by 24 · see citing works
“Markov models the state of a system with a random variable that changes through time (Gagniuc, 2017).”
BackgroundEntropy-Based Effect Evaluation of Delineators in Tunnels on Drivers’ Gaze Behavior
Han, X., Shao, Y., Yang, S., et al. (2020) · Entropy · doi:10.3390/e22010113 · Cited by 23 · see citing works
Revisiting Problem-Solution Co-Evolution in the Context of Team Conceptual Design Activity
Martinec, T., Škec, S., Perišić, M. M., et al. (2020) · Applied Sciences · doi:10.3390/app10186303 · Cited by 23 · see citing works
“The sequenc s wer inv stigat d by determining the probabilities of one stat tr nsi io following another; that is, what is the probability of either analysis, sy the or evaluati n to follow the previous type of state transition (Markov analysis [75]).”
BackgroundMethodInfluentialVeisi, H. (2023) · Industrial and applied mathematics · doi:10.1007/978-981-19-6553-1_1 · Cited by 25 · see citing works
Airborne Radar Anti-Jamming Waveform Design Based on Deep Reinforcement Learning
Zheng, Z., Li, W., & Zou, K. (2022) · Sensors · doi:10.3390/s22228689 · Cited by 22 · see citing works
“space, A is the action space, P is the state transfer probability, R is the payoff function, and is the discount factor [46].”
BackgroundA state-transition model of team conceptual design activity
Martinec, T., Škec, S., Horvat, N., et al. (2019) · Research in Engineering Design · doi:10.1007/s00163-018-00305-1 · Cited by 22 · see citing works
Automated model generation framework for material flow simulations of production systems
May, M. C., Nestroy, C., Overbeck, L., & Lanza, G. (2023) · International Journal of Production Research · doi:10.1080/00207543.2023.2284833 · Cited by 26 · see citing works
“A Markov chain is a sequence of states where the probability P for next state S t + 1 is always conditionally dependent on the current state S t , but not on states before the current state (Gagniuc 2017).”
BackgroundTriché, L. R., Futier, E., Carvalho, M. D., Piñol‐Domenech, N., Bodet-Contentin, L., Jabaudon, M., & Pereira, B. (2023) · Critical Care · doi:10.1186/s13054-023-04562-y · Cited by 21 · see citing works
MaBouDi, H., Solvi, C., & Chıttka, L. (2020) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2020.00137 · Cited by 20 · see citing works
Dehghani, N. L., Fereshtehnejad, E., & Shafieezadeh, A. (2020) · Earthquake Engineering & Structural Dynamics · doi:10.1002/eqe.3359 · Cited by 20 · see citing works
Detecting and Identifying Insider Threats Based on Advanced Clustering Methods
Ņikiforova, O., Romānovs, A., Zabiniako, V., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3365424 · Cited by 20 · see citing works
Mohammadi, A., Karımzadeh, S., Kamran, K. V., et al. (2020) · Sensors · doi:10.3390/s20247010 · Cited by 20 · see citing works
Anonymization of Daily Activity Data by Using ℓ-diversity Privacy Model
Parameshwarappa, P., Chen, Z., & Koru, G. (2021) · ACM Transactions on Management Information Systems · doi:10.1145/3456876 · Cited by 19 · see citing works
Network Digital Twins: A Systematic Review
Verdecchia, R., Scommegna, L., Picano, B., Becattini, M., & Vicario, E. (2024) · IEEE Access · doi:10.1109/access.2024.3453034 · Cited by 24 · see citing works
“In the realm of NDTs, we have identified a particular type of model that is frequently used beyond neural networks, namely Markovian models [39].”
MethodAnalyzing Sequence Data with Markov Chain Models in Scientific Experiments
Paxinou, E., Kalles, D., Panagiotakopoulos, C. T., & Verykios, V. S. (2021) · SN Computer Science · doi:10.1007/s42979-021-00768-5 · Cited by 23 · see citing works
Lefebvre, D., Seatzu, C., Hadjicostis, C. N., et al. (2021) · Discrete Event Dynamic Systems · doi:10.1007/s10626-021-00348-y · Cited by 19 · see citing works
“In particular, Poisson processes, birth and death processes, and parallel queuing systems are examples of uses of continuous-time Markov models in the domain of computer science and networked systems where cyber-attacks may occur (Gagniuc 2017).”
BackgroundMethodAccent classification in human speech biometrics for native and non-native english speakers
Bird, J. J., Wanner, E. F., Ekárt, A., et al. (2019) · ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3316782.3322780 · Cited by 19 · see citing works
Model‐based hypothesis testing of uncertain software systems
Camilli, M., Gargantini, A., & Scandurra, P. (2020) · Software Testing Verification and Reliability · doi:10.1002/stvr.1730 · Cited by 18 · see citing works
“In fact, because of the Markov property [41], the learning problem using Bayesian inference usually reduces to the independent learning of different independent categorical distributions [13].”
BackgroundDifferential scanning calorimetry (DSC) in fiber research
Menczel, J. D., & Kohl, W. S. (2020) · Elsevier eBooks · doi:10.1016/b978-0-08-100572-9.00003-3 · Cited by 19 · see citing works
The Applicability of the Extended Markov Chain Model to the Land Use Dynamics in Lebanon
Al-Shaar, W., Nehme, N., & Gérard, J. A. (2020) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-020-04645-w · Cited by 19 · see citing works
Yakusheva, O., Costa, D. K., Bobay, K. L., Parada, J. P., & Weiss, M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0218755 · Cited by 19 · see citing works
“We estimated a transition probability matrix [36] and examined the probabilities of remaining in the same quartile of the distribution of the adjusted individual CAABU-free rates in both years (a sign of consistent nurse CAABU-prevention performance), versus moving to another quartile in 2016.”
MethodTırkolaee, E. B., Aydın, N. S., & Mahdavi, I. (2022) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2022.3210879 · Cited by 18 · see citing works
Vogl, M. (2022) · SN Business & Economics · doi:10.1007/s43546-022-00359-3 · Cited by 18 · see citing works
“Autoregressive processes yield the vast majority of presented models, building on the AR model, which in the case of an AR(1) specification can be approximated via a two-step Markov chain (Florescu 2015; Gagniuc, 2017).”
BackgroundMethodInfluentialPast and Projected Weather Pattern Persistence with Associated Multi-Hazards in the British Isles
de luca, P., Harpham, C., Wilby, R. L., et al. (2019) · Atmosphere · doi:10.3390/atmos10100577 · Cited by 17 · see citing works
Using Instrumental Variables to Measure Causation over Time in Cross-Lagged Panel Models
Singh, M., Verhulst, B., Vinh, P., Zhou, Y., Castro-de-Araujo, L. F. S., Hottenga, J., Pool, R., de Geus, E. J. C., Vink, J. M., Boomsma, D. I., et al. (2024) · Multivariate Behavioral Research · doi:10.1080/00273171.2023.2283634 · Cited by 18 · see citing works
“In a stationary time-series, the time index ( i ) has essentially no impact on the cross-sectional covariances ( R i Þ among the variables (Gagniuc, 2017; Ryabko, 2019).”
BackgroundHao, H., Hasan, K. A., Qin, H., Macedo, M., Tian, Y., Ding, S. H. H., & Hassan, A. E. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10540-x · Cited by 19 · see citing works
“2, we use a Markov model (Gagniuc, 2017) to analyze the conversation flow patterns by plotting a Markov Transition Graph .”
Howard-Spink, E., Hayashi, M., Matsuzawa, T., Schofield, D. P., Gruber, T., & Biro, D. (2024) · PeerJ · doi:10.7717/peerj.18484 · Cited by 16 · see citing works
PALOT: Profiling and Authenticating Users Leveraging Internet of Things
Nespoli, P., Zago, M., Celdrán, A. H., et al. (2019) · Sensors · doi:10.3390/s19122832 · Cited by 17 · see citing works
“Formally, a Markov chain is defined as a stochastic model that describes a sequence of (possible) events whose probability of occurrence of a particular event depends on the state attained in the previous event [32].”
BackgroundFormal specification and verification of decentralized self-adaptive systems using symmetric nets
Camilli, M., & Capra, L. (2021) · Discrete Event Dynamic Systems · doi:10.1007/s10626-021-00343-3 · Cited by 17 · see citing works
“This (2017) – whose states are the SPN’s reachable markings.”
Persistence and disadoption of sustainable agricultural practices in the Mississippi Delta region
Pathak, S., Wang, H., Tran, D. Q., et al. (2023) · Agronomy Journal · doi:10.1002/agj2.21519 · Cited by 16 · see citing works
CASCADENET: An LSTM Based Deep Learning Model for Automated ICD-10 Coding
Azam, S. S., Raju, M., Pagidimarri, V., et al. (2019) · Lecture notes in networks and systems · doi:10.1007/978-3-030-12385-7_6 · Cited by 16 · see citing works
Intelli-Paint: Towards Developing More Human-Intelligible Painting Agents
Singh, J., Smith, C., Echevarria, J., & Zheng, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19787-1_39 · Cited by 18 · see citing works
“where t̃ P r0, 1s is the normalized episode timestep, pwmin, hminq are the minimum attention window dimensions and p∆Wt “ ∆xt, ∆yt, ∆wt, ∆htq P R(4) are successive Markovian [10] updates predicted by the RL agent.”
BackgroundModeling the Effort and Learning Ability of Students in MOOCs
Gao, L., Zhao, Z., Qi, L., Liang, Y., & Du, J. (2019) · IEEE Access · doi:10.1109/access.2019.2937985 · Cited by 16 · see citing works
Flinders, C., Barnhart, B., Morrison, E., Anderson, P. D., & Landis, W. G. (2025) · Integrated Environmental Assessment and Management · doi:10.1093/inteam/vjaf016 · Cited by 17 · see citing works
A Survey on Software-Defined Networking-Based 5G Mobile Core Architectures
Ghaffar, A. A. A., Mahmoud, A. S. H., Sheltami, T., et al. (2022) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-022-07183-9 · Cited by 15 · see citing works
“The authors propose an estimation-based handover procedure where the SDN controller forecasts the next base station for a UE based on the Markov chain model [93] and inform the UE about the expected next base station.”
BackgroundDabčević, Z., Škugor, B., Topić, J., & Deur, J. (2022) · Energies · doi:10.3390/en15114108 · Cited by 15 · see citing works
Cortesi, M., Pasini, A., Furini, S., & Giordano, E. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00575 · Cited by 14 · see citing works
“The DTMC (Gagniuc, 2017) was automatically identified through the analysis of the stable states of the BN.”
MethodGeometric Approach to Quantum Statistical Inference
Jarzyna, M., & Kolodynski, J. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.3017469 · Cited by 16 · see citing works
“discrete PDs p′ and p as non-negative and normalized vectors, the map S defines a left-stochastic transition matrix Si j with non-negative entries (∀i, j : Si j ≥ 0) and columns summing up to 1 (∀j : ∑i Si j = 1) [29].”
BackgroundRAPT: A Robust Attack Path Tracing Algorithm to Mitigate SYN-Flood DDoS Cyberattacks
Al-Arnaout, Z., Mostafa, N., Alabed, S. J., et al. (2022) · Sensors · doi:10.3390/s23010102 · Cited by 14 · see citing works
Sarı, F. (2020) · Journal of Apicultural Science · doi:10.2478/jas-2020-0020 · Cited by 14 · see citing works
“The Markov model is a stochastic model that has been frequently used to reveal the underlying complex reasons for land-use changes describing a sequence of possible events (Muller & Middleton, 1994; Gagniuc, 2017).”
MethodMonitoring forest cover and land use change in the Congo Basin under IPCC climate change scenarios
Yuh, Y. G., N’Goran, P. K., Kross, A., Heurich, M., Matthews, H. D., & Turner, S. E. (2024) · PLoS ONE · doi:10.1371/journal.pone.0311816 · Cited by 14 · see citing works
Chiang, E., Nguyen, D., Towler, A., Haas, M., & Grieve, J. (2021) · International Journal of Speech Language and the Law · doi:10.1558/ijsll.41446 · Cited by 13 · see citing works
Masedu, F., Vagnetti, R., Pino, M. C., Valenti, M., & Mazza, M. (2021) · Brain Sciences · doi:10.3390/brainsci12010010 · Cited by 14 · see citing works
Generation of Musculoskeletal Ultrasound Images with Diffusion Models
Katakis, S., Barotsis, N., Kakotaritis, A., Tsiganos, P., Economou, G., Panagiotopoulos, E. C., & Panayiotakis, G. (2023) · BioMedInformatics · doi:10.3390/biomedinformatics3020027 · Cited by 15 · see citing works
Geiß, C. M., Maier, J., So, E., et al. (2024) · Natural hazards and earth system sciences · doi:10.5194/nhess-24-1051-2024 · Cited by 13 · see citing works
Zhang, L., Gan, J. Q., Zhu, Y., Wang, J., & Wang, H. (2020) · Human Brain Mapping · doi:10.1002/hbm.25035 · Cited by 13 · see citing works
“Markov model describes the underlying dynamical nature of a system that follows a chain of linked states, where what happens at any time instant depends only on the preceding one (Gagniuc, 2017).”
BackgroundYuan, J., Zhao, R., Xu, J., Cheng, M., Qin, Z., Kou, X., & Fang, X. (2020) · Communications Biology · doi:10.1038/s42003-020-01389-z · Cited by 13 · see citing works
- Authors
- Gagniuc, P. A.
- Corresponding author
- Gagniuc, Paul A
- ORCID
- Gagniuc, Paul A
- Venue
- Wiley eBooks
- Publisher
- Wiley
- Published
- 2017-06-22
- OpenAlex
- W4235646468
- Cited by
- 415
- References
- 0
- Citations by year
- 2026: 19 · 2025: 37 · 2024: 64 · 2023: 60 · 2022: 54 · 2021: 68 · 2020: 58 · 2019: 38 · 2018: 17
- FWCI
- 7.99
- Type
- book
- Language
- en
- Open access
- closed
- Locations
- Wiley eBooks (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Mathematics › Mathematical Physics › Stochastic processes and statistical mechanics
- Keywords
- Markov chains, stochastic processes, transition probabilities, Markov property
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
