The DOI matches its Crossref record.
Markov Chains: From Theory to Implementation and Experimentation
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In addition, the Markov model [159, 296] (elaborated 50 years earlier) was leveraged to construct a process based on discrete-time state transitions and action rewards, named Markov Decision Process (MDP), which formalizes sequential decision-making problems in a fully observable, controlled environment [46].”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [159]
Citations
Cited by
Sorted by citations
Items 201–300 of 413
Memory in quantum dot blinking
Muñoz, R. N., Frazer, L., Yuan, G., et al. (2022) · Physical review. E · doi:10.1103/physreve.106.014127 · Cited by 4
In-Memory Computing for AI Accelerators: Challenges and Solutions
Krishnan, G., V, Mandal, S. K., Chakrabarti, C., Seo, J.-S., Ogras, U. Y., & Cao, Y. K. (2023) · Springer eBooks · doi:10.1007/978-3-031-19568-6_7 · Cited by 4
Chen, X., Yang, J., Chen, S., Wang, L., Jiang, M. X., & Zhao, Q. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3357631 · Cited by 4
“Inspired by information retrieval techniques [45], [63], we introduce novel learning objectives that involve propagating the intra-step attention measurements along the edges of the predicted solution graph, quantifying the impact of attention in achieving successful solution prediction.”
BackgroundMethodBadwi, I. M. (2024) · Journal of Urban Planning and Development · doi:10.1061/jupddm.upeng-4502 · Cited by 4
Osakwe, I., Chen, G., Fan, Y., Raković, M., Singh, S., Molenaar, I., & Gašević, D. (2024) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3636555.3636915 · Cited by 4
“To detect SRL strategies, the inferred SRL processes for each learner were processed using First Order Markov Models [10], and the resulting transition matrices were clustered.”
MethodVariational Connectionist Temporal Classification for Order-Preserving Sequence Modeling
Nan, Z., Dang, T., Sethu, V., & Ahmed, B. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447530 · Cited by 4
“…among the latent variables z t , it can be assumed to be Markovian [24].”
BackgroundMarkovian Maintenance Planning of Ship Propulsion System Accounting for CII and System Degradation
Garbatov, Y., & Georgiev, P. (2024) · Energies · doi:10.3390/en17164123 · Cited by 4
Robust Recovery of Sparse Signal from Compressed Measurements for Wireless Sensor Networks
Bapat, K. A., & Chakraborty, M. (2022) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas48785.2022.9937776 · Cited by 4
“The resulting adjacency matrix was normalized so as to make it a right stochastic [25].”
BackgroundA PageRank-like measure for evaluating process flexibility
Cui, F., Wang, C., & Li, L. (2022) · IISE Transactions · doi:10.1080/24725854.2021.2010152 · Cited by 4
“…matrix diag ð 1 = b j Þ d j 2D , and W a is the diagonal matrix diag ð 1 = a i Þ s i 2S : Then we can obtain where The elements of each column in M sum up to a value of one, so M is an m (cid:1) m left stochastic matrix (Gagniuc, 2017, pp. 9 – 11), which determines how to transform R ð t (cid”
BackgroundMarroni, S. S., Santos, V. R., Castro, O. W., Tejada, J., Santos, J., de Oliveira, J. A. D., & Garcia-Cairasco, N. (2019) · bioRxiv · doi:10.1101/638452 · Cited by 2
Chapman-Kolmogorov test for estimating memory length of two coupled processes
Motahari, H., Noudehi, M. G., Jamali, T., & Jafari, G. (2025) · Scientific Reports · doi:10.1038/s41598-025-92238-8 · Cited by 3
Li, G., Zhang, S., Han, Y., et al. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3369854 · Cited by 3
“However, the jumping among models is impossible previously [11], [12], [13].”
BackgroundSocial Network Structure: Groups and Their Influence
Jin, J. (2019) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc.2019.8832533 · Cited by 3
“Proposition 6 [16] matrix P is an irreducible Markov matrix. q = ( q 1 , . . . , q n ) is the normalized left eigenvector of eigenvalue λ = 1 of the matrix P , then q i > 0( ∀ i ) If the opinion reference network is strongly connected, then its direct influence matrix P is an irreducible matrix. q…”
BackgroundWang, Z., Ye, Z., Liu, W., et al. (2019) · IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS) · doi:10.1109/idaacs.2019.8924361 · Cited by 3
Li, X., Xue, Q., Zhao, J., & Wang, D. (2020) · IEEE Access · doi:10.1109/access.2020.2988677 · Cited by 3
“The Markov process is a stochastic model describing a sequence of possible events in which the probability of each event only depends on the state attained in the previous event [19].”
BackgroundThe volatility in financial time series based on granule complex network
Xueyi, L., & Luo, C. (2021) · International Journal of Modern Physics C · doi:10.1142/s0129183121501163 · Cited by 3
Markovian modeling of match results with an application
Allawee, Z. Y. A., & Thanoon, B. Y. (2022) · Journal of Interdisciplinary Mathematics · doi:10.1080/09720502.2022.2069342 · Cited by 3
“If {R1, R2,…} represents the results of consecutive matches for a specific team, then the probabilistic model expressing the Markovian property can be formulated mathematically as follows; see e.g. Gagniuc (2017) [6]: 1 2 1( \ .”
BackgroundProcess Mining IPTV Customer Eye Gaze Movement Using Discrete-Time Markov Chains
Chen, Z., Zhang, S., McClean, S. I., et al. (2023) · Algorithms · doi:10.3390/a16020082 · Cited by 3
Computation of the steady-state probability of Markov chain evolving on a mixed state space
Zakrad, A.-E., & Nasroallah, A. (2023) · Monte Carlo Methods and Applications · doi:10.1515/mcma-2023-2003 · Cited by 3
Likelihood and decoding problems for mixed space hidden Markov model
Kroumbi, H., & Nasroallah, A. (2024) · Monte Carlo Methods and Applications · doi:10.1515/mcma-2024-2005 · Cited by 3
Innovation in a Turbulent World: The Case for Creative Leadership
Press, J., McLean, S., & McCauley, C. (2019) · Springer eBooks · doi:10.1007/978-3-030-19681-3_23 · Cited by 3
A Neural Word Embedding Approach to System Trace Reconstruction
Lakhani, K., & Narayan, A. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8914322 · Cited by 3
“The first order Markov model [42] was trained using entire data set and next event was predicted.”
MethodZhang, F., & Chang, S. J. (2019) · Canadian Journal of Forest Research · doi:10.1139/cjfr-2019-0140 · Cited by 3
Human-Activity Recognition with Smartphone Sensors
Ilisei, D., & Suciu, D. M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-40907-4_18 · Cited by 4
“The most used and accurate algorithms are k-Nearest Neighbors (k-NN) [8], Support Vector Machines (SVMs) [7], Random Forests (RF) [3], Gaussian Mixture Models (GMM) [2], Markov Chains [10] and Hidden Markov Models (HMM) [4].”
MethodGene Sets Analysis using Network PatternsPreprint
Linkowski, G., Blatti, C. A., Kalari, K. R., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/629816 · Cited by 3
“Typically, a transition probability matrix parameterizes a random walk on a network: if one considers a random walk starting at a given node and taking a single step at random along any of its incident edges, then the (i,j)th cell of the transition probability matrix prescribes the probability of transitioning from no…”
BackgroundWorthington, J., He, E., Caruana, M., Wade, S., de Graaff, B., Nguyen, A., George, J., Canfell, K., & Feletto, E. (2024) · medRxiv · doi:10.1101/2024.02.20.24303111 · Cited by 2
“However, these typically require stochastic evaluation,[38, 85, 86] which slows model performance and therefore limits the capacity for iterative model design and methods which explore wide parameter spaces, such as probabilistic sensitivity analysis.”
BackgroundUllah, M., Tuhin, M. K. I., Shapla, T., et al. (2024) · The Dhaka University Journal of Earth and Environmental Sciences · doi:10.3329/dujees.v12i2.73166 · Cited by 2
“By guiding the CA Markov model’s (Gagniuc, 2017; Ross, 2014; Ross, 1995; Meyn and Tweedie, 2012; Ethier and Kurtz, 1986) decision-making process, it enhances the accuracy and reliability of the projected wetland scenarios, enabling better-informed land management and conservation planning efforts.”
MethodPredicting Next Useful Location with Context-Awareness: The State-of-the-Art
Nezhadettehad, A., Zaslavsky, A., Rakib, A., Shaikh, S. A., Loke, S. W., Huang, G., & Hassani, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744653 · Cited by 3
“It serves to document the specifics of a communication event, encompassing calls and other telecommunication transactions such as text messages and data usage, that traverse through the said facility or device.”
Yu, F., Wu, G., Li, W., Wang, Y., Wang, J., Wang, M., Kong, D., Yang, J. G., Ren, X., Yang, Z., et al. (2025) · BMC Medicine · doi:10.1186/s12916-025-04556-3 · Cited by 2
Supply Chain Optimization via Generative Simulation and Iterative Decision Policies
Bai, H., Wang, H., Gong, N., et al. (2025) · Winter Simulation Conference · doi:10.1109/wsc68292.2025.11338940 · Cited by 2
Markov and Semi-Markov Models of Real-Time Quests in Information Security Education
Nissenbaum, O. V., Александровна, М. Е., Ishchukova, E., et al. (2019) · Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT) · doi:10.1109/usbereit.2019.8736621 · Cited by 2
“Then it is easy to compose for them ChapmanKolmogorov equations [13] and, having solved them, to obtain the states probabilities as a function of time.”
MethodUsing Markov Chains to Predict Productivity of Maize in Iraq for the Period (2019-2025)
Alani, L. A. F., & AL-Hiyali, A. D. K. (2021) · Applied Economics and Finance · doi:10.11114/aef.v8i5.5306 · Cited by 2
“Markov chain A Markov process is a stochastic process that satisfies the Markov property (Gagniuc 2017).”
Tracking the Temporal-Evolution of Supernova Bubbles in Numerical Simulations
Canducci, M., Taghribi, A., Mastropietro, M., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-91608-4_49 · Cited by 2
“Here we tackle a different problem - we propose a methodology for tracking the evolution of a time series of low-dimensional manifolds based on the first-order Markovian assumption [5], where the spatial structure of the manifold at each time-step is modelled via GTM.”
MethodMarkov chains, eigenvalues and the stability of economic growth processes
Delbianco, F., Fioriti, A., & Tohmé, F. (2022) · Empirical Economics · doi:10.1007/s00181-022-02276-8 · Cited by 2
“A salient feature of Markov chains is that they are memoryless stochastic processes in which the future is conditioned only on the current state of the system (cf. Privault 2018; Gagniuc 2017).”
BackgroundApplied Research Analysis of Game Theory and Markov chain
Niu, C., & Zhou, H. (2023) · Highlights in Science Engineering and Technology · doi:10.54097/hset.v47i.8173 · Cited by 2
Learning Markov Chain Models from Sequential Data Under Local Differential Privacy
Guner, E., & Gürsoy, M. E. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-51476-0_18 · Cited by 2
Optimizing on-site testing for post-tensioned concrete bridges
Zorzi, S., Bado, M. F., & Zonta, D. (2024) · Structure and Infrastructure Engineering · doi:10.1080/15732479.2024.2407951 · Cited by 2
“In the absence of this standardized foundation, in this work, the authors consider an exemplary prior distribution resulting from a Markov process (Gagniuc, 2017), as suggested in several works, such as Frangopol et al. (2004) and Shahraki (2017).”
Carnes, N. C., Biggs, A. T., Hamilton, J. A., Huffman, G., & Dunn, T. L. (2024) · The Journal of Defense Modeling and Simulation Applications Methodology Technology · doi:10.1177/15485129241301074 · Cited by 2
“18 Moreover, a Markov chain allows for an estimation from a sequence of events based on performance and subsequent transition between states 19,20 ).”
Al-Shaar, W., Lehmann, X., Bonin, O., Frankhauser, P., Seregina, T., & Al-Shaar, M. (2025) · Earth Systems and Environment · doi:10.1007/s41748-025-00690-3 · Cited by 5
“The combination of these two models involves the incorporation of historical LULC transition probabilities across space and time (Al-Shaar et al. 2020; Gagniuc 2017).”
Simulating Information Retrieval Test Collections
Hawking, D., Billerbeck, B., Thomas, P., & Craswell, N. (2020) · Synthesis lectures on information concepts, retrieval, and services · doi:10.2200/s01043ed1v01y202008icr071 · Cited by 5
Simulated annealing approach for outpatient scheduling in a haemodialysis unit
Sundar, A., Rahmin, N. A. A., Chen, C. Y., et al. (2022) · Mathematical Modeling and Computing · doi:10.23939/mmc2022.04.860 · Cited by 2
Emergent Turing Machine as a General Purpose Approximator
Zheng, Z., Wu, X., & Weng, J. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489396 · Cited by 2
Dynamic Scheduling Based on Predicted Inventory Variation Rate for Public Bicycle System
Gao, L., Xu, W. H., & Duan, Y. (2019) · Sustainability · doi:10.3390/su11071885 · Cited by 2
“According to the stationary distribution of the Markov process [32], we obtain: { φb = φb·P φr = Q·φr (3)”
An epidemic model of Covid-19 disease with variable spreading
Popov, G., & Наков, О. (2021) · AIP conference proceedings · doi:10.1063/5.0041742 · Cited by 2
The Hybrid Incidence Susceptible-Transmissible-Removed Model for Pandemics
Benjamin, R. L. (2022) · Acta Biotheoretica · doi:10.1007/s10441-021-09431-1 · Cited by 2
“(Gani and Jerwood 1971) recognised these binomial chain models as examples of discrete time Markov chains (Gagniuc 2017; Allen and Burgin 2000).”
BackgroundDeep Reinforcement Learning-Based Energy Management for Hybrid Electric Vehicles
Li, Y., & He, H. (2022) · Synthesis lectures on advances in automotive technology · doi:10.2200/s01156ed1v01y202112aat016 · Cited by 4
Bishwas, A. K., Mani, A., & Palade, V. (2022) · International Conference for Advancement in Technology (ICONAT) · doi:10.1109/iconat53423.2022.9725821 · Cited by 2
“The HMM is a statistical Markov model [20], which follows Markov properties.”
MethodIOSIG: Declarative I/O-Stream Properties Using Pragmas
Gholami, M., & Schintke, F. (2022) · Datenbank-Spektrum · doi:10.1007/s13222-022-00419-w · Cited by 2
“For a complex block device consisting of multiple block devices (e.g., disks) MTTDL is defined by constructing Markov models [17] according to the internal (graph)structure of the complex block device (e.g., Ceph network of storage nodes, Lustre, RAID-0, RAID-1, etc.) and using MTTF, BER, and MTTDL…”
MethodDifferentiable rotamer sampling with molecular force fields
Sha, C. M., Wang, J., & Dokholyan, N. V. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad456 · Cited by 3
“We demonstrate this bias through a simple Markov chain [32] model of training in the Supplemental Methods (A Simplified Demonstration of Biased Sampling due to a Discontinuous Mapping).”
Assessment of potential carbon storage in North Korea based on forest restoration strategies
Cho, W., Kim, I., & Ko, D. W. (2023) · Environmental Biology Research · doi:10.11626/kjeb.2023.41.3.204 · Cited by 2
Performance evaluation of SCADA systems by means of Discrete Event Systems
Amri, O., & Lefebvre, D. (2023) · IEEE International Conference on Networking, Sensing and Control (ICNSC) · doi:10.1109/icnsc58704.2023.10319003 · Cited by 2
“Basically, our setting for the proposed analysis is based on the standard Markovian assumptions [7], [11].”
MethodA Generalized Multi-Stage P-Persistent MAC Protocol for V2V Communications
Chen, X., Leng, S., Mao, S., et al. (2018) · IEEE International Conference on Communication Technology (ICCT) · doi:10.1109/icct.2018.8600221 · Cited by 2
“3, we can obtain the following equations [13]: ) ( ), ( 0 i i i P PP F T i m (2)”
BackgroundAddressing the Embeddability Problem in Transition Rate Estimation
Goolsby, C., Losey, J., Xu, Y., Düker, M.-C., Sherman, M. G., Matteson, D. S., & Moradi, M. (2019) · doi:10.1101/707919 · Cited by 6
Probabilistic Classification Methods
Nandi, A. K., & Ahmed, H. (2019) · doi:10.1002/9781119544678.ch11 · Cited by 2
Matrix-graph model of the polymer materials destruction process
Khvostov, A. A., Tikhomirov, S. G., Khaustov, I. A., et al. (2018) · Proceedings of the Voronezh State University of Engineering Technologies · doi:10.20914/2310-1202-2018-3-50-55 · Cited by 1
A Spiking Neural Network Learning Markov Chain
Kiselev, M. V., Ivanitskiy, A. Y., & Lavrentyev, A. (2023) · doi:10.1109/ijcnn54540.2023.10191484 · Cited by 1
“Namely, we treat world dynamics as Markov chain [2].”
Dynamic Modeling of Strip Rolling Process Using Probabilistic Neural Network
Deng, J., Sierla, S., Sun, J., et al. (2024) · IEEE International Conference on Advanced Motion Control (AMC) · doi:10.1109/amc58169.2024.10505655 · Cited by 1
“[19].”
Vasco, D., Thompson, K., Alhadad, S. S. J., & Juri, M. Z. (2024) · Frontiers in Education · doi:10.3389/feduc.2024.1401996 · Cited by 1
Scheduling of Multifunction Multistatic Sensor
Suja, J., Kulmon, P., & Benko, M. (2025) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2025.3572871 · Cited by 1
“The suitability of approximating the first passage time (FPT) by an exponential distribution, as MEIT is derived from FPT [17], is demonstrated using Monte Carlo simulation.”
Heterogeneous Real-Time & Secure Networks: TSN over Anything & TLS over Anything
Sikora, A., & Yakovyna, V. S. (2025) · International Conference on Computer, Information and Telecommunication Systems · doi:10.1109/cits65975.2025.11099373 · Cited by 1
“• stochastic models, i.e. using (Hidden) Markov models, where a Markov chain is a stochastic process defined by a set of states and, for each state, a probability distribution on the states [16].”
Energy Prediction for Hotel Loads in Mild-Hybrid Heavy-Duty Vehicle
Khuntia, S., Hanif, A., Ahmed, Q., et al. (2025) · SAE International journal of commercial vehicles · doi:10.4271/02-18-03-0020 · Cited by 1
Beyond Clicks: Eye-Tracking Insights into User Responses to Different Recommendation Types
Koutroumpas, G., Mazzini, M., Idesis, S., Masias, M., Jose, J. M., Abadal, S., & Arapakis, I. (2025) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3705328.3759304 · Cited by 3
“To capture the dynamics of fixation behaviour, we extended this approach by modelling fixation sequences as first-order Markov chains, which assume that the probability of transitioning to a new AOI depends only on the current AOI [9].”
Gagniuc, P. A. (2025) · Algorithms · doi:10.3390/a18110709 · Cited by 1
Self-citationImpact of CII Regulations on Vehicle Carriers
Georgiev, P., Garbatov, Y., & Yalamov, D. (2025) · Springer eBooks · doi:10.1007/978-3-032-01620-1_7 · Cited by 1
Naved, R. T., Antu, J. F., Mamun, M. A., Parvin, K., & Ziaei, S. (2025) · BMJ Global Health · doi:10.1136/bmjgh-2024-018458 · Cited by 1
Miranda, Í. M., Ladeira, M., & Aranha, C. D. C. (2018) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2018.8477814 · Cited by 1
“Q-learning is a simple way for agents to learn the optimal policy in controlled Markovian domains [15], i.”
BackgroundHigh-dimensional quantum state manipulation and tracking
Licina, M. (2020) · International Journal of Quantum Information · doi:10.1142/s0219749920500458 · Cited by 1
Decision Making for Autonomous Vehicle at Single-Lane Road Under Uncertainties
Wang, Y. (2021) · doi:10.1109/conf-spml54095.2021.00055 · Cited by 1
Detection and Delineation of Agricultural Land Losses in Minna, Niger State, Nigeria
Bashir, I. Y., Osesienemo, S. R., Aliyu, H., et al. (2022) · Remote Sensing of Land · doi:10.21523/gcj1.2022060103 · Cited by 1
Akın, D., & ÖCALIR, E. V. (2022) · Journal of Urban Planning and Development · doi:10.1061/(asce)up.1943-5444.0000871 · Cited by 1
Revathi, B. S., & Meena, K. A. (2022) · i-manager’s Journal on Image Processing · doi:10.26634/jip.9.3.19054 · Cited by 1
The lethality paradox: Goodhart’s Law and the challenge of measuring lethality
Biggs, A. T., Hamilton, J. A., Huffman, G., & Markwald, R. R. (2023) · The Journal of Defense Modeling and Simulation Applications Methodology Technology · doi:10.1177/15485129231165527 · Cited by 1
LSTM models for spatiotemporal extrapolation of population data
Geiß, C., Maier, J., So, E., & Zhu, Y. (2023) · Joint Urban Remote Sensing Event (JURSE) · doi:10.1109/jurse57346.2023.10144145 · Cited by 3
Prioritizing Test Cases with Markov Chains: A Preliminary Investigation
dos Santos, L. B. R., de Souza, É. F., Berkenbrock, G. R., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43240-8_14 · Cited by 1
Kurantin, N., & Osei‐Hwedie, B. Z. (2023) · Springer eBooks · doi:10.1007/978-981-99-4405-7_6 · Cited by 1
Sequential Decision Making for Elevator Control
Tartan, E. Ö., & Çiflikli, C. (2023) · Journal of Advances in Information Technology · doi:10.12720/jait.14.5.1124-1131 · Cited by 1
Challenges of incorporating wounded personnel into small arms combat simulations
Biggs, A. T., Hamilton, J. A., & Markwald, R. R. (2023) · The Journal of Defense Modeling and Simulation Applications Methodology Technology · doi:10.1177/15485129231203704 · Cited by 1
Biggs, A. T., Hamilton, J. A., & Markwald, R. R. (2023) · The Journal of Defense Modeling and Simulation Applications Methodology Technology · doi:10.1177/15485129231203705 · Cited by 1
Sampaio, P. R., Caradot, N., Guilbert, A.-S., et al. (2024) · Water Practice & Technology · doi:10.2166/wpt.2024.001 · Cited by 1
Video Colorization Based on a Diffusion Model Implementation
Stival, L., Torres, R. D. S., & Pedrini, H. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-66329-1_10 · Cited by 1
Ņikiforova, O., & Zabiniako, V. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-66329-1_43 · Cited by 1
Stanislavsky, A. A., & Weron, A. (2024) · Journal of Mathematics in Industry · doi:10.1186/s13362-024-00151-y · Cited by 1
“Note, Markov models have long history and long list of various applications (Gagniuc [5]).”
The EM-BDD Algorithm For Learning Hidden Markov Models
Gunnarsdóttir, E. Ó., & Ingólfsdóttir, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-75107-3_8 · Cited by 1
Zhang, G., & Yan, Y. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802715 · Cited by 1
“On the other hand, DDPM reverses a discrete diffusion process through a Markov chain [6].”
Exploiting and Extracting Workload Patterns for Efficient IoT Processing
Cheng, L., Ting, H.-Y., Wu, S., & Bozorgzadeh, E. (2025) · Mediterranean Conference on Embedded Computing (MECO) · doi:10.1109/meco66322.2025.11049216 · Cited by 1
Pleshkova, S., & Kostov, K. V. (2025) · International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST) · doi:10.1109/icest66328.2025.11098448 · Cited by 1
D2T2 Analysis of a Loss of Main Feed Water Accident
Tolo, S., & Andrews, J. (2025) · Systems · doi:10.3390/systems13100927 · Cited by 1
iExplore: Accelerating Exploratory Data Analysis by Predicting User Intention
Yang, Z., Gong, J., Liu, C., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-91458-9_9 · Cited by 1
Triple absorbing Markov chain model to study the flow of higher education students
Walde, G. S. (2019) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1176/4/042066 · Cited by 1
A Novel Q- Learning Assisted Dynamic Power Sharing for Dual Connectivity Scenario
Chaudhari, A., Kumar, N., & Rao, P. (2020) · IEEE Consumer Communications & Networking Conference (CCNC) · doi:10.1109/ccnc46108.2020.9045097 · Cited by 1
“In this technique, an agent tries to learn optimal action policy π by observing various states in Markovian domain [16].”
MethodUser Assistance for Serious Games Using Hidden Markov Model
Yadav, V. H., Streicher, A., & Prabhune, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57717-9_31 · Cited by 1
Votion, J., Feng, T., & Cao, Y. (2021) · AIAA SciTech Forum · doi:10.2514/6.2021-0267 · Cited by 1
Discrete Markov Model Application for Decision-Making in Stock Investments
Tyvodar, O., & Prystavka, P. (2021) · Lecture notes in networks and systems · doi:10.1007/978-981-16-2380-6_27 · Cited by 1
Akhavan-Safar, M., Teimourpour, B., & Nowzari-Dalini, A. (2022) · Current Bioinformatics · doi:10.2174/1574893617666220127094224 · Cited by 1
Bayesian Latent-Observational Space Analysis of a Large Eddy Simulated Ocean Turbulence Field
Scott, N. V., & Kukulka, T. (2021) · MTS/IEEE OCEANS Conference (OCEANS) · doi:10.23919/oceans44145.2021.9705836 · Cited by 1
Duong, D. V. A., & Yoon, S. (2022) · Applied Sciences · doi:10.3390/app12083898 · Cited by 1
Kounev, S., Lange, K.-D., & von Kistowski, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-41705-5_1 · Cited by 1
Hidden Markov Modelling for Biological Sequence
Kannan, K. S., & Jeniffer, S. D. (2022) · Algorithms for intelligent systems · doi:10.1007/978-981-19-2126-1_31 · Cited by 1
Castrignanò, A., Bardini, R., Savino, A., & Carlo, S. D. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.04.25.538212 · Cited by 1
“(1), 𝑑 𝜋 ( 𝑠 ) is the stationary distribution of Markov chain [60] for 𝜋 𝜃 , 𝑎 ∈ are actions, 𝑠 ∈ are states, 𝑉 𝜋 ( 𝑠 ) and 𝑄 𝜋 ( 𝑠,𝑎 ) are, respectively, the value of state 𝑠 and the value of a ( 𝑠,𝑎 ) pair, when following a policy 𝜋 [61].”
Metadata
- Authors
- Gagniuc, P. A.
- Corresponding author
- Gagniuc, Paul A
- ORCID
- Gagniuc, Paul A
- Venue
- Wiley eBooks
- Publisher
- Wiley
- Published
- 2017-06-22
- OpenAlex
- W4235646468
- Cited by
- 415
- References
- 0
- Citations by year
- 2026: 19 · 2025: 37 · 2024: 64 · 2023: 60 · 2022: 54 · 2021: 68 · 2020: 58 · 2019: 38 · 2018: 17
- FWCI
- 7.99
- Type
- book
- Language
- en
- Open access
- closed
- Locations
- Wiley eBooks (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Mathematics › Mathematical Physics › Stochastic processes and statistical mechanics
- Keywords
- Markov chains, stochastic processes, transition probabilities, Markov property
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
