The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Sim-to-real transfer reinforcement learning for control of thermal effects of an atmospheric pressure plasma jet
Open accessAbstract
Applications of atmospheric pressure plasma jets (APPJs) present challenging feedback control problems due to the complexity of the plasma-substrate interactions. The plasma treatment of complex substrates is particularly sensitive to changes in the physical, chemical, and electrical properties of the substrate, which may vary considerably within and between target substrates. The increasingly popular reinforcement learning (RL) methods hold promise for learning-based control of APPJ applications that involve treatment of complex substrates with time-varying or non-uniform characteristics. This paper demonstrates the use of a deep RL method for regulation of thermal properties of APPJs on substrates with different thermal and electrical characteristics. Using simulated data from an experimentally-validated, physics-based model of the thermal dynamics of the plasma-substrate interactions, an RL agent is trained to perform temperature setpoint tracking. It is shown that training the RL agent using a wide range of simulated thermal dynamics of the plasma-substrate interactions allows for capturing the diverse temperature responses of different substrates. Experimental demonstrations on a kHz-excited APPJ in He show that the proposed RL agent enables effective temperature control over a wide variety of substrates with drastically different thermal and electrical properties.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In cases where system identification for one-shot transfer is the objective, researchers have often built or customized specific simulations that meet problem-specific requirements and constraints.
“In those cases where system identification for one-shot transfer is the objective, researchers have often built or customized specific simulations that meet problem-specific requirements and constraints [32], [36], [41].”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [32]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Siow, K. S., Britcher, L., Kumar, S., & Griesser, H. J. (2006) · Plasma Processes and Polymers · doi:10.1002/ppap.200600021 · Cited by 971
Laroussi, M., & Leipold, F. (2004) · International Journal of Mass Spectrometry · doi:10.1016/j.ijms.2003.11.016 · Cited by 863
Kalghatgi, S. U., Fridman, G., Cooper, M., et al. (2007) · IEEE Transactions on Plasma Science · doi:10.1109/tps.2007.905953 · Cited by 310
Neural network vehicle models for high-performance automated driving
Spielberg, N. A., Brown, M., Kapania, N. R., Kegelman, J. C., & Gerdes, J. C. (2019) · Science Robotics · doi:10.1126/scirobotics.aaw1975 · Cited by 263
Clinical Plasma Medicine: State and Perspectives of in Vivo Application of Cold Atmospheric Plasma
von Woedtke, T., Metelmann, H. R., & Weltmann, K. D. (2014) · Contributions to Plasma Physics · doi:10.1002/ctpp.201310068 · Cited by 249
Helium atmospheric pressure plasma jets touching dielectric and metal surfaces
Norberg, S., Johnsen, E., & Kushner, M. J. (2015) · Journal of Applied Physics · doi:10.1063/1.4923345 · Cited by 228
(2012) · Cambridge University Press eBooks · doi:10.1017/cbo9780511902598 · Cited by 179
Machine learning for modeling, diagnostics, and control of non-equilibrium plasmas
Mesbah, A., & Graves, D. B. (2019) · Journal of Physics D Applied Physics · doi:10.1088/1361-6463/ab1f3f · Cited by 132
Self-citationComputational study of the interaction of cold atmospheric helium plasma jets with surfaces
Breden, D. P., & Raja, L. L. (2014) · Plasma Sources Science and Technology · doi:10.1088/0963-0252/23/6/065020 · Cited by 124
Process control via artificial neural networks and reinforcement learning
Hoskins, J. C., & Himmelblau, D. M. (1992) · Computers & Chemical Engineering · doi:10.1016/0098-1354(92)80045-b · Cited by 121
Comprehensive Clinical Plasma Medicine
(2018) · Springer eBooks · doi:10.1007/978-3-319-67627-2 · Cited by 116
Numerical Model of an Argon Atmospheric Pressure RF Discharge
Balcon, N., Hagelaar, G. J. M., & Bœuf, J.-P. (2008) · IEEE Transactions on Plasma Science · doi:10.1109/tps.2008.2003135 · Cited by 109
Machine Learning for Real-Time Diagnostics of Cold Atmospheric Plasma Sources
Gidon, D., Pei, X., Bonzanini, A. D., et al. (2019) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2019.2910220 · Cited by 78
Self-citationAnderson, C. W., Hittle, D. C., Katz, A. D., & Kretchmar, R. M. (1997) · Artificial Intelligence in Engineering · doi:10.1016/s0954-1810(97)00004-6 · Cited by 65
Gidon, D., Graves, D. B., & Mesbah, A. (2017) · Plasma Sources Science and Technology · doi:10.1088/1361-6595/aa7c5d · Cited by 65
Self-citationTreating cancer with cold physical plasma: On the way to evidence‐based medicine
Metelmann, H., Seebauer, C., Rutkowski, R., et al. (2018) · Contributions to Plasma Physics · doi:10.1002/ctpp.201700085 · Cited by 56
Model-Based Feedback Control of a kHz-Excited Atmospheric Pressure Plasma Jet
Gidon, D., Curtis, B., Paulson, J. A., et al. (2017) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2017.2764629 · Cited by 45
Self-citationKeidar, M., Yan, D., & Sherman, J. H. (2019) · Morgan & Claypool Publishers eBooks · doi:10.1088/2053-2571/aafb9c · Cited by 44
Study on a Room-Temperature Air Plasma for Biomedical Application
Wu, S., Wand, Z., Huang, Q., et al. (2011) · IEEE Transactions on Plasma Science · doi:10.1109/tps.2011.2132152 · Cited by 34
Shin, J., & Raja, L. L. (2007) · Journal of Physics D Applied Physics · doi:10.1088/0022-3727/40/10/018 · Cited by 31
The effect of target materials on the propagation of atmospheric-pressure plasma jets
Ji, L., Yan, W., Xia, Y., et al. (2018) · Journal of Applied Physics · doi:10.1063/1.5024806 · Cited by 28
Spether, D., Scharpf, M. O., Hennenlotter, J., et al. (2015) · Biomedical Optics Express · doi:10.1364/boe.6.001419 · Cited by 14
Model-Free Learning Control of Chemical Processes
Syafiie, S., Tadeo, F., & Martínez, E. (2008) · doi:10.5772/5287 · Cited by 10
Reinforcement learning for process identification, control and optimisation
Govindhasamy, J. J., McLoone, S., & Irwin, G. W. (2004) · doi:10.1109/is.2004.1344702 · Cited by 5
Editorial: Atmospheric Pressure Plasma Polymerization
Morent, R. (2013) · The Open Plasma Physics Journal · doi:10.2174/1876534301306010006 · Cited by 5
Flagged first, then by citations
Research works printed but not linked: 8
- Winter, 2015, Atmospheric pressure plasma jets: an overview of devices and new directions, Plasma Sources Sci. Technol., 24, 10.1088/0963-0252/24/6/064001 · DOI
- Weltmann, 2016, Plasma medicine—current state of research and medical application, Plasma Phys. Control. Fusion, 59, 10.1088/0741-3335/59/1/014031 · DOI
- Klarenaar, 2018, How dielectric, metallic and liquid targets influence the evolution of electron properties in a pulsed He jet measured by Thomson and Raman scattering, Plasma Sources Sci. Technol., 27, 10.1088/1361-6595/aad4d7 · DOI
- Gidon, 2019, Spatial thermal dose delivery in atmospheric pressure plasma jets, Plasma Sources Sci. Technol., 28, 10.1088/1361-6595/aafff6 · DOI
- Lyu, 2019, Mathematical modeling and control for cancer treatment with cold atmospheric plasma jet, J. Phys. D: Appl. Phys., 52, 10.1088/1361-6463/ab061d · DOI
- Keidar, 2019, Adaptive plasmas and recent progress in plasma application in cancer therapy, 7-1, 10.1088/978-1-64327-434-8ch7 · DOI
- Konda, 2000, Actor-critic algorithms, Adv. Neural Inf. Process. Syst., 1008
- Gidon, 2019, Predictive control of 2D spatial thermal dose delivery in atmospheric pressure plasma jets, Plasma Sources Sci. Technol., 28, 10.1088/1361-6595/ab2c66 · DOI
Cited by
Sorted by citations
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
Zhao, W., Queralta, J. P., & Westerlund, T. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308468 · Cited by 941
“In those cases where system identification for one-shot transfer is the objective, researchers have often built or customized specific simulations that meet problem-specific requirements and constraints [32], [36], [41].”
Background
Metadata
- Authors
- Witman, M., Gidon, D., Graves, D. B., Smit, B., & Mesbah, A.
- Affiliations
- École Polytechnique Fédérale de Lausanne (CH)
- University of California, Berkeley (US)
- Corresponding author
- Mesbah, Ali
- Venue
- Plasma Sources Science and Technology
- Publisher
- IOP Publishing
- Published
- 2019-09-01
- Volume
- 28
- Issue
- 9
- Pages
- 095019
- ISSN
- 0963-0252 (ISSN-L)
- 1361-6595
- OpenAlex
- W2969408081
- Cited by
- 46
- References
- 27
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 4 · 2023: 8 · 2022: 5 · 2021: 8 · 2020: 6 · 2019: 1
- FWCI
- 3.33
- Type
- journal article
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Licence
- Locations
- Plasma Sources Science and Technology (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Physics and Astronomy › Atomic and Molecular Physics, and Optics › Atmospheric-Pressure Plasma Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Aerospace Engineering › Plasma and Flow Control in Aerodynamics
- Keywords
- atmospheric pressure plasma jet, sim-to-real transfer, reinforcement learning, thermal dynamics, helium plasma
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
