The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Recent advances in selection hyper-heuristics
Open accessAbstract
Hyper-heuristics have emerged as a way to raise the level of generality of search techniques for computational search problems. This is in contrast to many approaches, which represent customised methods for a single problem domain or a narrow class of problem instances. The term hyper-heuristic was defined in the early 2000s as a heuristic to choose heuristics, but the idea of designing high-level heuristic methodologies can be traced back to the early 1960s. The current state-of-the-art in hyper-heuristic research comprises a set of methods that are broadly concerned with intelligently selecting or generating a suitable heuristic for a given situation. Hyper-heuristics can be considered as search methods that operate on lower-level heuristics or heuristic components, and can be categorised into two main classes: heuristic selection and heuristic generation. Here we will focus on the first of these two categories, selection hyper-heuristics. This paper gives a brief history of this emerging area, reviews contemporary selection hyper-heuristic literature, and discusses recent selection hyper-heuristic frameworks. In addition, the existing classification of selection hyper-heuristics is extended, in order to reflect the nature of the challenges faced in contemporary research. Unlike the survey on hyper-heuristics published in 2013, this paper focuses only on selection hyper-heuristics and presents critical discussion, current research trends and directions for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Metaheuristics in combinatorial optimization: Overview and conceptual comparison
Blum, C., & Roli, A. (2003) · ACM Computing Surveys · doi:10.1145/937503.937505 · Cited by 3,236
Apprenticeship learning via inverse reinforcement learning
Abbeel, P., & Ng, A. Y. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015430 · Cited by 2,769
Metaheuristics in Combinatorial Optimization
Gendreau, M., & Potvin, J.-Y. (2005) · Annals of Operations Research · doi:10.1007/s10479-005-3971-7 · Cited by 2,338
Parameter control in evolutionary algorithms
Eiben, A. E., Hinterding, R., & Michalewicz, Z. (1999) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.771166 · Cited by 1,980
The irace package: Iterated racing for automatic algorithm configuration
López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., & Stützle, T. (2016) · Operations Research Perspectives · doi:10.1016/j.orp.2016.09.002 · Cited by 1,761
Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019
Auger, A., & Stützle, T. (2019) · Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) · Cited by 1,749
(2010) · International Series in Operations Research & Management Science · doi:10.1007/978-1-4419-1665-5 · Cited by 1,642
Evolutionary Optimization in Uncertain Environments—A Survey
Jin, Y., & Branke, J. (2005) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.846356 · Cited by 1,572
A general heuristic for vehicle routing problems
Pisinger, D., & Ropke, S. (2007) · Computers & Operations Research · doi:10.1016/j.cor.2005.09.012 · Cited by 1,443
Hyper-heuristics: a survey of the state of the art
Burke, E. K., Gendreau, M., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., & Qu, R. (2013) · Journal of the Operational Research Society · doi:10.1057/jors.2013.71 · Cited by 1,263
Self-citationjMetal: A Java framework for multi-objective optimization
Durillo, J. J., & Nebro, A. J. (2011) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2011.05.014 · Cited by 1,134
Variable neighbourhood search: methods and applications
Hansen, P., Mladenović, N., & Moreno Pérez, J. A. (2010) · Annals of Operations Research · doi:10.1007/s10479-009-0657-6 · Cited by 860
Probabilistic Learning Combinations of Local Job-Shop Scheduling Rules
Fisher, H. F. (1963) · Medical Entomology and Zoology · Cited by 729
Hyper-Heuristics: An Emerging Direction in Modern Search Technology
Burke, E., Kendall, G., Newall, J., Hart, E., Ross, P., & Schulenburg, S. (2006) · International Series in Operations Research & Management Science · doi:10.1007/0-306-48056-5_16 · Cited by 665
Self-citationImproved evolutionary optimization from genetically adaptive multimethod search
Vrugt, J. A., & Robinson, B. A. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0610471104 · Cited by 614
A Classification of Hyper-heuristic Approaches
Burke, E. K., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., & Woodward, J. R. (2010) · International Series in Operations Research & Management Science · doi:10.1007/978-1-4419-1665-5_15 · Cited by 563
Self-citationClassification of adaptive memetic algorithms: a comparative study
Ong, Y.-S., Lim, M.-H., Zhu, N., & Wong, K.-W. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.856143 · Cited by 514
A graph-based hyper-heuristic for educational timetabling problems
Burke, E. K., McCollum, B., Meisels, A., Petrovic, S., & Qu, R. (2007) · European Journal of Operational Research · doi:10.1016/j.ejor.2005.08.012 · Cited by 507
Self-citationAutomated Design of Production Scheduling Heuristics: A Review
Branke, J., Nguyen, S., Pickardt, C. W., & Zhang, M. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2015.2429314 · Cited by 469
PISA — A Platform and Programming Language Independent Interface for Search Algorithms
Bleuler, S., Laumanns, M., Thiele, L., & Zitzler, E. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-36970-8_35 · Cited by 457
Li, K., Fialho, A., Kwong, S., & Zhang, Q. (2014) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2013.2239648 · Cited by 418
2017 IEEE Congress on Evolutionary Computation (CEC)
(2017) · doi:10.1109/cec38257.2017 · Cited by 330
Exploring Hyper-heuristic Methodologies with Genetic Programming
Burke, E. K., Hyde, M. R., Kendall, G., Ochoa, G., Ozcan, E., & Woodward, J. R. (2009) · Intelligent Systems Reference Library · doi:10.1007/978-3-642-01799-5_6 · Cited by 304
Self-citationAlgorithm Selection for Combinatorial Search Problems: A Survey
Kotthoff, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-50137-6_7 · Cited by 270
A comprehensive analysis of hyper-heuristics
Özcan, E., Bilgin, B., & Korkmaz, E. E. (2008) · Intelligent Data Analysis · doi:10.3233/ida-2008-12102 · Cited by 246
Self-citationProceedings of the Genetic and Evolutionary Computation Conference
(2017) · doi:10.1145/3071178 · Cited by 245
Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems
Larson, K. S., Winikoff, M., Das, S., & Durfee, E. H. (2017) · Cited by 195
A Hyper-Heuristic Scheduling Algorithm for Cloud
Tsai, C.-W., Huang, W.-C., Chiang, M.-H., Chiang, M.-C., & Yang, C.-S. (2014) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2014.2315797 · Cited by 192
Metaheuristic optimization frameworks: a survey and benchmarking
Parejo, J. A., Ruiz-Cortés, A., Lozano, S., & Fernandez, P. (2012) · Soft Computing · doi:10.1007/s00500-011-0754-8 · Cited by 190
Practice and Theory of Automated Timetabling III
Burke, E. K., & Erben, W. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44629-x · Cited by 175
Self-citationAlgorithm Selection for Combinatorial Search Problems: A Survey
Kotthoff, L. (2014) · AI Magazine · doi:10.1609/aimag.v35i3.2460 · Cited by 172
Automatic design of hyper-heuristic based on reinforcement learning
Choong, S. S., Wong, L., & Lim, C. P. (2018) · Information Sciences · doi:10.1016/j.ins.2018.01.005 · Cited by 142
Grammatical Evolution Hyper-Heuristic for Combinatorial Optimization Problems
Sabar, N. R., Ayob, M. B., Kendall, G., & Qu, R. (2013) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2013.2281527 · Cited by 129
Sabar, N. R., Ayob, M. B., Kendall, G., & Qu, R. (2014) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2014.2319051 · Cited by 125
A Tabu Search hyper-heuristic strategy for t-way test suite generation
Zamli, K. Z., Alkazemi, B. Y., & Kendall, G. (2016) · Applied Soft Computing · doi:10.1016/j.asoc.2016.03.021 · Cited by 121
Extreme Value Based Adaptive Operator Selection
Fialho, A. R. S., Costa, L. D., Schoenauer, M., & Sébag, M. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87700-4_18 · Cited by 113
A multi-objective hyper-heuristic based on choice function
Maashi, M. S., Özcan, E., & Kendall, G. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.12.050 · Cited by 111
Self-citationContrasting meta-learning and hyper-heuristic research: the role of evolutionary algorithms
Pappa, G. L., Ochoa, G., Hyde, M. R., Freitas, A. A., Woodward, J. R., & Swan, J. (2013) · Genetic Programming and Evolvable Machines · doi:10.1007/s10710-013-9186-9 · Cited by 109
A review of hyper-heuristics for educational timetabling
Pillay, N. (2014) · Annals of Operations Research · doi:10.1007/s10479-014-1688-1 · Cited by 98
Hyper-heuristic approach for multi-objective software module clustering
Kumari, A. C., & Srinivas, K. (2016) · Journal of Systems and Software · doi:10.1016/j.jss.2016.04.007 · Cited by 93
Automatically improving the anytime behaviour of optimisation algorithms
López‐Ibáñez, M., & Stützle, T. (2013) · European Journal of Operational Research · doi:10.1016/j.ejor.2013.10.043 · Cited by 91
Classification of Metaheuristics and Design of Experiments for the Analysis of Components
Birattari, M., Paquete, L., Stützle, T., & Varrentrapp, K. (2001) · Cited by 63
An Intelligent Hyper-Heuristic Framework for CHeSC 2011
Mısır, M., Verbeeck, K., Causmaecker, P. D., & den Berghe, G. V. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34413-8_45 · Cited by 60
An Improved Choice Function Heuristic Selection for Cross Domain Heuristic Search
Drake, J. H., Özcan, E., & Burke, E. K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-32964-7_31 · Cited by 59
Self-citationHyperSPAM: A study on hyper-heuristic coordination strategies in the continuous domain
Caraffini, F., Neri, F., & Epitropakis, M. G. (2018) · Information Sciences · doi:10.1016/j.ins.2018.10.033 · Cited by 51
Quantum-Inspired Hyper-Heuristics for Energy-Aware Scheduling on Heterogeneous Computing Systems
Chen, S., Li, Z., Yang, B., & Rudolph, G. (2015) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2015.2462835 · Cited by 50
A methodology for determining an effective subset of heuristics in selection hyper-heuristics
Soria-Alcaraz, J. A., Ochoa, G., Sotelo-Figueroa, M. A., & Burke, E. K. (2017) · European Journal of Operational Research · doi:10.1016/j.ejor.2017.01.042 · Cited by 47
Self-citationA tensor-based selection hyper-heuristic for cross-domain heuristic search
Asta, S., & Özcan, E. (2014) · Information Sciences · doi:10.1016/j.ins.2014.12.020 · Cited by 46
Self-citationA tensor based hyper-heuristic for nurse rostering
Asta, S., Özcan, E., & Curtois, T. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.01.031 · Cited by 46
Self-citationLearning heuristic selection using a Time Delay Neural Network for Open Vehicle Routing
Tyasnurita, R., Özcan, E., & John, R. I. (2017) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2017.7969477 · Cited by 43
Self-citationDeriving products for variability test of Feature Models with a hyper-heuristic approach
Strickler, A., Lima, J. P., Vergílio, S. R., & Pozo, A. (2016) · Applied Soft Computing · doi:10.1016/j.asoc.2016.07.059 · Cited by 41
A multi-objective and evolutionary hyper-heuristic applied to the Integration and Test Order Problem
Guizzo, G., Vergílio, S. R., Pozo, A., & Fritsche, G. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2017.03.012 · Cited by 40
Markov Chain methods for the Bipartite Boolean Quadratic Programming Problem
Karapetyan, D., Punnen, A. P., & Parkes, A. J. (2017) · European Journal of Operational Research · doi:10.1016/j.ejor.2017.01.001 · Cited by 38
A hyperheuristic approach based on low-level heuristics for the travelling thief problem
Yafrani, M. E., Martins, M. S. R., Wagner, M., Ahiod, B., Delgado, M., & Lüders, R. (2017) · Genetic Programming and Evolvable Machines · doi:10.1007/s10710-017-9308-x · Cited by 38
Selection hyper-heuristics in dynamic environments
Kıraz, B., Uyar, A. Ş., & Özcan, E. (2013) · Journal of the Operational Research Society · doi:10.1057/jors.2013.24 · Cited by 37
Self-citationYin, P.-Y., Lyu, S.-R., & Chuang, Y.-L. (2016) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2016.02.006 · Cited by 37
Zhao, J., Zhou, Y., & Wahab, M. I. M. (2016) · European Journal of Operational Research · doi:10.1016/j.ejor.2016.05.025 · Cited by 36
Competitive travelling salesmen problem: A hyper-heuristic approach
Kendall, G., & Li, J. (2012) · Journal of the Operational Research Society · doi:10.1057/jors.2012.37 · Cited by 32
A new hyper-heuristic as a general problem solver: an implementation in HyFlex
Mısır, M., Verbeeck, K., Causmaecker, P. D., & den Berghe, G. V. (2012) · Journal of Scheduling · doi:10.1007/s10951-012-0295-8 · Cited by 31
Emergency Railway Transportation Planning Using a Hyper-Heuristic Approach
Zheng, Y., Zhang, M., Ling, H.-F., & Chen, S. (2014) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2014.2331239 · Cited by 31
Evaluation of a Family of Reinforcement Learning Cross-Domain Optimization Heuristics
Gaspero, L. D., & Urli, T. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34413-8_32 · Cited by 29
Martins, M. S. R., Yafrani, M. E., Delgado, M., Wagner, M., Ahiod, B., & Lüders, R. (2017) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3071178.3071235 · Cited by 27
On the investigation of hyper-heuristics on a financial forecasting problem
Kampouridis, M., Alsheddy, A., & Tsang, E. P. K. (2012) · Annals of Mathematics and Artificial Intelligence · doi:10.1007/s10472-012-9283-0 · Cited by 26
Gascón-Moreno, J., Salcedo‐Sanz, S., Saavedra-Moreno, B., Carro‐Calvo, L., & Portilla-Figueras, A. (2013) · Information Sciences · doi:10.1016/j.ins.2013.06.017 · Cited by 24
A mixture experiments multi-objective hyper-heuristic
Vázquez-Rodríguez, J. A., & Petrović, S. (2012) · Journal of the Operational Research Society · doi:10.1057/jors.2012.125 · Cited by 23
A Multi-Armed Bandit selection strategy for Hyper-heuristics
Ferreira, A. S., Gonçalves, R. A., & Pozo, A. (2017) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2017.7969356 · Cited by 21
Move acceptance in local search metaheuristics for cross-domain search
Jackson, W. G., Özcan, E., & John, R. I. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.05.006 · Cited by 20
Self-citationEvolvability metrics in adaptive operator selection
Alcaraz, J. A. S., Ochoa, G., Carpio, M., et al. (2014) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2576768.2598220 · Cited by 19
Limits to Learning in Reinforcement Learning Hyper-heuristics
Alanazi, F., & Lehre, P. K. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-30698-8_12 · Cited by 19
Designing reusable metaheuristic methods: A semi-automated approach
Adriænsen, S., Brys, T., & Nowé, A. (2014) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2014.6900575 · Cited by 17
A Re-characterization of Hyper-Heuristics
Swan, J., Causmaecker, P. D., Martin, S. P., & Özcan, E. (2017) · Operations research, computer science. Interface series · doi:10.1007/978-3-319-58253-5_5 · Cited by 17
Self-citationOptimising Deep Belief Networks by hyper-heuristic approach
Sabar, N. R., Turky, A. M., Song, A., & Sattar, A. (2017) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2017.7969640 · Cited by 16
Walker, D. J., & Keedwell, E. C. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45823-6_46 · Cited by 14
Pillay, N. (2012) · South African Computer Journal · doi:10.18489/sacj.v48i1.87 · Cited by 13
A New Hyperheuristic Algorithm for Cross-Domain Search Problems
Lehrbaum, A., & Musliu, N. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34413-8_41 · Cited by 13
Offline Learning for Selection Hyper-heuristics with Elman Networks
Yates, W. B., & Keedwell, E. C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-78133-4_16 · Cited by 12
A Non-adaptive Stochastic Local Search Algorithm for the CHeSC 2011 Competition
Mascia, F., & Stützle, T. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34413-8_8 · Cited by 7
A study of agnostic hyper-heuristics based on sampling solution chains
Chuang, C.-Y., & Smith, S. F. (2017) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2017.7969323 · Cited by 4
Crossover Control in Selection Hyper-heuristics: Case Studies using MKP and HyFlex
Drake, J. H. (2014) · Nottingham ePrints (University of Nottingham) · Cited by 3
Self-citation
Cited by
Karimi-Mamaghan, M., Mohammadi, M., Meyer, P., Karimi-Mamaghan, A. M., & Talbi, E.-G. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.04.032 · Cited by 462
Türk, S., Deveci, M., Özcan, E., Canıtez, F., & John, R. (2021) · Information Sciences · doi:10.1016/j.ins.2020.08.076 · Cited by 98
Self-citation
Metadata
- Authors
- Drake, J. H., Kheiri, A., Özcan, E., & Burke, E. K.
- Venue
- European Journal of Operational Research
- Publisher
- Elsevier BV
- Published
- 2020-09
- Volume
- 285
- Issue
- 2
- Pages
- 405-428
- ISSN
- 0377-2217 (ISSN-L)
- OpenAlex
- W2967783393
- Cited by
- 365
- References
- 81
- FWCI
- 14.74
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.