Cited by 101Metaheuristic Optimization Algorithms ResearchStochastic Gradient Optimization TechniquesArtificial Intelligence Applications

A review on the self and dual interactions between machine learning and optimisation

Open access
Song, Heda, Triguero, Isaac, Özcan, Ender·2019·Progress in Artificial Intelligence

Abstract

Machine learning and optimisation are two growing fields of artificial intelligence with an enormous number of computer science applications. The techniques in the former area aim to learn knowledge from data or experience, while the techniques from the latter search for the best option or solution to a given problem. To employ these techniques automatically and effectively aligning with the real aim of artificial intelligence, both sets of techniques are frequently hybridised, interacting with each other and themselves. This study focuses on such interactions aiming at (1) presenting a broad overview of the studies on self and dual interactions between machine learning and optimisation; (2) providing a useful tutorial for researchers and practitioners in both fields in support of collaborative work through investigation of the recent advances and analyses of the advantages and disadvantages of different techniques to tackle the same or similar problems; (3) clarifying the overlapping terminologies having different meanings used in both fields; (4) identifying research gaps and potential research directions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s13748-019-00185-z

Cites

197 of 197 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Some papers related to the Software Design KA were not related to the Software Construction KA.

    “On the contrary, some papers related to the Software Design KA were not related to the Software Construction KA [12,13,19,20].”
  • Mission critical systems in some domains strongly require safety and security, and conforming ML applications to industry standards such as DO-178, ISO26262 and ASIL, and Common Criteria is a difficult but important challenge to realize mission critical ML applications in industry.

    “-Safety and Security: Mission critical systems in some domains are strongly required safety and security.There are industry standards which address safety or security such as DO-178 [128], ISO26262 and ASIL [129] and Common Criteria [130].How to conform ML applications to these standards is difficult but important challenges to realize mission critical ML applications in industry [20,30,35,61,82,”
  • The ethics of ML applications relate to issues on safety, privacy and discrimination, and ethical topics should be included in quality evaluations of ML applications.

    “-Ethics and Regulations: The ethics of ML applications relate to issues on safety, privacy and discrimination [20,24,49,59,103,106].Ethical topics should be included in quality evaluations of ML applications.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

6 of the 203 references couldn’t be matched to a known paper.

Research works printed but not linked: 20

  1. Andréasson, N., Evgrafov, A., Patriksson, M., Gustavsson, E., Önnheim, M.: An introduction to continuous optimization: foun- dations and fundamental algorithms, vol. 28. Studentlitteratur Lund (2005)
  2. Deb, K., Hussein, R., Roy, P.C., Toscano, G.: A taxonomy for metamodeling frameworks for evolutionary multi-objective opti- mization. IEEE Trans. Evol. Comput. 23, 104-116 (2018)
  3. Dioşan, L., Oltean, M.: Evolutionary design of evolutionary algo- rithms. Genet. Program. Evol. Mach. 10(3), 263-306 (2009)
  4. Gama, J., Brazdil, P.: Characterization of classification algo- rithms. In: Portuguese Conference on Artificial Intelligence, pp. 189-200. Springer, Berlin (1995)
  5. Ha, D., Dai, A., Le, Q.V.: Hypernetworks. In: International Con- ference on the 4th Learning Representations (ICLR 2016). San Juan, Puerto Rico (2016)
  6. Hunger, J., Huttner, G.: Optimization and analysis of force field parameters by combination of genetic algorithms and neural net- works. J. Comput. Chem. 20(4), 455-471 (1999)
  7. Hutter, F., Kotthoff, L., Vanschoren, J.: Automatic Machine Learning: Methods, Systems. Challenges. Springer, Berlin (2019)
  8. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Pro- ceedings of ICNN'95-International Conference on Neural Net- works, vol. 4, pp. 1942-1948. Perth, Australia (1995)
  9. Li, K., Malik, J.: Learning to optimize. In: Proceedings of the 5th International Conference on Learning Representations (ICLR 2017). Toulon, France (2017) 103.
  10. Lorraine, J., Duvenaud, D.: Hyperparameter optimization with hypernets. In: 31st Conference on Neural Information Processing Systems (NIPS 2017), Workshop on Meta-learning. Long Beach, USA (2017) 106.
  11. Lukasiewycz, M., Glaß, M., Reimann, F., Teich, J.: Opt4j: a mod- ular framework for meta-heuristic optimization. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Com- putation, pp. 1723-1730. Dublin, Ireland (2011) 108.
  12. Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997) 114.
  13. Ren, Y., Zhang, L., Suganthan, P.N.: Ensemble classification and regression-recent developments, applications and future direc- tions. IEEE Comput. Intell. Mag. 11(1), 41-53 (2016) 141.
  14. Rosenbaum, C., Klinger, T., Riemer, M.: Routing networks: adap- tive selection of non-linear functions for multi-task learning. In: International Conference on the 6th Learning Representations (ICLR 2018). Vancouver, Canada (2018) 145.
  15. Schlag, I., Schmidhuber, J.: Gated fast weights for on-the- fly neural program generation. In: 31st Conference on Neural Information Processing Systems (NIPS 2017), Workshop on Meta-learning. Long Beach, USA (2017) 150.
  16. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: relation network for few-shot learn- ing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA (2018) 167.
  17. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduc- tion. MIT Press, Cambridge (1998) 168.
  18. Vercellino, C.J., Wang, W.Y.: Hyperactivations for activation function exploration. In: 31st Conference on Neural Information Processing Systems (NIPS 2017), Workshop on Meta-learning. Long Beach, USA (2017) 177.
  19. Wichrowska, O., Maheswaranathan, N., Hoffman, M.W., Col- menarejo, S.G., Denil, M., de Freitas, N., Sohl-Dickstein, J.: Learned optimizers that scale and generalize. In: Proceedings of the 34th International Conference on Machine Learning (ICML 2017). Sydney, Australia (2017) 182.
  20. Xiong, Z., Zhang, W., Zhu, W.: Learning decision trees with rein- forcement learning. In: 31st Conference on Neural Information Processing Systems (NIPS 2017), Workshop on Meta-learning. Long Beach, USA (2017) 189.

Cited by

Sorted by citations

Authors
Song, H., Triguero, I., & Özcan, E.
Affiliations
University of Nottingham (GB)
Corresponding author
Song, Heda
Venue
Progress in Artificial Intelligence
Published
2019-04-25
Volume
8
Issue
2
Pages
143-165
ISSN
  • 2192-6352 (ISSN-L)
  • 2192-6360
OpenAlex
W2941110559
Cited by
101
References
203
Citations by year
2026: 5 · 2025: 16 · 2024: 18 · 2023: 17 · 2022: 13 · 2021: 16 · 2020: 15 · 2019: 1
FWCI
5.19
Type
journal article
Language
en
Open access
Locations
  • Progress in Artificial Intelligence (publishedVersion, cc-by)
  • Repository@Nottingham (University of Nottingham) (publishedVersion, other-oa)
  • Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
Funders
University of Nottingham
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Metaheuristic Optimization Algorithms Research
  • Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
machine learning, optimization, self-interaction, research gaps

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review