AutoML Strategy Based on Grammatical Evolution: A Case Study about Knowledge Discovery from Text
Open accessAbstract
The process of extracting knowledge from natural language text poses a complex problem that requires both a combination of machine learning techniques and proper feature selection.Recent advances in Automatic Machine Learning (AutoML) provide effective tools to explore large sets of algorithms, hyperparameters and features to find out the most suitable combination of them.This paper proposes a novel AutoML strategy based on probabilistic grammatical evolution, which is evaluated on the health domain by facing the knowledge discovery challenge in Spanish text documents.Our approach achieves state-ofthe-art results and provides interesting insights into the best combination of parameters and algorithms to use when dealing with this challenge.Source code is provided for the research community.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AutoGOAL uses a search strategy based on probabilistic grammatical evolution to discover optimal machine learning pipelines.
“Furthermore, a search strategy based on probabilistic grammatical evolution is used to discover optimal machine learning pipelines (Estevez-Velarde et al., 2019) whose components can be from different back-end libraries.”
From Demo Application for the AutoGOAL Framework (2020) · cites this paper as Estevez-Velarde et al., 2019
Section 4 describes a modified probabilistic grammatical-evolution strategy for obtaining an optimal pipeline, and Section 4.2 explains how it searches for the best-performing pipeline. Our copy does not identify that strategy as AutoGOAL or establish that pipeline components can come from different back-end libraries. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 26
- Ken Barker, Bhalchandra Agashe, Shaw Yi Chaw, James Fan, Noah Friedland, Michael Glass, Jerry Hobbs, Eduard Hovy, David Israel, Doo Soon Kim, et al. 2007. Learning by reading: A prototype sys- tem, performance baseline and lessons learned. In AAAI, volume 7, pages 280-286.
- Boyuan Chen, Harvey Wu, Warren Mo, Ishanu Chat- topadhyay, and Hod Lipson. 2018. Autostacker: A compositional evolutionary learning system. In Pro- ceedings of the Genetic and Evolutionary Computa- tion Conference, GECCO '18, pages 402-409, New York, NY, USA. ACM. · DOI
- Philipp Cimiano, Alexander Mädche, Steffen Staab, and Johanna Völker. 2009. Ontology Learning, page 245-267. Springer Berlin Heidelberg, Berlin, Hei- delberg. · DOI
- David Crystal. 1997. The cambridge encyclopedia. Cambridge University Press New York.
- S Estevez-Velarde, Y Gutierrez, A Montoyo, A Piad- Morffis, R Munoz, and Y Almeida-Cruz. 2018. Gathering object interactions as semantic knowl- edge. In Proceedings on the International Con- ference on Artificial Intelligence (ICAI), pages 363-369. The Steering Committee of The World Congress in Computer Science, Computer . . . .
- Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hut- ter. 2015. Efficient and Robust Automated Machine Learning. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, page 2962-2970.
- Fausto Giunchiglia and Mattia Fumagalli. 2017. Tele- ologies: Objects, actions and functions. In Concep- tual Modeling, pages 520-534, Cham. Springer In- ternational Publishing.
- Graciela H Gonzalez, Tasnia Tahsin, Britton C Goodale, Anna C Greene, and Casey S Greene. 2015. Recent advances and emerging applications in text and data mining for biomedical discovery. Briefings in bioinformatics, 17(1):33-42. · DOI
- Haifeng Jin, Qingquan Song, and Xia Hu. 2018. Ef- ficient neural architecture search with network mor- phism. CoRR, abs/1806.10282. · arXiv
- Hyun-Tae Kim and Chang Wook Ahn. 2015. A New Grammatical Evolution Based on Probabilis- tic Context-free Grammar. In Proceedings of the 18th Asia Pacific Symposium on Intelligent and Evo- lutionary Systems -Volume 2, page 1-12, Cham. Springer International Publishing.
- Brent Komer, James Bergstra, and Chris Eliasmith. 2014. Hyperopt-sklearn: automatic hyperparameter configuration for scikit-learn. In ICML workshop on AutoML, pages 2825-2830. · DOI
- Pilar López-Ubeda, Manuel Carlos Díaz-Galiano, María Teresa Martín-Valdivia, and L. Alfonso Urena-Lopez. 2018. Sinai en tass 2018 task 3. clasi- ficando acciones y conceptos con umls en medline. In Proceedings of TASS 2018: Workshop on Se- mantic Analysis at SEPLN (TASS 2018) co-located with 34nd SEPLN Conference (SEPLN 2018), vol- ume 2172 of CEUR Workshop Proceedings, pages 83-88. CEUR-WS.org.
- Eugenio Martínez-Cámara, Yudivián Almeida-Cruz, Manuel Carlos Díaz-Galiano, Suilan Estévez- Velarde, Migueí A García-Cumbreras, Manuel García-Vega, Yoan Gutiérrez, Arturo Montejo-Ráez, Andrés Montoyo, Rafael Muñoz, Alejandro Piad- Morffis, and Julio Villena-Román. 2018. Overview of TASS 2018: Opinions, Health and Emotions Re- sumen de TASS 2018: Opiniones, Salud y Emo- ciones. CEUR Workshop Proceedings, 2172:13-27.
- Salvador Medina and Jordi Turmo. 2018. Joint clas- sification of key-phrases and relations in electronic health documents. In Proceedings of TASS 2018: Workshop on Semantic Analysis at SEPLN (TASS 2018) co-located with 34nd SEPLN Conference (SE- PLN 2018), volume 2172 of CEUR Workshop Pro- ceedings, pages 83-88. CEUR-WS.org.
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed Representa- tions of Words and Phrases and their Composition- ality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Ad- vances in Neural Information Processing Systems 26, page 3111-3119.
- Curran Associates, Inc. T. Mitchell, W. Cohen, E. Hruschka, P. Talukdar, B. Yang, J. Betteridge, A. Carlson, B. Dalvi, M. Gardner, B. Kisiel, J. Krishnamurthy, N. Lao, K. Mazaitis, T. Mohamed, N. Nakashole, E. Pla- tanios, A. Ritter, M. Samadi, B. Settles, R. Wang, D. Wijaya, A. Gupta, X. Chen, A. Saparov, M. Greaves, and J. Welling. 2018. Never-ending learning. Commun. ACM, 61(5):103-115. · DOI
- Randal S. Olson and Jason H. Moore. 2016. Tpot: A tree-based pipeline optimization tool for automating machine learning. In Proceedings of the Workshop on Automatic Machine Learning, volume 64 of Pro- ceedings of Machine Learning Research, pages 66- 74, New York, New York, USA.
- PMLR.
- M. O'Neill and C. Ryan. 2001. Grammatical evolu- tion. IEEE Transactions on Evolutionary Computa- tion, 5(4):349-358. · DOI · Paper page
- Jorge Vivaldi Palatresi and Horacio Rodríguez Honto- ria. 2018. Medical knowledge discovery by combin- ing multiple techniques and resources. In Proceed- ings of TASS 2018: Workshop on Semantic Anal- ysis at SEPLN (TASS 2018) co-located with 34nd SEPLN Conference (SEPLN 2018), volume 2172 of CEUR Workshop Proceedings, pages 83-88. CEUR- WS.org.
- Alejandro Piad-Morffis, Yoan Gutiérrez, and Rafael Muñoz. 2019a. A corpus to support ehealth knowl- edge discovery technologies. Journal of biomedical informatics, 94:103172.
- Alejandro Piad-Morffis, Yoan Gutiérrez, Suilan Estévez-Velarde, Yudivián Almeida-Cruz, Andrés Montoyo, and Rafael Muñoz. 2019b. Analysis of ehealth knowledge discovery systems in the tass 2018 workshop. Procesamiento del Lenguaje Natural, 62(1).
- Alex G. C. de Sá, Walter José G. S. Pinto, Luiz Otavio V. B. Oliveira, and Gisele L. Pappa. 2017. RECIPE: A Grammar-Based Framework for Automatically Evolving Classification Pipelines. In Genetic Pro- gramming, page 246-261, Cham. Springer Interna- tional Publishing.
- Chris Thornton, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2013. Auto-weka: Combined selection and hyperparameter optimization of clas- sification algorithms. In Proceedings of the 19th ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, KDD '13, pages 847-855, New York, NY, USA. ACM. · DOI · Paper page
- Renzo M. Rivera Zavala, Paloma Martínez, and Isabel Segura-Bedmar. 2018. A hybrid bi-lstm-crf model for knowledge recognition from ehealth documents. In Proceedings of TASS 2018: Workshop on Se- mantic Analysis at SEPLN (TASS 2018) co-located with 34nd SEPLN Conference (SEPLN 2018), vol- ume 2172 of CEUR Workshop Proceedings, pages 83-88. CEUR-WS.org.
- Barret Zoph and Quoc V. Le. 2016. Neural archi- tecture search with reinforcement learning. CoRR, abs/1611.01578. · arXiv
Cited by
Sorted by citations
Demo Application for the AutoGOAL Framework
Estevez-Velarde, S., Piad-Morffis, A., Gutiérrez, Y., Montoyo, A., Muñoz-Guillena, R., & Almeida Cruz, Y. (2020) · Proceedings of the 28th International Conference on Computational Linguistics: System Demonstrations · doi:10.18653/v1/2020.coling-demos.4
“Furthermore, a search strategy based on probabilistic grammatical evolution is used to discover optimal machine learning pipelines (Estevez-Velarde et al., 2019) whose components can be from different back-end libraries.”
MethodSelf-citation
- Authors
- Estevez-Velarde, S., Gutiérrez, Y., Montoyo, A., & Almeida-Cruz, Y.
- Affiliations
- University of Alicante (ES)
- University of Havana (CU)
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 4356-4365
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2951493960
- Cited by
- 15
- References
- 0
- Citations by year
- 2024: 2 · 2023: 1 · 2022: 3 · 2021: 5 · 2020: 3 · 2019: 1
- FWCI
- 1.15
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- Universidad de Alicante (RTI2018-094653-B-C22, RTI2018-094649-B-I00)
- Generalitat Valenciana (PROMETEU/2018/089)
- Universidad de La Habana
- Fundación Carolina
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Keywords
- automated machine learning, feature selection, hyperparameter optimization, health domain, algorithm selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
