Application of predictive modeling tools for the identification of Ocimum spp. herbal products

Linear models such as PLS tend to overfit data with more variables than samples.

“linear models like PLS tend to overfit data containing more variables than samples (as is the case with untargeted metabolomics) [5, 11, 12]”

• cites [5] Abraham EJ, Kellogg JJ. Chemometric-guided approaches for profiling and authenticating botanical materials. Front Nutr. 2021;8:780228. sourceSupported
• cites [11] Gad HA, El-Ahmady SH, Abou-Shoer MI, Al-Azizi MM. Application of chemometrics in authentication of herbal medicines: a review. Phytochem Anal. 2013;24(1):1–24. https://doi.org/10.1002/pca.2378. sourceNot checked — no full text of the source
• cites [12] Babyak MA. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med. 2004;66:411–21.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It remains impractical to produce many standards for every commercially viable herb.

“it is still infeasible to generate numerous standards for every commercially viable herb [6]”

• cites [6] Hosbas Coskun S, Wise SA, Kuszak AJ. The importance of reference materials and method validation for advancing research on the health effects of dietary supplements and other natural products. Front Nutr. 2021;8:786261. https://doi.org/10.3389/fnut.2021.786261. sourcePartly supported

  The ‘Current Availability of Reference Materials for Dietary Supplements’ section says that botanical reference standards were limited in the early 2000s and describes standards available for selected ingredients. It does not say that producing numerous standards for every commercially viable herb remains infeasible. (LLM, conf=0.75)

  Experimental — accuracy not yet measured

Untargeted metabolite profiles combined with multivariate statistics have successfully identified botanical adulteration.

“these methods, combined with multivariate statistical approaches, have been successful in identifying botanical adulteration [5]”

• cites [5] Abraham EJ, Kellogg JJ. Chemometric-guided approaches for profiling and authenticating botanical materials. Front Nutr. 2021;8:780228. sourceSupported

A single reference standard would not capture all possible chemical variation.

“a single reference standard would not incorporate all potential chemical variations [2]”

• cites [2] Spinardi A, Cola G, Gardana CS, Mignani I. Variation of anthocyanin content and profile throughout fruit development and ripening of highbush blueberry cultivars grown at two different altitudes. Front Plant Sci. 2019;10:1045. https://doi.org/10.3389/fpls.2019.01045. sourcePartly supported

  The Anthocyanin Profile section (page unavailable) reports that the proportions of different blueberry anthocyanins change with ripening stage and altitude, supporting the premise of chemical variation. The paper does not say that a single reference standard would fail to capture that variation; its analytical-methods sections describe standards and measurements without making that methodological claim. (LLM, conf=0.80)

  Experimental — accuracy not yet measured

Non-targeted metabolite profiles give a broader view of plants' chemical similarities and relationships.

“These non-targeted metabolite profiles provide a more extensive look into plants’ chemical similarities and relationships [4]”

• cites [4] Perez De Souza L, Alseekh S, Naake T, Fernie A. Mass spectrometry-based untargeted plant metabolomics. CP Plant Biol. 2019;4(4):e20100. https://doi.org/10.1002/cppb.20100. sourceSupported

Introducing external conditions such as temperature, moisture, drying, and storage causes large variation in Ocimum chemical profiles.

“When introducing external conditions, like temperature, moisture, drying conditions, and storage, Ocimum chemical profiles experience large variations [39, 40].”

• cites [39] Ekren S, Sönmez Ç, Özçakal E, et al. The effect of different irrigation water levels on yield and quality characteristics of purple basil (Ocimum basilicum L.). Agric Water Manag. 2012;109:155–61. https://doi.org/10.1016/j.agwat.2012.03.004. sourceNot checked — no full text of the source
• cites [40] Abozeed A, El Shafey R, Osman Y. Effect of location and environmental conditions on growth, yields and chemical constituents of sweet basil (Ocimum basilicum L.). J Agric Chem Biotechnol. 2015;6(1):1–13. https://doi.org/10.21608/jacb.2015.43961. sourcePartly supported

  The ‘4-Meteorological data’ section (Table 10; page unavailable) shows substantial differences in sweet basil essential-oil composition between two growing locations. The paper does not establish that drying conditions or storage cause those differences; it uses air-dried herb for analysis but does not compare drying or storage conditions. Temperature and moisture are discussed, but their separate effects on chemical profiles are not demonstrated. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

A botanical's chemical profile is strongly affected by environmental, harvesting, and processing conditions.

“a botanical’s chemical profile is greatly impacted by environmental, harvesting, and processing conditions.”

• cites [39] Ekren S, Sönmez Ç, Özçakal E, et al. The effect of different irrigation water levels on yield and quality characteristics of purple basil (Ocimum basilicum L.). Agric Water Manag. 2012;109:155–61. https://doi.org/10.1016/j.agwat.2012.03.004. sourceNot checked — no full text of the source
• cites [40] Abozeed A, El Shafey R, Osman Y. Effect of location and environmental conditions on growth, yields and chemical constituents of sweet basil (Ocimum basilicum L.). J Agric Chem Biotechnol. 2015;6(1):1–13. https://doi.org/10.21608/jacb.2015.43961. sourcePartly supported

  The ‘4-Meteorological data’ section (page not provided) reports substantial differences in sweet basil’s essential-oil composition between two locations, supporting an environmental effect on its chemical profile. The Materials and Methods section describes drying and extraction procedures, but does not show that harvesting or processing conditions affect the profile. The evidence is also specific to sweet basil rather than botanicals generally. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

The relationship between genetics, metabolomics, and environmental interactions in herbal products has not been thoroughly investigated.

“the relationship between genetics, metabolomics, and environmental interactions in herbal products has not been thoroughly investigated [43].”

• cites [43] Napier JD, Heckman RW, Juenger TE. Gene-by-environment interactions in plants: molecular mechanisms, environmental drivers, and adaptive plasticity. Plant Cell. 2023;35(1):109–24. https://doi.org/10.1093/plcell/koac322. sourcePartly supported

  The Introduction says that genetic drivers of plant gene-by-environment interactions need further investigation, including molecular responses under realistic field conditions. It does not establish that the relationship involving metabolomics in herbal products has been insufficiently investigated; its examples concern plant gene expression and transcriptomic studies. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

Consumer and validation set samples were located close together on the PCA scores plot.

“Similarly, consumer and validation set samples were positioned close together on the PCA scores plot (Fig. 3a).”

The greenhouse-grown materials could not be translated to predict the classification of external materials.

“the greenhouse-grown materials were not translatable to predict the classification of external materials”

An unexpected PCA overlap showed that five O. basilicum materials were more chemically similar to O. gratissimum materials.

“there was an unexpected overlap seen in the PCA scores plots (Fig. 2). Figure 2a shows that 5 O. basilicum materials are more chemically similar to the O. gratissimum materials,”

Among greenhouse samples, PerMANOVA detected significant differences between all three species as labeled on the seed packets.

“Among the greenhouse samples, there was a significant difference between all three labeled species (Table 3).”

Four distinct chemotypes were found across three Ocimum species.

“ultimately identifying 4 distinct chemotypes across three species of Ocimum”

The species O. tenuiflorum, O. gratissimum, and O. basilicum showed chemical separation in untargeted UPLC-MS/MS profiles under controlled growth.

“We found that three species, O. tenuiflorum, O. gratissimum, and O. basilicum, were chemically distinct based on their untargeted UPLC-MS/MS profiles when grown in controlled settings”

The three consumer products separated from the remaining samples in the upper right quadrant were tea blends B14 and B17 and a culinary product B13.

“The three consumer products that were separated from the remaining samples (in the upper right quadrant) were the two tea blends—B14 and B17—and a product marketed for culinary use—B13.”

PerMANOVA using morphological identification also found all three species to be significantly different from one another.

“PerMANOVA based on the morphological identification show similar results; all three species are significantly different from each other (Table 3).”

• AssumptionRests on an assumption: Seed morphology correctly identifies Ocimum species, so the morphologically identified species labels are accurate.

No species-related trends were observed in biomass, leaf dry weight, or growth patterns.

“We observed no species-based trends in biomass, leaf dry weight, or growth patterns (data not shown).”

Every mislabeled seed sample was O. basilicum that had been labeled as a holy basil species.

“All mislabeled seeds were O. basilicum characterized as a holy basil species,”

The consumer product and external validation profiles did not differ significantly (p = 0.53330).

“However, the consumer and external validation profiles did not significantly differ (p = 0.53330).”

HPTLC analysis confirmed that these materials belong to a unique cultivar distinct from O. tenuiflorum and O. gratissimum.

“Upon further investigation via HPTLC, we confirmed that these materials belong to a unique cultivar separate from O. tenuiflorum and O. gratissimum (Fig. 2e).”

HPTLC fingerprints showed that consumer product fingerprints were much more complex than those of greenhouse-grown materials.

“These results are supported with HPTLC fingerprints, where the consumer product fingerprints are much more complicated than the greenhouse-grown materials (Fig. 3b).”

The comparison of O. tenuiflorum with O. gratissimum yielded a p-value of 0.0014.

“O. tenuiflorum vs O. gratissimum resulted in a p-value of 0.0014,”

The controlled greenhouse materials differed significantly from the external samples.

“the controlled materials were significantly different from external samples”

Seed-morphology-based identification assigned the dataset to 5 O. gratissimum, 7 O. tenuiflorum, and 16 O. basilicum materials.

“based on morphological investigations using seed characteristics, our dataset contained 5 O. gratissimum, 7 O. teniflorum, and 16 O. basilicum materials.”

• AssumptionRests on an assumption: Seed morphology correctly identifies Ocimum species, so the morphologically identified species labels are accurate.

Those same five materials were all labeled as O. tenuiflorum by the seed distributors.

“Fig. 2b highlights that these same materials are all labeled as O. tenuiflorum by the seed distributors.”

LASSO performed best among the models when classifying new greenhouse samples.

“LASSO was the most successful when classifying new greenhouse samples”

All three chemometric models-PLS, LASSO, and random forest-were unreliable for classifying external materials.

“all three chemometric models were unreliable in classifying external materials”

Four of the five samples were labeled as 'Kapoor' varieties.

“Four of the five samples were labeled as “Kapoor” varieties,”

The PCA scores plots contained distinct clusters for materials labeled and morphologically identified as O. basilicum.

“There were distinct clusters of the materials labeled and morphologically ID’d as O. basilicum in the PCA scores plots (Fig. 2).”

O. tenuiflorum samples displayed two separate chemotypes when HPTLC was used alongside untargeted profiling.

“combined with an orthogonal high-performance thin-layer chromatography (HPTLC) approach, O. tenuiflorum materials revealed two distinct chemotypes which could confound analysis”

The chemotypes did not necessarily match the species.

“that did not necessarily align with the species”

Species-based models had higher accuracies and correct classification rates than chemotype-based models.

“Overall, the models based on species had higher accuracies and correct classification rates (CCRs) than the models based on chemotype.”

The LASSO model for O. basilicum had the highest balanced accuracy at 71%.

“The LASSO model for O. basilicum had the highest balanced accuracy at 71%.”

The correct classification rate was often higher than the average balanced accuracy across the 100 runs.

“It is of note that the CCR was often higher than the average balanced accuracy over the 100 runs,”

When the three species models were combined for overall probability-based predictions, the final prediction was rarely correct.

“However, irrespective of the models’ accuracies, when the three species’ models were combined for overall probability-based predictions, it was rare that the final prediction was correct (Table 6).”

Overall, none of the three models performed well in classifying the reference materials.

“Overall, none of the three models performed well in classifying the reference materials (Table 4, Fig. 5 (row 2)).”

Balanced accuracies were slightly higher for consumer product predictions than for validation set predictions.

“While in most cases the balanced accuracies were slightly higher with the consumer products than the validation set predictions,”

The O. tenuiflorum and O. basilicum models achieved high balanced accuracies above 0.85.

“O. tenuiflorum and O. basilicum models resulted in high balanced accuracies (above 0.85),”

The external validation samples were reference standards that originated from different sources, locations, and growing conditions capable of influencing their chemical profiles.

“In our current study, samples in the external validation set were reference standards; they all originated from different sources, locations, and growing conditions that can influence their chemical profiles.”

Both PLS-DA and RF had low balanced accuracies overall.

“Both the PLS-DA and RF had low balanced accuracies overall,”

The O. gratissimum models had lower accuracies than the O. tenuiflorum and O. basilicum models.

“Indeed, the O. gratissimum models resulted in lower accuracies than the O. tenuiflorum and O. basilicum models.”

Sensitivity showed an inverse relationship between O. basilicum and the other two species: as O. basilicum sensitivity increased, O. gratissimum and O. tenuiflorum sensitivity decreased.

“When considering specificity and sensitivity, we observed inverse relationships between O. basilicum and O. gratissimum/O. tenuiflorum; as O. basilicum’s sensitivity increased, the other two species’ sensitivity decreased.”

Those methods do not capture intraspecies variation arising from chemotypes, environmental factors, and processing practices.

“this fails to incorporate the intraspecies variation that results from differences in chemotypes, environmental factors, and processing practices [1]”

• cites [1] Applequist WL, Miller JS. Selection and authentication of botanical materials for the development of analytical methods. Anal Bioanal Chem. 2013;405(13):4419–28. https://doi.org/10.1007/s00216-012-6595-1. sourceNot checked — no full text of the source

The greenhouse samples had highly controlled growth and processing conditions, reducing the variation typical of herbal product manufacturing.

“These findings agreed with the original hypotheses: greenhouse samples had highly controlled growth and processing conditions, which reduced sources of variation typical of herbal product manufacturing.”

Growing and processing conditions can affect the complexity of botanical metabolome profiles.

“Overall, this study highlights how growing and processing conditions can influence the complexity of botanical metabolome profiles”

Because the growth, processing, packaging, and shipping conditions of the ordered samples were unknown, the study could not include all potentially relevant variables.

“Since the conditions under which the ordered samples were grown, processed, packaged, and shipped were unknown (as is typical in a regulatory testing environment), we could not introduce all potentially relevant variables into our dataset.”

Available reference materials often lack suitable voucher specimens and identifying information.

“those available often lack appropriate voucher specimens and identifying information [1]”

• cites [1] Applequist WL, Miller JS. Selection and authentication of botanical materials for the development of analytical methods. Anal Bioanal Chem. 2013;405(13):4419–28. https://doi.org/10.1007/s00216-012-6595-1. sourceNot checked — no full text of the source

Many species are represented by only one or two reference materials.

“many species are represented by one or two reference materials”

• cites [1] Applequist WL, Miller JS. Selection and authentication of botanical materials for the development of analytical methods. Anal Bioanal Chem. 2013;405(13):4419–28. https://doi.org/10.1007/s00216-012-6595-1. sourceNot checked — no full text of the source

Species labels may not fully capture distinctions in metabolite composition.

“species labels may not fully incorporate these distinctions”

• AssumptionRests on an assumption: Herbal products' therapeutic benefits come from their unique metabolite compositions.

The PerMANOVA and PCA results clearly showed that the greenhouse samples were chemically distinct from the commercial samples.

“The PerMANOVA and PCA results clearly demonstrated that the greenhouse samples are chemically distinct from the commercial samples.”

The two O. tenuiflorum chemotypes could complicate analysis.

“O. tenuiflorum materials revealed two distinct chemotypes which could confound analysis”

Most identification methods compare a sample against a small set of reference standards intended to represent the whole species.

“Most identification approaches compare a sample to a limited number of reference standards meant to represent the characteristics of the entire species.”

• cites [1] Applequist WL, Miller JS. Selection and authentication of botanical materials for the development of analytical methods. Anal Bioanal Chem. 2013;405(13):4419–28. https://doi.org/10.1007/s00216-012-6595-1. sourceNot checked — no full text of the source

Green tea leaves have different metabolomes depending on season, geography, and harvest time.

“green tea leaves have differential metabolomes based on season, geography, and harvest time [3]”

• cites [3] Wang H, Cao X, Yuan Z, Guo G. Untargeted metabolomics coupled with chemometrics approach for Xinyang Maojian green tea with cultivar, elevation and processing variations. Food Chem. 2021;352:129359. https://doi.org/10.1016/j.foodchem.2021.129359. sourceNot checked — no full text of the source

The study demonstrated that chemometric models built on untargeted metabolomics data from controlled greenhouse samples could help classify new O. tenuiflorum and O. basilicum greenhouse samples.

“This study demonstrated that using chemometric models built on untargeted metabolomics data from controlled greenhouse samples could aid in classifying new O. tenuiflorum and O. basilicum greenhouse samples.”

All three models were generally able to correctly predict the species identity of unseen samples.

“meaning all three models were mostly able to correctly predict the species identity of unseen samples.”

These accuracies were still too low for reliable identification in an industry setting.

“they were still too low for reliable identification in an industry setting.”

Environmental variation is a major driver of Ocimum diversity, potentially greater than taxonomic classifications.

“This suggests that environmental variation is a large driver of Ocimum diversity, potentially greater than taxonomic classifications; this is borne out by other recent botanical studies.”

It is conceivable that a LASSO model could serve as an internal material verification system to check whether plants grown and processed under similar conditions have the expected chemical composition or unusual batch-to-batch metabolite variation.

“Thus, it is conceivable that using a LASSO model would be acceptable as an internal material verification system to confirm whether the chemical composition of the plants grown/processed under similar conditions conforms as expected or if there is unusual metabolite variation between batches.”

This pattern suggests that a probability-based prediction scheme could improve prediction results.

“suggesting that a probability-based prediction scheme could be useful for optimizing prediction results.”

Because the greenhouse samples shared the same environmental and processing conditions, this loosely resembled the conditions faced by a raw material grower.

“Since the greenhouse samples all had the same environmental and processing conditions, this loosely resembled the conditions that a raw material grower would face.”

The results cast doubt on the usefulness of controlled-environment samples as the foundation for machine learning and predictive modeling in herbal product identification studies.

“Taken together, these results cast a doubtful light on the usefulness of controlled-environment samples serving as the foundation of machine learning and predictive modeling in herbal product identification studies.”

Only one O. basilicum sample was included in the external validation set because reference materials were lacking.

“One note, only one O. basilicum sample was included in the external validation set due to a lack of available reference materials.”

Ultimately, chemometric modeling of controlled reference materials could not accurately predict the identity of commercially available products.

“Ultimately, chemometric modeling of controlled reference materials was not able to accurately predict the identity of commercially available products,”

The models built from greenhouse materials showed no indication of successfully predicting the species of external materials.

“and there was no indication that the models built with greenhouse materials could successfully predict the species of external materials (Table 5).”

There were significant differences in chemical composition between the greenhouse and commercial samples.

“as it was evident that there were significant differences in chemical composition between the greenhouse and commercial samples.”

Despite interspecies chemical overlap, species classifications gave more accurate predictions for new materials than chemotype classifications.

“However, these results indicate that even though there are interspecies chemical overlaps, species classifications provide more accurate predictions for new materials.”

Overall, the models developed from controlled greenhouse samples could not be applied to the external samples.

“Overall, the models developed based on controlled greenhouse samples could not be applied to these external samples.”