The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
A Usage-centric Take on Intent Understanding in E-Commerce
Open accessAbstract
Identifying and understanding user intents is a pivotal task for E-Commerce.Despite its essential role in product recommendation and business user profiling analysis, intent understanding has not been consistently defined or accurately benchmarked.In this paper, we focus on predicative user intents as "how a customer uses a product", and pose intent understanding as a natural language reasoning task, independent of product ontologies.We identify two weaknesses of FolkScope, the SOTA E-Commerce Intent Knowledge Graph: categoryrigidity and property-ambiguity.They limit its ability to strongly align user intents with products having the most desirable property, and to recommend useful products across diverse categories.Following these observations, we introduce a Product Recovery Benchmark featuring a novel evaluation framework and an example dataset.We further validate the above FolkScope weaknesses on this benchmark.Our code and dataset are available at https://github.com/stayones/Usgae-Centric- Intent-Understanding.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper proposes a usage-centric paradigm for E-Commerce intent understanding, where intents are natural-language usage scenarios and targets are “kinds of products” defined by category plus property. It refactors and analyzes the FolkScope intent KG, identifies property-ambiguity and category-rigidity, and introduces a Product Recovery Benchmark on Amazon Reviews data.
Key findings
- FolkScope suffers from property-ambiguity: for up to 20% of cases, Jensen-Shannon Divergence between intent-conditioned edge-weight distributions and prior property distributions is < 0.1, while only 2% of cases have JSD > 0.5.
- FolkScope suffers from category-rigidity: category-entropy values concentrate in [0, 0.02) and [0.68, 0.70), meaning over 80% of intents are associated with only one or two product categories.
- On bought-product-recovery, MRR max is 0.192 for Clothing and 0.263 for Electronics with FolkScope; FolkScope-properties scores 0.116 and 0.166; FolkScope + GPT scores 0.187 and 0.257.
- On co-bought-product-recovery across categories, FolkScope reaches low MRR max of 0.077 for Clothing and 0.033 for Electronics. LLM reranking with GPT-3.5-turbo shows no clear benefit; hits are polarized, with hit@1 of 16% for Clothing and 22% for Electronics, and hit > 10 of 73% and 63%, respectively.
Methods
- The authors define a usage-centric intent understanding task: predict natural-language usage intents for a product listing, then recover useful “kinds of products” defined as fine-grained category plus one property, independent of product ontologies.
- They refactor FolkScope by keeping five predicative relations (UsedFor, CapableOf, Result, Cause, CauseDesire), grouping ARD product listings into kinds using fine-grained categories and FolkScope PropertyOf attributes, and computing intent-to-kind association strengths via aggregation with PMI.
- They build a Product Recovery Benchmark from the Amazon Reviews Dataset, with 80/10/10 train/validation/test splits: Clothing has 30,296/2,027/2,088 listings and Electronics has 85,086/7,853/7,900. Evaluation uses MRR max, mainly bought-product-recovery, plus co-bought-product-recovery and GPT-3.5-turbo reranking.
Limitations
- The paper states it analyzes only one SOTA intent understanding KG (FolkScope) and one SOTA LLM due to lack of consistent task definition and limited computational budget.
- The paper states it does not provide empirical evidence for the hypothesis that using genuine user reviews would relieve property-ambiguity and category-rigidity, leaving this to future work.
- The paper notes risks that methods developed on the benchmark may bias customer decisions, but argues the task definition is purely natural language and does not involve individual product listings.
- The text also makes evident that dataset artefacts from the common source corpus (AWD) could inflate reported MRR max values, and that GPT-4 matching for end-to-end evaluation is over-permissive and can produce spurious matches.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 9 cited sources checked · 0 supported · 0 flagged
- Findingown data
GPT-3.5-turbo does not outperform the FolkScope KG baseline on the product recovery benchmark.
“GPT-3.5turbodoes not outperform the FolkScope KG baseline on the product recovery benchmark.”
Quote in the paperSubstantial claim - Findingown data
FolkScope reaches a slightly higher MRR max in Clothing than in Electronics in the co-bought-product-recovery setup.
“Notably, between the two domains, FolkScope reaches a slightly higher MRR max in Clothing.”
Quote in the paperNot a substantial claim - Findingown data
FolkScope_properties reached respectable performance with only moderate regression from FolkScope predictions.
“From Table 1, we observe that FolkScopeproperties reached respectable performance with only moderate regression from FolkScope predictions.”
Quote in the paperSubstantial claim - Findingown data
LLM reranking with GPT-3.5-turbo provides no clear benefit on the product recovery benchmark.
“we observe no clear benefit with LLM-reranking.”
Quote in the paperSubstantial claim - Findingown data
The entropy values in FolkScope are concentrated in two narrow ranges, [0, 0.02) and [0.68, 0.70).
“entropy values are concentrated in 2 narrow ranges, [0, 0.02) and [0.68, 0.70).”
Quote in the paperSubstantial claim
- Findingown data
FolkScope KG exhibits property-ambiguity: generated user intents are poorly aligned with relevant product properties, so the KG often maps user intents to kinds of products with relevant category but fairly random properties.
“We identify two weaknesses of FolkScope, the SOTA E-Commerce Intent Knowledge Graph: categoryrigidity and property-ambiguity.”
- Also stated in: § Intents Kinds of products Products · plain
“1) property-ambiguity: generated user intents are poorly aligned with relevant product properties, such that the KG often maps user intents to kinds of products with relevant category but fairly random properties”
- Also stated in: § 3.2 Statistical Analysis · plain
“We identify two major weaknesses of FolkScope KG under the usage-centric paradigm: it is overspecific about categories of useful products, but under-specific about the required properties of these products within each category.”
Quote in the paperSubstantial claim - Also stated in: § Intents Kinds of products Products · plain
- Findingown data
GPT-4 evaluation has a significantly more permissive criterion on matching than strict string matching, yielding much higher MRR max values.
“Compared to the strict string matching results in Table 1, GPT-4 evaluation has a significantly more permissive criterion on matching, yielding much higher MRR max values.”
Quote in the paperSubstantial claim - Findingown data
Over 80% of the intents in FolkScope are associated with only one or two product categories.
“from Figure 3 we can conclude that over 80% of the intents are associated with only one or two categories.”
Quote in the paperSubstantial claim - Findingown data
FolkScope KG exhibits category-rigidity: each intent is strongly associated with a single category of product, so the KG cannot recommend diverse products across different categories that serve common intents.
“2) category-rigidity: each intent is strongly associated with a single category of product, such that the KG is unable to recommend diverse products across different categories that serve common intents.”
- Also stated in: § 3.2 Statistical Analysis · plain
“This category-rigidity in FolkScope hampers its ability to recommend diverse kinds of products”
Quote in the paperSubstantial claim - Also stated in: § 3.2 Statistical Analysis · plain
- Findingown data
Many of the matched verdicts by GPT-4 are spurious.
“We find many of these”
Quote in the paperSubstantial claim - Interpretationreasoning
Grouping based on kinds of products strikes a balance between sparsity that comes with specificity and ambiguity that comes with generality.
“In short, grouping based on kinds of products strikes a balance between sparsity that comes with specificity, and ambiguity that comes with generality.”
Quote in the paperNot a substantial claim - Premisecitation
In prior work, user intents are often blended with product properties or similar products, which are related to the products and not the users.
“In prior work (Yu et al., 2023;Luo et al., 2021), user intents are often blended with”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Yu et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Luo et al., 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
User intents are a crucial source of information for E-Commerce.
“User intents are a crucial source of information for E-Commerce (Deng et al., 2023;Er-Rahmadi et al., 2023;Zhang et al., 2016;Hao et al., 2022).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Deng et al., 2023 source
Checked before — its earlier verdict is held back until a new check runs.
- cites Er-Rahmadi et al., 2023 source
Checked before — its earlier verdict is held back until a new check runs.
- cites Zhang et al., 2016
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
The paper's task definition would not bias customer choices among directly competing listings of the same kinds of products.
“therefore it would not bias customer choices among directly competing listings of the same kinds of products.”
Quote in the paperNot a substantial claim - Study design and conductnone given
Compared to the product recommendation evaluation in Yu et al. (2023), the proposed framework marginalizes factors inciting co-buy behaviour such as brand loyalty and geolocation.
“Compared to the product recommendation evaluation in Yu et al. (2023), this framework marginalizes factors inciting co-buy behaviour (e.g.brand loyalty, geolocation, etc.).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Yu et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
Traditional product recommendation approaches would fail to handle queries or remind customers of products they may need but have forgotten in cases where user intent spans multiple product types.
“In these cases, traditional product recommendation approaches would fail to handle these queries or to remind customers of the products they may need but have forgotten.”
Quote in the paperNot a substantial claim - Interpretationown data
The FolkScope KG cannot effectively recommend superficially distinct kinds of products connected with the same user intents.
“the FolkScope KG cannot effectively recommend superficially distinct kinds of products connected with the same user intents.”
Quote in the paperSubstantial claim - Premisecitation
FolkScope is the state-of-the-art E-Commerce Intent Knowledge Graph.
“FolkScope, the SOTA E-Commerce Intent Knowledge Graph”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Yu et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- It may state two separate claims as one. 14
- Probably not a substantial claim. 11
It may state two separate claims as one.
Findingown dataMost hits in the predictions are either at first or not in the top 10.
“we find that most hits are either at first or not in the top 10.”
Quote in the paperSubstantial claimIt may state two separate claims as one.
Findingown dataFor up to 20% of cases, the Jensen-Shannon Divergence between conditional and prior edge-weight distributions is less than 0.1, and only 2% of cases have JSD greater than 0.5.
“for up to 20% of cases, JSD is < 0.1, where only 2% of cases have JSD > 0.5.”
Quote in the paperSubstantial claimIt may state two separate claims as one.
Findingown dataThe KG's edge weights among differently-propertied kinds of products within the same category are strongly predicted by their prior distribution and are insensitive to the specific usages depicted by user intents.
“This shows, the KG's edge weights among differently-propertied kinds of products within the same category are strongly predicted by their prior distribution, and are insensitive to the specific usages depicted by user intents.”
Quote in the paperSubstantial claimIt may state two separate claims as one.
Findingown dataIn the co-bought-product-recovery setup, FolkScope achieves low MRR max of 0.077 for Clothing and 0.033 for Electronics.
“we observe low MRR max of 0.077 and 0.033 for Clothing and Electronics domains, respectively”
Quote in the paperSubstantial claimIt may state two separate claims as one.
Findingnone givenThe attributes under the relation PropertyOf in the original FolkScope KG do not fit the criteria for usage user intents, but are acquired through generic LLM prompted summarization and are borrowed as product properties.
“These attributes do not fit the criteria for usage user intents, but they are acquired through generic LLM prompted summarization, and thus are borrowed as product properties.”
Quote in the paperSubstantial claimIt may state two separate claims as one.
InterpretationreasoningThe weaknesses of FolkScope limit its ability to strongly align user intents with products having the most desirable property and to recommend useful products across diverse categories.
“They limit its ability to strongly align user intents with products having the most desirable property, and to recommend useful products across diverse categories.”
Quote in the paperSubstantial claimProbably not a substantial claim. It may state two separate claims as one.
InterpretationreasoningDataset artefacts from the common source corpus (AWD) could be behind the abnormally high hit-at-1 rate, and the reported MRR max values may have been inflated.
“We raise the warning that dataset artefacts from the common source corpus (AWD) could be behind this abnormally high hit-at-1 rate (compared with the MRR max value), where the reported MRR max values may have been inflated.”
Quote in the paperSubstantial claimIt may state two separate claims as one.
Interpretationown dataThe slightly higher MRR max in Clothing is consistent with Figure 3, where category-entropy values are slightly more spread than in Electronics, indicating category rigidity is less severe.
“This is consistent with our findings in Figure 3, where category-entropy values are slightly more spread than in Electronics (i.e.category rigidity is less severe).”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
InterpretationcitationThese shortcuts may benefit existing product recommendation benchmarks, but are not aligned with the intent understanding objective of retrieving superficially distinct kinds of products serving common intents.
“These shortcuts may benefit existing product recommendation benchmarks, but are not aligned with the intent understanding objective, namely, to retrieve superficially distinct kinds of products serving common intents”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Huang et al., 2024 source
Probably not a substantial claim. The paper’s conclusions may not depend on it.
PremisecitationFolkScope reported promising results on an intrinsic co-buy prediction task.
“FolkScope (Yu et al., 2023), which reported promising results on an intrinsic co-buy prediction task.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Yu et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
It may state two separate claims as one.
Interpretationown dataAn entropy in [0, 0.02) indicates that the associations about an intent are focused on only one product category, and [0.68, 0.70) indicates that the associations are focused on two product categories.
“We notice that an entropy in [0, 0.02) indicates that the associations about this intent are focused on only one product category; [0.68, 0.70) indicates that the associations are focused on two product categories.”
Quote in the paperSubstantial claimProbably not a substantial claim.
Study design and conductreasoningThe paper takes bought-product-recovery as the main evaluation setup because it focuses on intent-to-kinds-of-product associations.
“We take bought-product-recovery as our main evaluation setup, since it focuses on intent-to-kindsof-product associations.”
Quote in the paperSubstantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
Interpretationnone givenAll intent understanding methods developed on the Amazon Reviews Dataset can be evaluated using the proposed benchmark.
“All intent understanding methods developed on the ARD can be evaluated using this benchmark.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
Study design and conductreasoningDue to the lack of consistent task definition and limited computational budget, the paper analyses only one state-of-the-art intent understanding KG (FolkScope) and one state-of-the-art LLM.
“Due to the lack of consistent task definition and limited computational budget, we are only able to analyse one SOTA intent understanding KG (namely FolkScope) and one SOTA LLM.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Study design and conductreasoningThe paper includes the co-bought-product-recovery setup to validate statistical findings on cross-category recommendation performance.
“We also include the cobought-product-recovery setup to validate statistical findings on cross-category recommendation performance.”
Quote in the paperNot a substantial claimProbably not a substantial claim. The paper’s conclusions may not depend on it.
Study design and conductnone givenThe paper's elicitation procedure is corpus-agnostic.
“Our elicitation procedure is corpus-agnostic”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
Interpretationown dataThe limited MRR gap between FolkScope_properties and FolkScope shows the impact of property-ambiguity, and performance gains could be expected with better property alignment.
“This limited MRR gap shows the impact of property-ambiguity, where performance gains could be expected with better property alignment.”
Quote in the paperSubstantial claimProbably not a substantial claim. The paper may give no reason why it holds.
Interpretationnone givenPredicting at the level of kinds of products guarantees that the list of relevant predictions is not endless.
“Predicting at the level of the kinds of products guarantees that the list of relevant predictions is not endless.”
Quote in the paperNot a substantial claimProbably not a substantial claim. It may state two separate claims as one.
Premisenone givenThe paper's task definition is purely natural language and does not involve any individual product listings.
“our task definition is purely natural language and does not involve any individual product listings”
Quote in the paperSubstantial claimProbably not a substantial claim.
Premisenone givenIntents reveal users' motivation in E-Commerce interactions.
“Intents reveal users' motivation in E-Commerce interactions”
Quote in the paperSubstantial claimProbably not a substantial claim. It may be too vague to check against evidence.
InterpretationreasoningIntent Understanding offers great benefits in recommending distinct products based on common user intents they fulfil.
“Intent Understanding offers great benefits in recommending distinct products based on common user intents they fulfil.”
Quote in the paperNot a substantial claimIt may state two separate claims as one.
Interpretationown dataGPT-4 cannot easily achieve reliable matching for the product recovery benchmark, and more robust criteria are needed before replacing the exact match criterion.
“conclude that GPT-4 cannot easily achieve reliable matching for the product recovery benchmark, and more robust criteria are needed before replacing the exact match criterion.”
Quote in the paperSubstantial claim
The paper relies on this without support.
User intents rarely require combinations of properties in a product category.
“User intents rarely require combinations of properties in a product category.”
The paper relies on this without support.
The Amazon Reviews Dataset is the largest available dataset.
“we empirically select ARD as it is the largest available dataset”
The paper relies on this without support.
A kind of product, represented by category plus property, contains just enough information to represent the product listings inside for intent understanding.
“It contains just enough information (category + property) to represent the product listings inside for intent understanding.”
The paper relies on this without support.
For a given user intent, the edge-weight distribution among different kinds of products within a category should depend on the specific usage scenario.
“For example, for the user intent of outdoor barbecues, its edge weights distribution among different kinds of scrub brush products should depend on this specific usage scenario.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Source corpus is called both AWD and ARD
MinorThe benchmark is described as based on the Amazon Reviews Dataset (ARD), but the discussion of artefacts from its common source corpus calls it AWD.
“dataset artefacts from the common source corpus (AWD) could be behind this abnormally high hit-at-1 rate”
“We instantiate the proposed evaluation framework with a product recovery benchmark, based on the ARD (Ni et al., 2019)”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 8
- Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chai- tanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: Commonsense Transformers for Automatic Knowledge Graph Construction. In Pro- ceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 4762- 4779, Florence, Italy. Association for Computational Linguistics. · DOI · Paper page
- Li Chen and Feng Wang. 2013. Preference-based clus- tering reviews for augmenting e-commerce recom- mendation. Knowledge-Based Systems, 50:44-59. · DOI
- Xiao Ding, Ting Liu, Junwen Duan, and Jian-Yun Nie. 2015. Mining User Consumption Intention from So- cial Media Using Domain Adaptive Convolutional Neural Network. Proceedings of the AAAI Confer- ence on Artificial Intelligence, 29(1). Number: 1. · DOI
- Zhenyun Hao, Jianing Hao, Zhaohui Peng, Senzhang Wang, Philip S. Yu, Xue Wang, and Jian Wang. 2022. Dy-hien: Dynamic evolution based deep hierarchi- cal intention network for membership prediction. In Proceedings of the Fifteenth ACM International Con- ference on Web Search and Data Mining, WSDM '22, page 363-371, New York, NY, USA. Association for Computing Machinery. · DOI
- Xusheng Luo, Le Bo, Jinhang Wu, Lin Li, Zhiy Luo, Yonghua Yang, and Keping Yang. 2021. AliCoCo2: Commonsense Knowledge Extraction, Representa- tion and Application in E-commerce. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 3385-3393, Virtual Event Singapore. ACM. · DOI
- Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Jus- tifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Interna- tional Joint Conference on Natural Language Pro- cessing (EMNLP-IJCNLP), pages 188-197, Hong Kong, China. Association for Computational Lin- guistics. · DOI
- Maarten Sap, Ronan Le Bras, Emily Allaway, Chan- dra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, and Yejin Choi. 2019. ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning. Proceedings of the AAAI Confer- ence on Artificial Intelligence, 33:3027-3035. · DOI
- Chenwei Zhang, Wei Fan, Nan Du, and Philip S. Yu. 2016. Mining user intentions from medical queries: A neural network based heterogeneous jointly mod- eling approach. In Proceedings of the 25th Interna- tional Conference on World Wide Web, WWW '16, page 1373-1384, Republic and Canton of Geneva, CHE. International World Wide Web Conferences Steering Committee. · DOI
Cited by
Sorted by citations
Performance Trade-Offs of Optimizing Small Language Models for E-Commerce
Licardo, J. T., Tanković, N., Osman, I., Lorencin, I., & Šegota, S. B. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10050155 · Cited by 5
Metadata
- Authors
- Zhou, W., Li, T., Vougiouklis, P., Steedman, M., & Pan, J. Z.
- Affiliations
- University of Edinburgh (GB)
- Venue
- Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
- Published
- 2024
- Pages
- 228-236
- OpenAlex
- W4404783444
- Cited by
- 1
- References
- 0
- Citations by year
- 2026: 1
- FWCI
- 0.38
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- dx.doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- user intent understanding, e-commerce, product recommendation, product recovery, benchmark, natural language reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
