CoChat
Sign Up
Cited by 107Speech and dialogue systemsMulti-Agent Systems and NegotiationNatural Language Processing Techniques

WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents

Open access
Yao, Shunyu, Chen, Howard, Yang, John, Narasimhan, Karthik·2022·Advances in Neural Information Processing Systems 35
DOI 10.52202/068431-1508
Open this paper in CoChat — free accountOpen PDF

Abstract

Existing benchmarks for grounding language in interactive environments either lack real-world linguistic elements, or prove difficult to scale up due to substantial human involvement in the collection of data or feedback signals. To bridge this gap, we develop WebShop -- a simulated e-commerce website environment with $1.18$ million real-world products and $12,087$ crowd-sourced text instructions. Given a text instruction specifying a product requirement, an agent needs to navigate multiple types of webpages and issue diverse actions to find, customize, and purchase an item. WebShop provides several challenges for language grounding including understanding compositional instructions, query (re-)formulation, comprehending and acting on noisy text in webpages, and performing strategic exploration. We collect over $1,600$ human demonstrations for the task, and train and evaluate a diverse range of agents using reinforcement learning, imitation learning, and pre-trained image and language models. Our best model achieves a task success rate of $29\%$, which outperforms rule-based heuristics ($9.6\%$) but is far lower than human expert performance ($59\%$). We also analyze agent and human trajectories and ablate various model components to provide insights for developing future agents with stronger language understanding and decision making abilities. Finally, we show that agents trained on WebShop exhibit non-trivial sim-to-real transfer when evaluated on amazon.com and ebay.com, indicating the potential value of WebShop in developing practical web-based agents that can operate in the wild.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.52202/068431-1508

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Webshop contains sensitive user queries about Amazon products.

    “And the queries in RAP's memory are randomly selected from Webshop (Yao et al., 2022), which contains sensitive users queries about Amazon products.”

    From Unveiling Privacy Risks in LLM Agent Memory (2025) · cites this paper as Yao et al., 2022

    Section 3.2 describes WebShop’s Amazon product data and crowdsourced shopping instructions; the human-trajectory instructions also ask participants to write product search queries. Our copy does not describe those queries as sensitive. Appendix A.1 reports no personally identifiable information in sampled product data, but that check does not establish whether the queries are sensitive. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Realworld simulation evaluates agents within immersive environments like games and interactive simulators, using metrics like task success rate and human similarity based on trajectories and completed objectives.

    “In this method, the agents are evaluated within immersive environments like games and interactive simulators.The agents are required to perform tasks autonomously, and then metrics like task success rate and human similarity are leveraged to evaluate the capability of the agents based on their trajectories and completed objectives [16,22,33,38,59,80,122,162,163,164].”

    From A survey on large language model based autonomous agents (2024) · cites this paper as [80]

    The Discussion describes WebShop as a simulated web-shopping benchmark for evaluating autonomous agents, and §5.4 reports task scores and success rates. Section 5.3 compares agent and human trajectories, but does not use human similarity as an evaluation metric. The paper also does not evaluate agents in games or other immersive environments. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Multi-task evaluation uses a set of diverse tasks from different domains to evaluate agent generalization capability in open-domain environments.

    “In this method, people use a set of diverse tasks from different domains to evaluate the agent, which can effectively measure the agent generalization capability in open-domain environments [29,80,153,163,165,166,167].”

    From A survey on large language model based autonomous agents (2024) · cites this paper as [80]

    The Introduction describes the web as an open-domain interactive environment, and §5.4 evaluates transfer to Amazon and eBay despite domain shifts. But §5.1 evaluates WebShop shopping instructions with an i.i.d. split and says splits with larger generalization gaps, such as by product category, are future work. The paper does not support the attributed claim that its evaluation uses diverse tasks from different domains to measure open-domain generalization. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • WebShop simulates e-commerce search over 12,087 products, requiring agents to follow natural-language instructions via multi-step navigation and filtering.

    “WebShop [25] (1,624 train, 200 test) simulates e-commerce search over 12,087 products, requiring agents to follow natural-language instructions via multi-step navigation and filtering.”

    From Learning Hierarchical Procedural Memory for LLM Agents through Bayesian Selection and Contrastive Refinement (2025) · cites this paper as [25]

    Section 3 describes WebShop as a simulated shopping environment in which agents follow natural-language instructions through searches and multi-step webpage navigation. But §3.2 reports 1,181,436 products; 12,087 is the number of instructions, not products. Section 5.1 gives a train/development/test split of 10,587/1,000/500 instructions, rather than the citing sentence’s 1,624 train and 200 test. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • Frameworks such as ToolBench, WebShop, and WorkArena rely on fixed APIs or self-hosted environments that abstract away authentication, failure recovery, and long-term user or task state.

    “Frameworks such as ToolBench (Qin et al., 2024), WebShop (Yao et al., 2022), and WorkArena (Drouin et al., 2024) rely on fixed APIs or self-hosted environments that abstract away authentication, failure recovery, and long-term user or task state.”

    From A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents (2026) · cites this paper as Yao et al., 2022

    Section 3 (p. 3) describes WebShop as a simulated shopping environment with a defined set of actions, and §3.1 models its state as a web page with deterministic transitions. This supports the characterization of WebShop as a bounded environment, but the paper does not explicitly address authentication, failure recovery, or long-term user or task state. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Cited by 107

Part of this paper’s reference list could not be split into single entries.

Research works printed but not linked: 58

  1. [1][1] L. Adolphs, B. Boerschinger, C. Buck, M. C. Huebscher, M. Ciaramita, L. Espeholt, T. Hof- mann, and Y. Kilcher. Boosting Search Engines with Interactive Agents. arXiv preprint arXiv:2109.00527, 2021. · arXiv
  2. [2][2] A. Aghajanyan, D. Okhonko, M. Lewis, M. Joshi, H. Xu, G. Ghosh, and L. Zettlemoyer. Htlm: Hyper-text pre-training and prompting of language models. ArXiv, abs/2107.06955, 2021. · arXiv
  3. [3][3] J. Andreas, J. Bufe, D. Burkett, C. Chen, J. Clausman, J. Crawford, K. Crim, J. DeLoach, L. Dorner, J. Eisner, et al. Task-oriented dialogue as dataflow synthesis. Transactions of the Association for Computational Linguistics, 8:556–571, 2020.
  4. [4][4] E. M. Bender and A. Koller. Climbing towards NLU: On Meaning, Form, and Understand- ing in the Age of Data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198, 2020.
  5. [5][5] G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. OpenAI Gym. arXiv preprint arXiv:1606.01540, 2016. · arXiv
  6. [6][6] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  7. [7][7] P. Budzianowski, T.-H. Wen, B.-H. Tseng, I. Casanueva, S. Ultes, O. Ramadan, and M. Gašić. Multiwoz–a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. arXiv preprint arXiv:1810.00278, 2018. · arXiv
  8. [8][8] A. Burns, D. Arsan, S. Agrawal, R. Kumar, K. Saenko, and B. A. Plummer. Interactive Mobile App Navigation with Uncertain or Under-specified Natural Language Commands. arXiv preprint arXiv:2202.02312, 2022. · arXiv
  9. [9][9] J. Chung, Çaglar Gülçehre, K. Cho, and Y. Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. ArXiv, abs/1412.3555, 2014. · arXiv
  10. [10][10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv, abs/1810.04805, 2019. · arXiv
  11. [11][11] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1), 2019.
  12. [12][12] A. Ecoffet, J. Huizinga, J. Lehman, K. O. Stanley, and J. Clune. Go-explore: a new approach for hard-exploration problems. arXiv preprint arXiv:1901.10995, 2019. · arXiv
  13. [13][13] M. Fortunato, M. Tan, R. Faulkner, S. Hansen, A. Puigdomènech Badia, G. Buttimore, C. Deck, J. Z. Leibo, and C. Blundell. Generalization of reinforcement learners with working and episodic memory. Advances in neural information processing systems, 32, 2019.
  14. [14][14] X. Guo, M. Yu, Y. Gao, C. Gan, M. Campbell, and S. Chang. Interactive fiction game playing as multi-paragraph reading comprehension with reinforcement learning. arXiv preprint arXiv:2010.02386, 2020. · arXiv
  15. [15][15] I. Gur, U. Rueckert, A. Faust, and D. Hakkani-Tur. Learning to Navigate the Web. arXiv preprint arXiv:1812.09195, 2018. · arXiv
  16. [16][16] I. Gur, N. Jaques, K. Malta, M. Tiwari, H. Lee, and A. Faust. Adversarial Environment Generation for Learning to Navigate the Web. arXiv preprint arXiv:2103.01991, 2021. · arXiv
  17. [17][17] M. Hausknecht, P. Ammanabrolu, M.-A. Côté, and X. Yuan. Interactive fiction games: A colossal adventure. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7903–7910, 2020. 11
  18. [18][18] K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
  19. [19][19] A. Hotti, R. S. Risuleo, S. Magureanu, A. Moradi, and J. Lagergren. The Klarna Prod- uct Page Dataset: A RealisticBenchmark for Web Representation Learning. arXiv preprint arXiv:2111.02168, 2021. · arXiv
  20. [20][20] P. C. Humphreys, D. Raposo, T. Pohlen, G. Thornton, R. Chhaparia, A. Muldal, J. Abramson, P. Georgiev, A. Goldin, A. Santoro, et al. A data-driven approach for learning to control computers. arXiv preprint arXiv:2202.08137, 2022. · arXiv
  21. [21][21] S. Jia, J. Kiros, and J. Ba. Dom-q-net: Grounded RL on Structured Language. arXiv preprint arXiv:1902.07257, 2019. · arXiv
  22. [22][22] M. Komeili, K. Shuster, and J. Weston. Internet-augmented dialogue generation. arXiv preprint arXiv:2107.07566, 2021. · arXiv
  23. [23][23] A. Lampinen, S. Chan, A. Banino, and F. Hill. Towards mental time travel: a hierarchical memory for reinforcement learning agents. Advances in Neural Information Processing Systems, 34:28182–28195, 2021.
  24. [24][24] A. Lazaridou, A. Potapenko, and O. Tieleman. Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning. In ACL, 2020.
  25. [25][25] A. Lazaridou, E. Gribovskaya, W. Stokowiec, and N. Grigorev. Internet-augmented lan- guage models through few-shot prompting for open-domain question answering. ArXiv, abs/2203.05115, 2022. · arXiv
  26. [26][26] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. arXiv preprint arXiv:1910.13461, 2019. · arXiv
  27. [27][27] X. Li, X. Yin, C. Li, P. Zhang, X. Hu, L. Zhang, L. Wang, H. Hu, L. Dong, F. Wei, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. In European Conference on Computer Vision, pages 121–137. Springer, 2020.
  28. [28][28] J. Lin, X. Ma, S.-C. Lin, J.-H. Yang, R. Pradeep, and R. Nogueira. Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representationss. arXiv preprint arXiv:2102.10073, 2021. · arXiv
  29. [29][29] E. Z. Liu, K. Guu, P. Pasupat, T. Shi, and P. Liang. Reinforcement Learning on Web Interfaces using Workflow-Guided Exploration. arXiv preprint arXiv:1802.08802, 2018. · arXiv
  30. [30][30] J. Luketina, N. Nardelli, G. Farquhar, J. N. Foerster, J. Andreas, E. Grefenstette, S. Whiteson, and T. Rocktäschel. A survey of reinforcement learning informed by natural language. In IJCAI, 2019.
  31. [31][31] S. Mazumder and O. Riva. FLIN: A Flexible Natural Language Interface for Web Navigation. arXiv preprint arXiv:2010.12844, 2020. · arXiv
  32. [32][32] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928–1937. PMLR, 2016.
  33. [33][33] R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, et al. WebGPT: Browser-Assisted Question-Answering with Human Feedback. arXiv preprint arXiv:2112.09332, 2021. · arXiv
  34. [34][34] K. Narasimhan, A. Yala, and R. Barzilay. Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2355–2365, 2016.
  35. [36][36] R. Nogueira and K. Cho. End-to-End Goal-Driven Web Navigation. Advances in Neural Information Processing Systems, 29, 2016.
  36. [37][37] R. Nogueira and K. Cho. Task-Oriented Query Reformulation with Reinforcement Learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 574–583, 2017.
  37. [38][38] P. Pasupat, T.-S. Jiang, E. Z. Liu, K. Guu, and P. Liang. Mapping natural language commands to web elements. In Empirical Methods in Natural Language Processing (EMNLP), 2018.
  38. [39][39] P. Pasupat, T.-S. Jiang, E. Z. Liu, K. Guu, and P. Liang. Mapping natural language commands to web elements. In EMNLP, 2018.
  39. [40][40] D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell. Curiosity-driven exploration by self- supervised prediction. In International conference on machine learning, pages 2778–2787. PMLR, 2017.
  40. [42][42] M. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi. Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603, 2016. · arXiv
  41. [43][43] T. Shi, A. Karpathy, L. Fan, J. Hernandez, and P. Liang. World of Bits: An Open-Domain platform for web-based agents. In International Conference on Machine Learning, pages 3135–3144. PMLR, 2017.
  42. [44][44] M. Shridhar, J. Thomason, D. Gordon, Y. Bisk, W. Han, R. Mottaghi, L. Zettlemoyer, and D. Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10740–10749, 2020.
  43. [45][45] K. Shuster, M. Komeili, L. Adolphs, S. Roller, A. D. Szlam, and J. Weston. Language models that seek for knowledge: Modular search & generation for dialogue and prompt completion. ArXiv, abs/2203.13224, 2022. · arXiv
  44. [46][46] Y. Su, A. H. Awadallah, M. Khabsa, P. Pantel, M. Gamon, and M. Encarnacion. Building Natural Language Interfaces to Web APIs. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pages 177–186, 2017.
  45. [47][47] Y. Su, A. Hassan Awadallah, M. Wang, and R. W. White. Natural Language Interfaces with Fine-Grained User Interaction: A Case Study on Web APIs. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 855–864, 2018.
  46. [48][48] D. Toyama, P. Hamel, A. Gergely, G. Comanici, A. Glaese, Z. Ahmed, T. Jackson, S. Mourad, and D. Precup. AndroidEnv: A Reinforcement Learning Platform for Android. arXiv preprint arXiv:2105.13231, 2021. · arXiv
  47. [49][49] J. Tuyls, S. Yao, S. Kakade, and K. Narasimhan. Multi-stage episodic control for strategic exploration in text games. arXiv preprint arXiv:2201.01251, 2022. · arXiv
  48. [50][50] V. Uc-Cetina, N. Navarro-Guerrero, A. Martin-Gonzalez, C. Weber, and S. Wermter. Survey on reinforcement learning for language processing. arXiv preprint arXiv:2104.05565, 2021. · arXiv
  49. [51][51] X. Wang, C. Macdonald, and I. Ounis. Deep reinforced query reformulation for information retrieval. arXiv preprint arXiv:2007.07987, 2020. · arXiv
  50. [52][52] Z. Wang, J. Yu, A. W. Yu, Z. Dai, Y. Tsvetkov, and Y. Cao. Simvlm: Simple visual language model pretraining with weak supervision. arXiv preprint arXiv:2108.10904, 2021. · arXiv
  51. [53][53] G. Wayne, C.-C. Hung, D. Amos, M. Mirza, A. Ahuja, A. Grabska-Barwinska, J. Rae, P. Mirowski, J. Z. Leibo, A. Santoro, et al. Unsupervised predictive memory in a goal-directed agent. arXiv preprint arXiv:1803.10760, 2018. 13 https://flask.palletsprojects.com/en/2.1.x/ · arXiv
  52. [54][54] K. Williams, S. H. Hashemi, and I. Zitouni. Automatic Task Completion Flows from Web APIs. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1009–1012, 2019.
  53. [55][55] S. Yao, R. Rao, M. J. Hausknecht, and K. Narasimhan. Keep CALM and Explore: Language Models for Action Generation in Text-based Games. ArXiv, abs/2010.02903, 2020. · arXiv
  54. [56][56] S. Yao, K. Narasimhan, and M. Hausknecht. Reading and acting while blindfolded: The need for semantics in text game agents. arXiv preprint arXiv:2103.13552, 2021. · arXiv
  55. [57][57] X. Yuan, J. Fu, M.-A. Côté, Y. Tay, C. J. Pal, and A. Trischler. Interactive machine comprehen- sion with information seeking agents. In ACL, 2020.
  56. [58][58] V. Zhong, A. W. Hanjie, S. Wang, K. Narasimhan, and L. Zettlemoyer. Silg: The multi- domain symbolic interactive language grounding benchmark. Advances in Neural Information Processing Systems, 34:21505–21519, 2021.
  57. [59][59] S. Zhuang, H. Ren, L. Shou, J. Pei, M. Gong, G. Zuccon, and D. Jiang. Bridging the gap between indexing and retrieval for differentiable search index with query generation. arXiv preprint arXiv:2206.10128, 2022. Checklist · arXiv
  58. [1]1. For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] (b) Did you describe the limitations of your work? [Yes] See Section 6 Discussion and Appendix. (c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section 6 Discussion and Appendix. (d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]

Authors
Yao, S., Chen, H., Yang, J., & Narasimhan, K.
ORCID
Venue
Advances in Neural Information Processing Systems 35
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2022
Pages
20744-20757
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/068431-1508
OpenAlex
W7133227044
Cited by
107
References
0
Citations by year
2026: 56 · 2025: 32 · 2024: 14 · 2023: 5
FWCI
7.05
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
National Science Foundation (2107048)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
web interaction, WebShop, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review