Rousillon: Scraping Distributed Hierarchical Web Data
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Programming by Demonstration (PBD) promises to enable data scientists to collect web data. However, in formative interviews with social scientists, we learned that current PBD tools are insufficient for many real-world web scraping tasks. The missing piece is the capability to collect hierarchically-structured data from across many different webpages. We present Rousillon, a programming system for writing complex web automation scripts by demonstration. Users demonstrate how to collect the first row of a 'universal table' view of a hierarchical dataset to teach Rousillon how to collect all rows. To offer this new demonstration model, we developed novel relation selection and generalization algorithms. In a within-subject user study on 15 computer scientists, users can write hierarchical web scrapers 8 times more quickly with Rousillon than with traditional programming.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., et al. (2009) · Communications of the ACM · doi:10.1145/1592761.1592779 · Cited by 3,333
Automating string processing in spreadsheets using input-output examples
Gulwani, S. (2011) · ACM SIGPLAN Notices · doi:10.1145/1925844.1926423 · Cited by 581
EAGER: programming repetitive tasks by example
Cypher, A. (1991) · Proceedings of the SIGCHI conference on Human factors in computing systems Reaching through technology - CHI '91 · doi:10.1145/108844.108850 · Cited by 277
Research works printed but not linked: 43
- a9t9. 2018. Web Browser Automation with KantuX , Desktop Automation, OCR - Fresh 2017 Robotic Process Automation (RPA). (March 2018 ). https://a9t9.com/ a9t9. 2018. Web Browser Automation with KantuX, Desktop Automation, OCR - Fresh 2017 Robotic Process Automation (RPA). (March 2018). https://a9t9.com/
- 10.1145/276305.276330 · doi:10.1145/276305.276330
- 10.1108/IJM-02-2015-0029 · DOI
- 10.1145/2983990.2984020 · DOI
- 10.1109/TKDE.2006.152 · DOI · Paper page
- 10.1016/j.jvlc.2016.07.004 · doi:10.1016/j.jvlc.2016.07.004
- Sarah Chasins . 2017 . Helena: Web Automation for End Users . http://helena-lang.org/. (Oct. 2017). Sarah Chasins. 2017. Helena: Web Automation for End Users. http://helena-lang.org/. (Oct. 2017).
- 10.1145/2740908.2742849 · DOI
- 10.1145/3133875 · DOI
- Sergio Flesca , Giuseppe Manco , Elio Masciari , Eugenio Rende , and Andrea Tagarelli . 2004 . Web Wrapper Induction: A Brief Survey . AI Commun. 17 , 2 (April 2004), 57--61. http://dl.acm.org/citation.cfm?id=1218702.1218707 Sergio Flesca, Giuseppe Manco, Elio Masciari, Eugenio Rende, and Andrea Tagarelli. 2004. Web Wrapper Induction: A Brief Survey. AI Commun. 17, 2 (April 2004), 57--61. http://dl.acm.org/citation.cfm?id=1218702.1218707
- 10.1145/2882903.2915214 · doi:10.1145/2882903.2915214
- Greasemonkey. 2015. Greasemonkey :: Add-ons for Firefox. https://addons.mozilla.org/en-us/firefox/addon/greasemonkey/. (Nov . 2015 ). Greasemonkey. 2015. Greasemonkey :: Add-ons for Firefox. https://addons.mozilla.org/en-us/firefox/addon/greasemonkey/. (Nov. 2015).
- HaskellWiki. 2017. HXT - HaskellWiki . https://wiki.haskell.org/HXT#Selecting_text_from_an_HTML_document. (Nov . 2017 ). HaskellWiki. 2017. HXT - HaskellWiki. https://wiki.haskell.org/HXT#Selecting_text_from_an_HTML_document. (Nov. 2017).
- 10.5555/306766.306775 · doi:10.5555/306766.306775
- 10.1145/1294211.1294226 · doi:10.1145/1294211.1294226
- 10.1145/1166253.1166274 · doi:10.1145/1166253.1166274
- KimonoLabs. 2016 . Kimono: Turn websites into structured APIs from your browser in seconds. (March 2016). https://www.kimonolabs.com KimonoLabs. 2016. Kimono: Turn websites into structured APIs from your browser in seconds. (March 2016). https://www.kimonolabs.com
- 10.1145/1453101.1453119 · DOI
- 10.1016/S0004-3702(99)00100-9 · DOI · Paper page
- Nicholas Kushmerick , Daniel S. Weld , and Robert Doorenbos . 1997 . Wrapper Induction for Information Extraction . In Proc. IJCAI-97 . http://citeseer.nj.nec.com/kushmerick97wrapper.html Nicholas Kushmerick, Daniel S. Weld, and Robert Doorenbos. 1997. Wrapper Induction for Information Extraction. In Proc. IJCAI-97. http://citeseer.nj.nec.com/kushmerick97wrapper.html
- 10.1023/A:1025671410623 · DOI · Paper page
- Tessa A. Lau , Pedro Domingos , and Daniel S. Weld . 2000. Version Space Algebra and its Application to Programming by Demonstration . In ICML '00: Proceedings of the Seventeenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc. , San Francisco, CA, USA, 527--534. http://portal.acm.org/citation.cfm?id=657973 Tessa A. Lau, Pedro Domingos, and Daniel S. Weld. 2000. Version Space Algebra and its Application to Programming by Demonstration. In ICML '00: Proceedings of the Seventeenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 527--534. http://portal.acm.org/citation.cfm?id=657973
- 10.1145/2594291.2594333 · DOI · Paper page
- 10.1145/1357054.1357323 · DOI · Paper page
- 10.1145/1753326.1753432 · DOI · Paper page
- 10.1145/1502650.1502667 · DOI · Paper page
- 10.1145/1719970.1719994 · doi:10.1145/1719970.1719994
- 10.1080/17530350.2013.772070 · DOI · Paper page
- Mozenda. 2018. Web Scraping Solutions for Every Need. (March 2018 ). https://www.mozenda.com/?utm_source=googleadwords&utm_medium=cpc&utm_term=Mozenda&gclid=EAIaIQobChMIuM71jfGc2gIVC7nACh2mOwz8EAAYASAAEgLSzPD_BwE Mozenda. 2018. Web Scraping Solutions for Every Need. (March 2018). https://www.mozenda.com/?utm_source=googleadwords&utm_medium=cpc&utm_term=Mozenda&gclid=EAIaIQobChMIuM71jfGc2gIVC7nACh2mOwz8EAAYASAAEgLSzPD_BwE
- 10.1145/301136.301191 · DOI · Paper page
- 10.1145/301136.301191 · DOI · Paper page
- Nokogiri. 2016. Tutorials - Nokogiri . http://www.nokogiri.org/. (Nov . 2016 ). Nokogiri. 2016. Tutorials - Nokogiri. http://www.nokogiri.org/. (Nov. 2016).
- Donald A. Norman . 2002. The Design of Everyday Things . Basic Books, Inc. , New York, NY, USA . Donald A. Norman. 2002. The Design of Everyday Things. Basic Books, Inc., New York, NY, USA.
- Octoparse. 2018. Web Scraping Tool & Free Web Crawlers for Data Extraction | Octoparse. (March 2018 ). https://www.octoparse.com/ Octoparse. 2018. Web Scraping Tool & Free Web Crawlers for Data Extraction | Octoparse. (March 2018). https://www.octoparse.com/
- 10.1145/3018661.3018740 · doi:10.1145/3018661.3018740
- ParseHub. 2018. Free web scraping - Download the most powerful web scraper | ParseHub. (March 2018 ). https://www.parsehub.com/ ParseHub. 2018. Free web scraping - Download the most powerful web scraper | ParseHub. (March 2018). https://www.parsehub.com/
- Platypus. 2013. Platypus. http://platypus.mozdev.org/. (Nov . 2013 ). Platypus. 2013. Platypus. http://platypus.mozdev.org/. (Nov. 2013).
- Leonard Richardson . 2016 . Beautiful Soup: We called him Tortoise because he taught us . http://www.crummy.com/software/BeautifulSoup/. (March 2016). Leonard Richardson. 2016. Beautiful Soup: We called him Tortoise because he taught us. http://www.crummy.com/software/BeautifulSoup/. (March 2016).
- Scrapinghub. 2018. Visual scraping with Portia. (March 2018 ). https://scrapinghub.com/portia Scrapinghub. 2018. Visual scraping with Portia. (March 2018). https://scrapinghub.com/portia
- Scrapy. 2013. Scrapy. http://scrapy.org/. (July 2013 ). Scrapy. 2013. Scrapy. http://scrapy.org/. (July 2013).
- Howard T Welser , Marc Smith , Danyel Fisher , and Eric Gleave . 2008. Distilling digital traces: Computational social science approaches to studying the internet. Handbook of online research methods ( 2008 ), 116--140. Howard T Welser, Marc Smith, Danyel Fisher, and Eric Gleave. 2008. Distilling digital traces: Computational social science approaches to studying the internet. Handbook of online research methods (2008), 116--140.
- 10.1145/1240624.1240842 · DOI · Paper page
- 10.1145/1645953.1645962 · doi:10.1145/1645953.1645962
Cited by
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660
Grounded Copilot: How Programmers Interact with Code-Generating Models
Barke, S., James, M. B., & Polikarpova, N. (2023) · Proceedings of the ACM on Programming Languages · doi:10.1145/3586030 · Cited by 436
A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges
Liang, J. T., Yang, C., & Myers, B. A. (2023) · arXiv · doi:10.48550/arxiv.2303.17125 · Cited by 206
A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges
Liang, J. T., Yang, C., & Myers, B. A. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3608128 · Cited by 203
In-IDE Code Generation from Natural Language: Promise and Challenges
Xu, F. F., Vasilescu, B., & Neubig, G. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487569 · Cited by 154
Metadata
- Authors
- Chasins, S. E., Mueller, M., & Bodik, R.
- Venue
- Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology
- Publisher
- ACM
- Published
- 2018-10-11
- Pages
- 963-975
- OpenAlex
- W2896298055
- Cited by
- 85
- References
- 3
- FWCI
- 7.57
- Type
- conference paper
- Language
- en
- Open access
- closed
- Funders
- CONIX Research Center
- NSF (CCF--1139138 CCF--1337415 NSF ACI--1535191 and NSF 16-606)
- U.S. Department of Energy (FOA--0000619)
- DARPA (FA8750--14--C--0011 FA8750--16--2--0032)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Rousillon: Scraping Distributed Hierarchical Web Data. https://app.cochat.ai/papers/10.1145/3242587.3242661
- The paper
- Chasins, S. E., Mueller, M., & Bodik, R. (2018). Rousillon: Scraping Distributed Hierarchical Web Data. Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. https://doi.org/10.1145/3242587.3242661
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.