CoChat

Rousillon: Scraping Distributed Hierarchical Web Data

Chasins, Sarah E., Mueller, Maria, Bodik, Rastislav · 2018 · Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology
Cited by 85

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

Programming by Demonstration (PBD) promises to enable data scientists to collect web data. However, in formative interviews with social scientists, we learned that current PBD tools are insufficient for many real-world web scraping tasks. The missing piece is the capability to collect hierarchically-structured data from across many different webpages. We present Rousillon, a programming system for writing complex web automation scripts by demonstration. Users demonstrate how to collect the first row of a 'universal table' view of a hierarchical dataset to teach Rousillon how to collect all rows. To offer this new demonstration model, we developed novel relation selection and generalization algorithms. In a within-subject user study on 15 computer scientists, users can write hierarchical web scrapers 8 times more quickly with Rousillon than with traditional programming.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3242587.3242661

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 3 references

Research works printed but not linked: 43

  1. a9t9. 2018. Web Browser Automation with KantuX , Desktop Automation, OCR - Fresh 2017 Robotic Process Automation (RPA). (March 2018 ). https://a9t9.com/ a9t9. 2018. Web Browser Automation with KantuX, Desktop Automation, OCR - Fresh 2017 Robotic Process Automation (RPA). (March 2018). https://a9t9.com/
  2. 10.1145/276305.276330 · doi:10.1145/276305.276330
  3. 10.1108/IJM-02-2015-0029 · DOI
  4. 10.1145/2983990.2984020 · DOI
  5. 10.1109/TKDE.2006.152 · DOI · Paper page
  6. 10.1016/j.jvlc.2016.07.004 · doi:10.1016/j.jvlc.2016.07.004
  7. Sarah Chasins . 2017 . Helena: Web Automation for End Users . http://helena-lang.org/. (Oct. 2017). Sarah Chasins. 2017. Helena: Web Automation for End Users. http://helena-lang.org/. (Oct. 2017).
  8. 10.1145/2740908.2742849 · DOI
  9. 10.1145/3133875 · DOI
  10. Sergio Flesca , Giuseppe Manco , Elio Masciari , Eugenio Rende , and Andrea Tagarelli . 2004 . Web Wrapper Induction: A Brief Survey . AI Commun. 17 , 2 (April 2004), 57--61. http://dl.acm.org/citation.cfm?id=1218702.1218707 Sergio Flesca, Giuseppe Manco, Elio Masciari, Eugenio Rende, and Andrea Tagarelli. 2004. Web Wrapper Induction: A Brief Survey. AI Commun. 17, 2 (April 2004), 57--61. http://dl.acm.org/citation.cfm?id=1218702.1218707
  11. 10.1145/2882903.2915214 · doi:10.1145/2882903.2915214
  12. Greasemonkey. 2015. Greasemonkey :: Add-ons for Firefox. https://addons.mozilla.org/en-us/firefox/addon/greasemonkey/. (Nov . 2015 ). Greasemonkey. 2015. Greasemonkey :: Add-ons for Firefox. https://addons.mozilla.org/en-us/firefox/addon/greasemonkey/. (Nov. 2015).
  13. HaskellWiki. 2017. HXT - HaskellWiki . https://wiki.haskell.org/HXT#Selecting_text_from_an_HTML_document. (Nov . 2017 ). HaskellWiki. 2017. HXT - HaskellWiki. https://wiki.haskell.org/HXT#Selecting_text_from_an_HTML_document. (Nov. 2017).
  14. 10.5555/306766.306775 · doi:10.5555/306766.306775
  15. 10.1145/1294211.1294226 · doi:10.1145/1294211.1294226
  16. 10.1145/1166253.1166274 · doi:10.1145/1166253.1166274
  17. KimonoLabs. 2016 . Kimono: Turn websites into structured APIs from your browser in seconds. (March 2016). https://www.kimonolabs.com KimonoLabs. 2016. Kimono: Turn websites into structured APIs from your browser in seconds. (March 2016). https://www.kimonolabs.com
  18. 10.1145/1453101.1453119 · DOI
  19. 10.1016/S0004-3702(99)00100-9 · DOI · Paper page
  20. Nicholas Kushmerick , Daniel S. Weld , and Robert Doorenbos . 1997 . Wrapper Induction for Information Extraction . In Proc. IJCAI-97 . http://citeseer.nj.nec.com/kushmerick97wrapper.html Nicholas Kushmerick, Daniel S. Weld, and Robert Doorenbos. 1997. Wrapper Induction for Information Extraction. In Proc. IJCAI-97. http://citeseer.nj.nec.com/kushmerick97wrapper.html
  21. 10.1023/A:1025671410623 · DOI · Paper page
  22. Tessa A. Lau , Pedro Domingos , and Daniel S. Weld . 2000. Version Space Algebra and its Application to Programming by Demonstration . In ICML '00: Proceedings of the Seventeenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc. , San Francisco, CA, USA, 527--534. http://portal.acm.org/citation.cfm?id=657973 Tessa A. Lau, Pedro Domingos, and Daniel S. Weld. 2000. Version Space Algebra and its Application to Programming by Demonstration. In ICML '00: Proceedings of the Seventeenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 527--534. http://portal.acm.org/citation.cfm?id=657973
  23. 10.1145/2594291.2594333 · DOI · Paper page
  24. 10.1145/1357054.1357323 · DOI · Paper page
  25. 10.1145/1753326.1753432 · DOI · Paper page
  26. 10.1145/1502650.1502667 · DOI · Paper page
  27. 10.1145/1719970.1719994 · doi:10.1145/1719970.1719994
  28. 10.1080/17530350.2013.772070 · DOI · Paper page
  29. Mozenda. 2018. Web Scraping Solutions for Every Need. (March 2018 ). https://www.mozenda.com/?utm_source=googleadwords&utm_medium=cpc&utm_term=Mozenda&gclid=EAIaIQobChMIuM71jfGc2gIVC7nACh2mOwz8EAAYASAAEgLSzPD_BwE Mozenda. 2018. Web Scraping Solutions for Every Need. (March 2018). https://www.mozenda.com/?utm_source=googleadwords&utm_medium=cpc&utm_term=Mozenda&gclid=EAIaIQobChMIuM71jfGc2gIVC7nACh2mOwz8EAAYASAAEgLSzPD_BwE
  30. 10.1145/301136.301191 · DOI · Paper page
  31. 10.1145/301136.301191 · DOI · Paper page
  32. Nokogiri. 2016. Tutorials - Nokogiri . http://www.nokogiri.org/. (Nov . 2016 ). Nokogiri. 2016. Tutorials - Nokogiri. http://www.nokogiri.org/. (Nov. 2016).
  33. Donald A. Norman . 2002. The Design of Everyday Things . Basic Books, Inc. , New York, NY, USA . Donald A. Norman. 2002. The Design of Everyday Things. Basic Books, Inc., New York, NY, USA.
  34. Octoparse. 2018. Web Scraping Tool & Free Web Crawlers for Data Extraction | Octoparse. (March 2018 ). https://www.octoparse.com/ Octoparse. 2018. Web Scraping Tool & Free Web Crawlers for Data Extraction | Octoparse. (March 2018). https://www.octoparse.com/
  35. 10.1145/3018661.3018740 · doi:10.1145/3018661.3018740
  36. ParseHub. 2018. Free web scraping - Download the most powerful web scraper | ParseHub. (March 2018 ). https://www.parsehub.com/ ParseHub. 2018. Free web scraping - Download the most powerful web scraper | ParseHub. (March 2018). https://www.parsehub.com/
  37. Platypus. 2013. Platypus. http://platypus.mozdev.org/. (Nov . 2013 ). Platypus. 2013. Platypus. http://platypus.mozdev.org/. (Nov. 2013).
  38. Leonard Richardson . 2016 . Beautiful Soup: We called him Tortoise because he taught us . http://www.crummy.com/software/BeautifulSoup/. (March 2016). Leonard Richardson. 2016. Beautiful Soup: We called him Tortoise because he taught us. http://www.crummy.com/software/BeautifulSoup/. (March 2016).
  39. Scrapinghub. 2018. Visual scraping with Portia. (March 2018 ). https://scrapinghub.com/portia Scrapinghub. 2018. Visual scraping with Portia. (March 2018). https://scrapinghub.com/portia
  40. Scrapy. 2013. Scrapy. http://scrapy.org/. (July 2013 ). Scrapy. 2013. Scrapy. http://scrapy.org/. (July 2013).
  41. Howard T Welser , Marc Smith , Danyel Fisher , and Eric Gleave . 2008. Distilling digital traces: Computational social science approaches to studying the internet. Handbook of online research methods ( 2008 ), 116--140. Howard T Welser, Marc Smith, Danyel Fisher, and Eric Gleave. 2008. Distilling digital traces: Computational social science approaches to studying the internet. Handbook of online research methods (2008), 116--140.
  42. 10.1145/1240624.1240842 · DOI · Paper page
  43. 10.1145/1645953.1645962 · doi:10.1145/1645953.1645962

Cited by

See all 85

Metadata

Authors
Chasins, S. E., Mueller, M., & Bodik, R.
Venue
Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology
Publisher
ACM
Published
2018-10-11
Pages
963-975
OpenAlex
W2896298055
Cited by
85
References
3
FWCI
7.57
Type
conference paper
Language
en
Open access
closed
Funders
  • CONIX Research Center
  • NSF (CCF--1139138 CCF--1337415 NSF ACI--1535191 and NSF 16-606)
  • U.S. Department of Energy (FOA--0000619)
  • DARPA (FA8750--14--C--0011 FA8750--16--2--0032)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Rousillon: Scraping Distributed Hierarchical Web Data. https://app.cochat.ai/papers/10.1145/3242587.3242661
The paper
Chasins, S. E., Mueller, M., & Bodik, R. (2018). Rousillon: Scraping Distributed Hierarchical Web Data. Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. https://doi.org/10.1145/3242587.3242661

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.