CoChat

Overview of the interactive task in BioCreative V

Open access
Wang, Qinghua, S. Abdul, Shabbir, Almeida, Lara, Ananiadou, Sophia, Balderas-Martínez, Yalbi I., Batista-Navarro, Riza, Campos, David, Chilton, Lucy, Chou, Hui-Jou, Contreras, Gabriela, Cooper, Laurel, Dai, Hong-Jie, Ferrell, Barbra, Fluck, Juliane, Gama-Castro, Socorro, George, Nancy, Gkoutos, Georgios, Irin, Afroza K., Jensen, Lars J., Jimenez, Silvia, Jue, Toni R., Keseler, Ingrid, Madan, Sumit, Matos, Sérgio, McQuilton, Peter, Milacic, Marija, Mort, Matthew, Natarajan, Jeyakumar, Pafilis, Evangelos, Pereira, Emiliano, Rao, Shruti, Rinaldi, Fabio, Rothfels, Karen, Salgado, David, Silva, Raquel M., Singh, Onkar, Stefancsik, Raymund, Su, Chu-Hsien, Subramani, Suresh, Tadepally, Hamsa D., Tsaprouni, Loukia, Vasilevsky, Nicole, Wang, Xiaodong, Chatr-Aryamontri, Andrew, Laulederkind, Stanley J. F., Matis-Mitchell, Sherri, McEntyre, Johanna, Orchard, Sandra, Pundir, Sangya, Rodriguez-Esteban, Raul, Van Auken, Kimberly, Lu, Zhiyong, Schaeffer, Mary, Wu, Cathy H., Hirschman, Lynette, Arighi, Cecilia N. · 2016 · Database
Cited by 97

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

Fully automated text mining (TM) systems promote efficient literature searching, retrieval, and review but are not sufficient to produce ready-to-consume curated documents. These systems are not meant to replace biocurators, but instead to assist them in one or more literature curation steps. To do so, the user interface is an important aspect that needs to be considered for tool adoption. The BioCreative Interactive task (IAT) is a track designed for exploring user-system interactions, promoting development of useful TM tools, and providing a communication channel between the biocuration and the TM communities. In BioCreative V, the IAT track followed a format similar to previous interactive tracks, where the utility and usability of TM tools, as well as the generation of use cases, have been the focal points. The proposed curation tasks are user-centric and formally evaluated by biocurators. In BioCreative V IAT, seven TM systems and 43 biocurators participated. Two levels of user participation were offered to broaden curator involvement and obtain more feedback on usability aspects. The full level participation involved training on the system, curation of a set of documents with and without TM assistance, tracking of time-on-task, and completion of a user survey. The partial level participation was designed to focus on usability aspects of the interface and not the performance per se In this case, biocurators navigated the system by performing pre-designed tasks and then were asked whether they were able to achieve the task and the level of difficulty in completing the task. In this manuscript, we describe the development of the interactive task, from planning to execution and discuss major findings for the systems tested.Database URL: http://www.biocreative.org.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/database/baw119

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 3 references

Cited by

Metadata

Authors
Wang, Q., S. Abdul, S., Almeida, L., Ananiadou, S., Balderas-Martínez, Y. I., Batista-Navarro, R., Campos, D., Chilton, L., Chou, H.-J., Contreras, G., Cooper, L., Dai, H.-J., Ferrell, B., Fluck, J., Gama-Castro, S., George, N., Gkoutos, G., Irin, A. K., Jensen, L. J., Jimenez, S., Jue, T. R., Keseler, I., Madan, S., Matos, S., McQuilton, P., Milacic, M., Mort, M., Natarajan, J., Pafilis, E., Pereira, E., Rao, S., Rinaldi, F., Rothfels, K., Salgado, D., Silva, R. M., Singh, O., Stefancsik, R., Su, C.-H., Subramani, S., Tadepally, H. D., Tsaprouni, L., Vasilevsky, N., Wang, X., Chatr-Aryamontri, A., Laulederkind, S. J. F., Matis-Mitchell, S., McEntyre, J., Orchard, S., Pundir, S., Rodriguez-Esteban, R., Van Auken, K., Lu, Z., Schaeffer, M., Wu, C. H., Hirschman, L., & Arighi, C. N.
Venue
Database
Published
2016
Volume
2016
Pages
baw119
ISSN
1758-0463 (ISSN-L)
OpenAlex
W2516150684
Cited by
97
References
3
FWCI
9.63
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.