CoChat

Source evaluation of domain experts and novices during Web search

Open access
Brand‐Gruwel, S., Kammerer, Y., van Meeuwen, L., van Gog, T. · 2017 · Journal of Computer Assisted Learning
Cited by 115

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Abstract

Nowadays, almost everyone uses the World Wide Web (WWW) to search for information of any kind. In education, students frequently use the WWW for selecting information to accomplish assignments such as writing an essay or preparing a presentation. The evaluation of sources and information is an important sub‐skill in this process. But many students have not yet optimally developed this skill. On the basis of verbal reports, eye‐tracking data and navigation logs, this study investigated how novices in the domain of psychology evaluate Internet sources as compared to domain experts. In addition, two different verbal reporting techniques, namely thinking aloud and cued retrospective reporting, were compared in order to examine students' evaluation behaviour. Results revealed that domain expertise has an impact on individuals' evaluation behaviour during Web search, such that domain experts showed a more sophisticated use of evaluation criteria to judge the reliability of sources and information and selected more reliable information than domain novices. Furthermore, the different verbal reporting techniques did not lead to different conclusions on criteria use in relation to domain expertise, although in general more utterances concerning evaluation of sources and information were expressed during cued retrospective reporting. Lay Description: What is already known about this topic: When searching the Web, students assess the reliability of information and sources in a spontaneous manner. Students tend to use superficial criteria when judging the trustworthiness of web‐based information and sources. Little is known about the differences between domain experts and novices in terms of the ways they spontaneously evaluate web‐based information and sources. What this paper adds: Domain knowledge influences evaluation behaviour, with greater domain knowledge leading to more reliable selection of sources. A comparison of techniques for measuring evaluation behaviour when searching the Web for information reveals that different techniques, such as thinking aloud and cued retrospective reporting, have different pros and cons whose implications depend on the research questions being addressed. Implications for practice and/or policy: In formal education, more attention should be paid to equipping students with the requisite skills in searching for and evaluating web‐based information. Embedded process‐oriented instructional designs are needed that foster the appropriate evaluation behaviours.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/jcal.12162

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 115

Metadata

Authors
Brand‐Gruwel, S., Kammerer, Y., van Meeuwen, L., & van Gog, T.
Venue
Journal of Computer Assisted Learning
Publisher
Wiley
Published
2017-01-05
Volume
33
Issue
3
Pages
234-251
ISSN
  • 0266-4909 (ISSN-L)
  • 1365-2729
OpenAlex
W2590842578
Cited by
115
References
11
FWCI
18.99
Type
journal article
Language
en
Open access
bronze

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.