CoChat

The Dialog State Tracking Challenge Series: A Review

Open access
Williams, Jason D., Raux, Antoine, Henderson, Matthew · 2016 · Dialogue & Discourse
Cited by 192

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Abstract

In a spoken dialog system, dialog state tracking refers to the task of correctly inferring the state of the conversation -- such as the user's goal -- given all of the dialog history up to that turn. Dialog state tracking is crucial to the success of a dialog system, yet until recently there were no common resources, hampering progress. The Dialog State Tracking Challenge series of 3 tasks introduced the first shared testbed and evaluation metrics for dialog state tracking, and has underpinned three key advances in dialog state tracking: the move from generative to discriminative models; the adoption of discriminative sequential techniques; and the incorporation of the speech recognition results directly into the dialog state tracker. This paper reviews this research area, covering both the challenge tasks themselves and summarizing the work they have enabled.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.5087/dad.2016.301

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Models for multiparty engagement in open-world dialog

    Bohus, D., & Horvitz, E. (2009) · Proceedings of the SIGDIAL 2009 Conference on The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue - SIGDIAL '09 · doi:10.3115/1708376.1708409 · Cited by 145 · cited 1×

    “Dialog state tracking in situated environments – for example, robots or embodied agents – is also out of scope for this review, but it is noted that dialog state tracking is also used in this setting (Bohus and Horvitz, 2009; Ma et al., 2012).”
    Background
  • Conversation as Action Under Uncertainty

    Paek, T., & Horvitz, E. J. (2013) · arXiv · doi:10.48550/arxiv.1301.3883 · Cited by 145 · cited 1×

    “…in detail in Section 3; illustrative examples include hand-crafted rules (Larsson and Traum, 2000; Bohus and Rudnicky, 2003), heuristic scores (Higashinaka et al., 2003), Bayesian networks (Paek and Horvitz, 2000; Williams and Young, 2007), and discriminative models (Bohus and Rud-nicky, 2006).”
    Background

Cited by

See all 192

Metadata

Authors
Williams, J. D., Raux, A., & Henderson, M.
Venue
Dialogue & Discourse
Published
2016-04-15
Volume
7
Issue
3
Pages
4-33
ISSN
2152-9620 (ISSN-L)
OpenAlex
W2468710617
Cited by
192
References
2
FWCI
20.19
Type
journal article
Language
en
Open access
diamond

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.