Cited by 74User Authentication and Security SystemsUsability and User Interface DesignAdvanced Authentication Protocols Security

A Tale of Two Studies: The Best and Worst of YubiKey Usability

Open access
Reynolds, Joshua, Smith, Trevor, Reese, Ken, Dickinson, Luke, Ruoti, Scott, Seamons, Kent·2018·2018 IEEE Symposium on Security and Privacy (SP)

Abstract

Two-factor authentication (2FA) significantly improves the security of password-based authentication. Recently, there has been increased interest in Universal 2nd Factor (U2F) security keys-small hardware devices that require users to press a button on the security key to authenticate. To examine the usability of security keys in non-enterprise usage, we conducted two user studies of the YubiKey, a popular line of U2F security keys. The first study tasked 31 participants with configuring a Windows, Google, and Facebook account to authenticate using a YubiKey. This study revealed problems with setup instructions and workflow including users locking themselves out of their operating system or thinking they had successfully enabled 2FA when they had not. In contrast, the second study had 25 participants use a YubiKey in their daily lives over a period of four weeks, revealing that participants generally enjoyed the experience. Conducting both a laboratory and longitudinal study yielded insights into the usability of security keys that would not have been evident from either study in isolation. Based on our analysis, we recommend standardizing the setup process, enabling verification of success, allowing shared accounts, integrating with operating systems, and preventing lockouts.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/sp.2018.00067

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The Yubico Security Key has already been the subject of studies in the past.

    “we focused on the Yubico Security Key as the authentication device because it is the most popular end-user security key on the market and has already been the subject of studies in the past [40], [26], [25].”

    From Is FIDO2 the Kingslayer of User Authentication? A Comparative Usability Study of FIDO2 Passwordless Authentication (2020) · cites this paper as [40]

    The Abstract and Introduction report two studies of Yubico YubiKeys, supporting the claim that Yubico security keys have been studied. The Introduction specifies the models as YubiKey 4, YubiKey NEO, and YubiKey Nano; it does not establish that the particular device called the “Yubico Security Key” was studied. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • The study focused on the Yubico Security Key as the authentication device.

    “we focused on the Yubico Security Key as the authentication device because it is the most popular end-user security key on the market and has already been the subject of studies in the past [40], [26], [25].”

    From Is FIDO2 the Kingslayer of User Authentication? A Comparative Usability Study of FIDO2 Passwordless Authentication (2020) · cites this paper as [40]

    The Introduction says the paper studied Yubico security keys, supporting the citation as an example of prior research on such devices. But the Methodology identifies the devices as a YubiKey 4, YubiKey NEO, and YubiKey Nano—not a device identified as the Yubico Security Key. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • Prior works encountered poor user experiences of Windows' support for security keys.

    “Moreover, prior works encountered poor user experiences of Windows' support for security keys [40], which we wanted to avoid in our study”
  • The open-ended questions about general impressions were inspired by closest related work and adapted using best-practices from commercial UX testing and literature.

    “Our open-ended questions about general impressions were inspired by closest related work [40] and adapted to our specific study setting using best-practices from commercial user experience testing [58] and literature [59] (e.g., recommendations for question form and wording).”
  • Prior work has identified the setup phase as problematic and recommended to study this phase separately.

    “Prior work has identified the setup phase as problematic [40] and recommended to study this phase separately.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 28 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. [9]9. S. Srinivas, D. Balfanz, E. Tiffany, and F. Alliance, "Universal 2nd Factor (U2F) Overview," FIDO Alliance Proposed Standard, pp. 1-5, 2015.
  2. [33]33. B. Dosono, J. Hayes, and Y. Wang, "'I'm stuck!': A Contextual Inquiry of People with Visual Impairments in Authentication." in Symposium On Usable Privacy and Security, 2015.

Cited by

Sorted by citations

Authors
Reynolds, J., Smith, T., Reese, K., Dickinson, L., Ruoti, S., & Seamons, K.
Affiliations
  • University of Illinois Urbana-Champaign (US)
  • Brigham Young University (US)
  • MIT Lincoln Laboratory (US)
Venue
2018 IEEE Symposium on Security and Privacy (SP)
Publisher
IEEE
Published
2018-05
Pages
872-888
ISSN
1081-6011 (ISSN-L)
OpenAlex
W2794591479
Cited by
73
References
28
Citations by year
2026: 5 · 2025: 10 · 2024: 4 · 2023: 11 · 2022: 7 · 2021: 12 · 2020: 14 · 2019: 9 · 2018: 1
FWCI
7.06
Type
conference paper
Language
en
Open access
Licence
other-oa
Locations
IEEE Symposium on Security and Privacy (S&P) (publishedVersion, other-oa)
Funders
  • National Science Foundation (1528022)
  • Sandia National Laboratories
Topics
  • Physical Sciences › Computer Science › Information Systems › User Authentication and Security Systems
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Authentication Protocols Security
Keywords
two-factor authentication, hardware authentication, account lockout, user experience, operating system integration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review