CoChat

JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs

Open access
Koga, Yurie, Kando, Shunsuke, Miyao, Yusuke · 2026 · Findings of the Association for Computational Linguistics: EMNLP 2026

The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index; none of the 14 sources its claims cite is checked for support yet.

Abstract

Self-supervised speech models are known to encode phonological information, yet it remains underexplored whether they capture phonological properties characteristic of Japanese, such as mora and pitch accent.This paper introduces JPhonABX, a Japanese phonological discrimination dataset targeting moraic and accentual contrasts.It consists of 192,000 moraic and 98,000 accentual ABX test items, covering various types of contrasts such as those involving moraic nasal, geminate consonant, vowel length, and pitch accent.Our human evaluation by native Japanese speakers shows high discrimination accuracy and inter-rater agreement, validating the quality of the dataset.We further evaluate several self-supervised speech models and find that while almost all of them show no significant difference from humans in mora discrimination, they perform significantly worse in accent discrimination, and even models trained on Japanese fail to reach humanlevel performance.

Overview

CoChat Summary

TL;DR

JPhonABX is a large-scale Japanese ABX dataset for moraic and accentual minimal pairs, built from the JVS corpus with 192,000 mora and 98,000 accent test items. Human evaluation validates the items, while self-supervised speech models match humans on mora discrimination but fall significantly below humans on pitch accent discrimination.

Key findings

  • The dataset contains 192,000 mora discrimination ABX items and 98,000 accent discrimination ABX items, covering moraic nasal, geminate consonant, vowel length, and pitch accent contrasts.
  • Human evaluation on 500 mora and 500 accent items found overall accuracy 0.862 for mora discrimination and 0.873 for accent discrimination, with Krippendorff’s α of 0.665 and 0.721, respectively.
  • In model evaluation, 13 out of 14 S3Ms showed no significant difference from humans on mora discrimination, but all S3Ms performed significantly worse than humans on accent discrimination, including Japanese-trained models.
  • Additional WavLM models reached peak mora accuracy of 92.9%, 92.9%, and 94.1%, and accent accuracy of 76.2%, 74.1%, and 76.6%, consistent with the main asymmetry.

Methods

  • The ABX task presents stimuli A, B, and X; A and B differ only in mora sequence or pitch accent pattern, and the model/human selects whether X is closer to A or B. Model discrimination uses DTW with cosine similarity between representations.
  • Audio was extracted from the parallel100 subset of the JVS corpus: 22 hours of clean recordings, 100 sentences read by 100 speakers in Standard Japanese. Accent labels were automatically assigned and validated with F0 ratios from CREPE; 276 files with failed alignment were removed.
  • Human evaluation used 1,000 randomly sampled triplets (500 mora + 500 accent) with three native Japanese speakers per task. Model evaluation used six Japanese, five English, and three multilingual HuBERT/wav2vec 2.0 models, plus MFCC and mel-spectrogram baselines, in with-context and without-context settings; statistical significance used chi-squared tests with Yates’ correction or Fisher’s exact test.

Limitations

  • The paper states that human evaluation involved only three participants per task, which may be insufficient for reliable estimation of human performance.
  • The paper states that stimuli were extracted automatically using text-speech alignment from a speech recognition model, may contain mora boundary misalignments, and were not manually verified; low human accuracy and agreement on vowel length suggest some stimuli may be insufficient quality.
  • The paper states that the benchmark uses clean read speech in Standard Japanese and may not reflect noisy, dialectal, or spontaneous speech; it also notes limited S3M coverage and that without-context and human evaluations used only 500 items per task, limiting fine-grained analyses.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.18653/2026.findings-emnlp.1072

Cites

70 of 70 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Overview of the mora discrimination (left) and pitch accent discrimination (right) ABX task used in the JPhonABX dataset. The task is to select the closer stimuli to X from A and B. For mora discrimination, the morae 'ma' and 'ta' are discriminated. For pitch accent discrimination, the accent patterns 'HL' and 'LH' are discriminated.
Figure 1 Overview of the mora discrimination (left) and pitch accent discrimination (right) ABX task used in the JPhonABX dataset. The task is to select the closer stimuli to X from A and B. For mora discrimination, the morae 'ma' and 'ta' are discriminated. For pitch accent discrimination, the accent patterns 'HL' and 'LH' are discriminated.CC-BY-NC-SA · From the PDF
Figure 2 Mora discrimination accuracy in the withcontext (n=192,000) setting. Figures in brackets show training data size (h).
Figure 2 Mora discrimination accuracy in the withcontext (n=192,000) setting. Figures in brackets show training data size (h).CC-BY-NC-SA · From the PDF
Figure 3 Comparison of mora discrimination accuracy between the six contrast types. (a) Results in the withcontext (n=192,000) setting. See Appendix D for full results. (b) Peak mora discrimination accuracy in the without-context (n=421) setting averaged within each model group. Error bars indicate standard deviation across models.
Figure 3 Comparison of mora discrimination accuracy between the six contrast types. (a) Results in the withcontext (n=192,000) setting. See Appendix D for full results. (b) Peak mora discrimination accuracy in the without-context (n=421) setting averaged within each model group. Error bars indicate standard deviation across models.CC-BY-NC-SA · From the PDF
Figure 4 Comparison of mora discrimination accuracy between within- and across- (a) speaker, (b) gender, and (c) accent conditions in the with-context (n=192,000) setting. In (a), gender condition is within and in (b), speaker condition is across. Accuracy is averaged within each model group. Error bars indicate standard deviation across models.
Figure 4 Comparison of mora discrimination accuracy between within- and across- (a) speaker, (b) gender, and (c) accent conditions in the with-context (n=192,000) setting. In (a), gender condition is within and in (b), speaker condition is across. Accuracy is averaged within each model group. Error bars indicate standard deviation across models.CC-BY-NC-SA · From the PDF
Figure 5 Accent discrimination accuracy in the withcontext (n=98,000) setting. Figures in brackets show training data size (h).
Figure 5 Accent discrimination accuracy in the withcontext (n=98,000) setting. Figures in brackets show training data size (h).CC-BY-NC-SA · From the PDF
Figure 6 Comparison between word- and nonwordaccent contrasts. Word-accent contrasts are those between two pitch accent patterns that are both possible as word accent patterns in Standard Japanese (i.e., HL, LH, HLL, LHH, LHL), and nonword-accent contrasts are those involving at least one impossible pattern. (a) Results in the with-context (n=4,337 (word) + 93,663 (nonword)) setting. See Appendix D for full results. (b) Peak ABX accuracy in the without-context setting averaged within each model group. Error bars indicate standard deviation across models.
Figure 6 Comparison between word- and nonwordaccent contrasts. Word-accent contrasts are those between two pitch accent patterns that are both possible as word accent patterns in Standard Japanese (i.e., HL, LH, HLL, LHH, LHL), and nonword-accent contrasts are those involving at least one impossible pattern. (a) Results in the with-context (n=4,337 (word) + 93,663 (nonword)) setting. See Appendix D for full results. (b) Peak ABX accuracy in the without-context setting averaged within each model group. Error bars indicate standard deviation across models.CC-BY-NC-SA · From the PDF
Figure 7 Comparison of accent discrimination accuracy between within- and across- (a) speaker and (b) gender conditions in the with-context (n=98,000) setting. In (a), gender condition is within and in (b), speaker condition is across. Accuracy is averaged within each model group. Error bars indicate standard deviation across models.
Figure 7 Comparison of accent discrimination accuracy between within- and across- (a) speaker and (b) gender conditions in the with-context (n=98,000) setting. In (a), gender condition is within and in (b), speaker condition is across. Accuracy is averaged within each model group. Error bars indicate standard deviation across models.CC-BY-NC-SA · From the PDF
Figure 9 Layer-wise ABX accuracy for mora discrimination in the without-context (n=421) setting.
Figure 9 Layer-wise ABX accuracy for mora discrimination in the without-context (n=421) setting.CC-BY-NC-SA · From the PDF
Figure 10 Layer-wise ABX accuracy for accent discrimination in the without-context (n=495) settings.
Figure 10 Layer-wise ABX accuracy for accent discrimination in the without-context (n=495) settings.CC-BY-NC-SA · From the PDF
Image 10
CC-BY-NC-SA · From the PDF
Figure 11 Layer-wise ABX accuracy of each contrast type for mora discrimination in the (left) with-context (n=192,000) and (right) without-context (n=421) settings.
Figure 11 Layer-wise ABX accuracy of each contrast type for mora discrimination in the (left) with-context (n=192,000) and (right) without-context (n=421) settings.CC-BY-NC-SA · From the PDF
Figure 12 Layer-wise ABX accuracy comparison for accent discrimination between word- and nonword-accent contrasts. Word-accent contrasts are those between two pitch accent patterns that are both possible as word accent patterns in Standard Japanese (i.e., HL, LH, HLL, LHH, LHL), and nonword-accent contrasts are those involving at least one impossible pattern. The left panel shows results in the with-context (n=4,337 (word) + 93,663 (nonword)) setting and the right panel shows results in the without-context (n=22 (word) + 473 (nonword)) setting.
Figure 12 Layer-wise ABX accuracy comparison for accent discrimination between word- and nonword-accent contrasts. Word-accent contrasts are those between two pitch accent patterns that are both possible as word accent patterns in Standard Japanese (i.e., HL, LH, HLL, LHH, LHL), and nonword-accent contrasts are those involving at least one impossible pattern. The left panel shows results in the with-context (n=4,337 (word) + 93,663 (nonword)) setting and the right panel shows results in the without-context (n=22 (word) + 473 (nonword)) setting.CC-BY-NC-SA · From the PDF
Image 13
CC-BY-NC-SA · From the PDF
Image 14
CC-BY-NC-SA · From the PDF

Verification

Claims the paper makes

0 Valid · 2 Not valid · 6 Not checked · 89 Own results and interpretations

51 more claims

Create a free account to see all 97 claims and to verify the 11 references still open.

See all claims — free account
  • It may state two separate claims as one. 30
  • Probably not a substantial claim. 14

Create a free account to see these claims.

Citations

Cites

Show all 70 references

Research works printed but not linked: 2

  1. Marco Fonseca. 2023. THE PERCEPTION OF JAPANESE LEXICAL CONTRASTS BY L2 LEARNERS: THE ROLE OF THE LEXICON. Pro- ceedings of the 20th International Congress of Pho- netic Sciences, pages 2606-2611.
  2. Aisikaer Rouzi, Yin Shi, Zhang Zhiyong, Wang Dong, Askar Hamdulla, and ZHENG Fang. 2017. THUYG- 20: A free Uyghur speech database. Journal of Tsinghua University (Science and Technology), 57(2):182-187.

Cited by

Still being checked.

Metadata

Authors
Koga, Y., Kando, S., & Miyao, Y.
Affiliations
The University of Tokyo (JP)
Venue
Findings of the Association for Computational Linguistics: EMNLP 2026
Published
2026
Pages
22676-22695
OpenAlex
W7224491108
Cited by
0
References
69
FWCI
0.00
Type
conference paper
Language
en
Open access
Licence
cc-by-nc-sa
Locations
Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by-nc-sa)
Funders
  • Japan Society for the Promotion of Science
  • ACT-X
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Keywords
Japanese phonology, mora, pitch accent, phonological discrimination, minimal pairs, self-supervised speech models, geminate consonants, vowel duration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-20, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-20 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs. https://app.cochat.ai/papers/10.18653/2026.findings-emnlp.1072
The paper
Koga, Y., Kando, S., & Miyao, Y. (2026). JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs. Findings of the Association for Computational Linguistics: EMNLP 2026. https://doi.org/10.18653/2026.findings-emnlp.1072

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.