JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs
Open accessThe DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index; none of the 14 sources its claims cite is checked for support yet.
Abstract
Self-supervised speech models are known to encode phonological information, yet it remains underexplored whether they capture phonological properties characteristic of Japanese, such as mora and pitch accent.This paper introduces JPhonABX, a Japanese phonological discrimination dataset targeting moraic and accentual contrasts.It consists of 192,000 moraic and 98,000 accentual ABX test items, covering various types of contrasts such as those involving moraic nasal, geminate consonant, vowel length, and pitch accent.Our human evaluation by native Japanese speakers shows high discrimination accuracy and inter-rater agreement, validating the quality of the dataset.We further evaluate several self-supervised speech models and find that while almost all of them show no significant difference from humans in mora discrimination, they perform significantly worse in accent discrimination, and even models trained on Japanese fail to reach humanlevel performance.
Overview
CoChat Summary
TL;DR
JPhonABX is a large-scale Japanese ABX dataset for moraic and accentual minimal pairs, built from the JVS corpus with 192,000 mora and 98,000 accent test items. Human evaluation validates the items, while self-supervised speech models match humans on mora discrimination but fall significantly below humans on pitch accent discrimination.
Key findings
- The dataset contains 192,000 mora discrimination ABX items and 98,000 accent discrimination ABX items, covering moraic nasal, geminate consonant, vowel length, and pitch accent contrasts.
- Human evaluation on 500 mora and 500 accent items found overall accuracy 0.862 for mora discrimination and 0.873 for accent discrimination, with Krippendorff’s α of 0.665 and 0.721, respectively.
- In model evaluation, 13 out of 14 S3Ms showed no significant difference from humans on mora discrimination, but all S3Ms performed significantly worse than humans on accent discrimination, including Japanese-trained models.
- Additional WavLM models reached peak mora accuracy of 92.9%, 92.9%, and 94.1%, and accent accuracy of 76.2%, 74.1%, and 76.6%, consistent with the main asymmetry.
Methods
- The ABX task presents stimuli A, B, and X; A and B differ only in mora sequence or pitch accent pattern, and the model/human selects whether X is closer to A or B. Model discrimination uses DTW with cosine similarity between representations.
- Audio was extracted from the parallel100 subset of the JVS corpus: 22 hours of clean recordings, 100 sentences read by 100 speakers in Standard Japanese. Accent labels were automatically assigned and validated with F0 ratios from CREPE; 276 files with failed alignment were removed.
- Human evaluation used 1,000 randomly sampled triplets (500 mora + 500 accent) with three native Japanese speakers per task. Model evaluation used six Japanese, five English, and three multilingual HuBERT/wav2vec 2.0 models, plus MFCC and mel-spectrogram baselines, in with-context and without-context settings; statistical significance used chi-squared tests with Yates’ correction or Fisher’s exact test.
Limitations
- The paper states that human evaluation involved only three participants per task, which may be insufficient for reliable estimation of human performance.
- The paper states that stimuli were extracted automatically using text-speech alignment from a speech recognition model, may contain mora boundary misalignments, and were not manually verified; low human accuracy and agreement on vowel length suggest some stimuli may be insufficient quality.
- The paper states that the benchmark uses clean read speech in Standard Japanese and may not reflect noisy, dialectal, or spontaneous speech; it also notes limited S3M coverage and that without-context and human evaluations used only 500 items per task, limiting fine-grained analyses.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures














Verification
Claims the paper makes
0 Valid · 2 Not valid · 6 Not checked · 89 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Across-speaker accuracy significantly exceeds within-speaker accuracy in 98.8% of layers”
§ JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs > 5.3.2 Difference between Contrast Types
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For accent discrimination, however, S3M accuracy was markedly lower than in mora discrimination, particularly for models not trained on Japanese data.”
§ JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs > 6 Conclusion
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In terms of inter-rater agreement, most contrast types exhibit α values above 0.6, indicating reasonable reliability of the dataset.”
§ JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs > 4 Human Evaluation
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition, the accuracy of HuBERT models peaks at earlier layers than in mora discrimination.”
§ JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs > 5.3.1 Overview
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For accent discrimination, however, most models achieve lower peak accuracy in the without-context setting.”
§ JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs > D.1 Overall Results in the Without-Context Setting
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
51 more claims
Create a free account to see all 97 claims and to verify the 11 references still open.
- It may state two separate claims as one. 30
- Probably not a substantial claim. 14
Citations
Cites
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, Y., Mohamed, A., & Auli, M. (2020) · neural information processing systems · Cited by 5,822
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926
Common Voice: A Massively-Multilingual Speech Corpus
Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F. M., & Weber, G. (2019) · arXiv · doi:10.48550/arxiv.1912.06670 · Cited by 905
AISHELL-1: An open-source Mandarin speech corpus and a speech recognition baseline
Bu, H., Du, J., Na, X., Wu, B., & Zheng, H. (2017) · 2017 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment (O-COCOSDA) · doi:10.1109/icsda.2017.8384449 · Cited by 791
SUPERB: Speech Processing Universal PERformance Benchmark
Yang, S.-W., Chi, P.-H., Chuang, Y.-S., Lai, C.-I. J., Lakhotia, K., Lin, Y. Y., Liu, A. T., Shi, J., Chang, X., Lin, G.-T., et al. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1775 · Cited by 681
Unsupervised Cross-Lingual Representation Learning for Speech Recognition
Conneau, A., Baevski, A., Collobert, R., Mohamed, A., & Auli, M. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-329 · Cited by 637
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
Babu, A., Wang, C., Tjandra, A., Lakhotia, K., Xu, Q., Goyal, N., Singh, K., von Platen, P., Saraf, Y., Pino, J., et al. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-143 · Cited by 586
Corpus of Spontaneous Japanese : Its design and evaluation
Maekawa, K. (2003) · Medical Entomology and Zoology · Cited by 479
Research works printed but not linked: 2
- Marco Fonseca. 2023. THE PERCEPTION OF JAPANESE LEXICAL CONTRASTS BY L2 LEARNERS: THE ROLE OF THE LEXICON. Pro- ceedings of the 20th International Congress of Pho- netic Sciences, pages 2606-2611.
- Aisikaer Rouzi, Yin Shi, Zhang Zhiyong, Wang Dong, Askar Hamdulla, and ZHENG Fang. 2017. THUYG- 20: A free Uyghur speech database. Journal of Tsinghua University (Science and Technology), 57(2):182-187.
Cited by
Still being checked.
Metadata
- Authors
- Koga, Y., Kando, S., & Miyao, Y.
- Affiliations
- The University of Tokyo (JP)
- ORCID
- Miyao, Yusuke
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2026
- Published
- 2026
- Pages
- 22676-22695
- OpenAlex
- W7224491108
- Cited by
- 0
- References
- 69
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-sa
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by-nc-sa)
- Funders
- Japan Society for the Promotion of Science
- ACT-X
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- Japanese phonology, mora, pitch accent, phonological discrimination, minimal pairs, self-supervised speech models, geminate consonants, vowel duration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-20, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-20 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs. https://app.cochat.ai/papers/10.18653/2026.findings-emnlp.1072
- The paper
- Koga, Y., Kando, S., & Miyao, Y. (2026). JPhonABX: Japanese Phonological Discrimination Dataset for Moraic and Accentual Minimal Pairs. Findings of the Association for Computational Linguistics: EMNLP 2026. https://doi.org/10.18653/2026.findings-emnlp.1072
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.