Fred Jelinek
Open accessAbstract
Frederick Jelinek died, peacefully and unexpectedly, on 14 September 2010.Over a distinguished career of nearly fifty years, Fred made important contributions in areas ranging from coding theory and speech recognition to parsing and machine translation.But more important than these specific contributions was his role in persuading the fields of speech and language engineering to adopt statistical methods and the "noisy channel model," returning to the path opened up by Claude Shannon in 1948.And more important still was the role that he played in defining, exemplifying, and promoting what has become the standard research paradigm in an even wider range of fields: the competitive evaluation of alternative algorithms based on precise quantitative criteria defined in advance, relative to a shared body of training and testing material.After receiving his Ph.D. from MIT in 1962, Fred taught at Cornell from 1962 to 1972, worked at IBM from 1972 to 1993, and taught at Johns Hopkins from 1993 to 2010.Fred's many technical accomplishments during this long and productive career can be seen as episodes in two epic narratives, which, like the Iliad and the Odyssey, are related but have separate themes and story lines.The theme of the first epic is the return of Information Theory to center stage in speech and language processing; and the theme of the second epic is the development of a new relationship between science and engineering in speech recognition, computational linguistics, and artificial intelligence (AI) more generally.Fred gave us a vivid first-person narrative of the first of these epic adventures in his ACL Lifetime Achievement Award speech (Jelinek 2009).But missing from this recital is a picture of the dire state of the information-theoretic forces when Fred joined the fray by undertaking speech recognition research at IBM in 1972.To understand what the world was like then, we need to go back to the mid 1950s.Claude Shannon (1956, page 3) wrote:Information theory has, in the last few years, become something of a scientific bandwagon.Starting as a technical tool for the communication engineer, it has received an extraordinary amount of publicity in the popular as well as the scientific press. . . .Our fellow scientists in many different fields, attracted by the fanfare and by the new avenues opened to scientific analysis, are using these ideas in their own problems.Applications are being made to biology, psychology, linguistics, fundamental physics, economics, the theory of organization, and many others. . . .Although this wave of popularity is certainly pleasant and exciting for those of us working in the field, it carries at the same time an element of danger. . . .It will be all too easy for our somewhat artificial prosperity to collapse overnight when it is realized that the use of a few exciting words like information, entropy, redundancy, do not solve all our problems.Ironically, the seeds of that overnight collapse were at that very moment being sown by a young linguist with a fresh Ph.D. from Penn, whose lecture notes for an MIT
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Shannon, C. (1956) · IRE Transactions on Information Theory · doi:10.1109/tit.1956.1056774 · Cited by 237 · see citing works
Language and Machines: Computers in Translation and Linguistics
Pierce, J. R., & Carroll, J. B. (1966) · National Academy of Sciences-National Research Council eBooks · Cited by 229 · see citing works
Formal grammar and information theory: together again?
Pereira, F. M. B. (2000) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2000.0583 · Cited by 106 · see citing works
Pierce, J. R. (1969) · The Journal of the Acoustical Society of America · doi:10.1121/1.1911801 · Cited by 77 · see citing works
The Dawn of Statistical ASR and MT
Jelinek, F. (2009) · Computational Linguistics · doi:10.1162/coli.2009.35.4.35401 · Cited by 58 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Shannon, Claude. 1956. The bandwagon. IRE Transactions on Information Theory, 2(1):3.
Cited by
Sorted by citations
Donoho, D. (2017) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2017.1384734 · Cited by 695 · see citing works
Holistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480 · see citing works
Prediction and explanation in social systems
Hofman, J. M., Sharma, A., & Watts, D. J. (2017) · Science · doi:10.1126/science.aal3856 · Cited by 429 · see citing works
Machine Learning for Social Science: An Agnostic Approach
Grimmer, J., Roberts, M. E., & Stewart, B. (2021) · Annual Review of Political Science · doi:10.1146/annurev-polisci-053119-015921 · Cited by 324 · see citing works
Should social science be more solution-oriented?
Watts, D. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0015 · Cited by 301 · see citing works
Megastudies improve the impact of applied behavioural science
Milkman, K. L., Gromet, D. M., Ho, H., Kay, J., Lee, T. W., Pandiloski, P., Park, Y., Rai, A., Bazerman, M. H., Beshears, J. L., et al. (2021) · Nature · doi:10.1038/s41586-021-04128-4 · Cited by 249 · see citing works
Passing the Data Baton : A Retrospective Analysis on Data Science Work and Workers
Crisan, A., Fiore-Gartland, B., & Tory, M. K. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3030340 · Cited by 66 · see citing works
Church, K. W., & Kordoni, V. (2022) · Natural Language Engineering · doi:10.1017/s1351324922000043 · Cited by 28 · see citing works
Passing the Data Baton: A Retrospective Analysis on Data Science Work and Workers
Crisan, A., Fiore-Gartland, B., & Tory, M. K. (2020) · IEEE Visualization in Data Science (VDS) · doi:10.1109/vds51726.2020.00005 · Cited by 10 · see citing works
The incentive gap in data work in the era of large models
Gero, K. I., Das, P., Dognin, P., Padhi, I., Sattigeri, P., & Varshney, K. R. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00673-x · Cited by 9 · see citing works
Artificial intelligence research at IBM
Fan, J., Campbell, M. S., & Kingsbury, B. E. D. (2011) · IBM Journal of Research and Development · doi:10.1147/jrd.2011.2163282 · Cited by 7 · see citing works
Bornkamp, B., Zaoli, S., Azzarito, M., Martin, R., Müller, C. P., Moloney, C., Capestro, G., Ohlssen, D. I., & Baillie, M. (2024) · Pharmaceutical Statistics · doi:10.1002/pst.2368 · Cited by 6 · see citing works
More than Marketing? On the Information Value of AI Benchmarks for Practitioners
Hardy, A., Reuel, A., Meimandi, K. J., Soder, L., Griffith, A., Asmar, D. M., Koyejo, S., Bernstein, M. S., & Kochenderfer, M. J. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712152 · Cited by 10 · see citing works
State-of-the-Art: The Temporal Order of Benchmarking Culture
Campolo, A. (2025) · Digital Society · doi:10.1007/s44206-025-00190-x · Cited by 5 · see citing works
Browsing and Searching Metadata of TREC
Breuer, T., Voorhees, E. M., & Soboroff, I. M. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657873 · Cited by 5 · see citing works
Tensor extrapolation: an adaptation to data sets with missing entries
Schosser, J. (2022) · Journal Of Big Data · doi:10.1186/s40537-022-00574-7 · Cited by 3 · see citing works
When black box algorithms are (not) appropriate: a principled prediction-problem ontologyPreprint
Rodu, J. S., & Baiocchi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.07648 · Cited by 3 · see citing works
Schosser, J. (2024) · Communications in Statistics Case Studies Data Analysis and Applications · doi:10.1080/23737484.2024.2329171 · Cited by 2 · see citing works
Kauermann, G. (2023) · Springer eBooks · doi:10.1007/978-3-662-66278-6_1
(2025) · Duke University Press eBooks · doi:10.1215/9781478060529-005
(2025) · Duke University Press eBooks · doi:10.1215/9781478060529-003
(2025) · Duke University Press eBooks · doi:10.1215/9781478060529-002
Could AI Become Neurodivergent?
(2025) · Duke University Press eBooks · doi:10.1215/9781478060529-004
Introduction: Deep Machines and Surfaces of Experience
Munster, A. (2025) · Duke University Press eBooks · doi:10.1215/9781478060529-001
(2025) · Duke University Press eBooks · doi:10.1215/9781478060529-008
(2025) · Duke University Press eBooks · doi:10.1215/9781478060529-009
(2025) · Duke University Press eBooks · doi:10.1215/9781478060529-006
The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models
Freiesleben, T., & Zezulka, S. (2026) · Philosophy of Science · doi:10.1017/psa.2026.10280 · Cited by 5 · see citing works
Methodology for Design and Analysis of Machine Learning Competitions
Pavão, A. (2023) · theses.fr (ABES) · doi:10.70675/9234d2f0zebc5z4c1czaf3bzb0ad7a481e04
- Authors
- Liberman, M.
- Affiliations
- University of Pennsylvania (US)
- Corresponding author
- Liberman, Mark
- ORCID
- Liberman, Mark
- Venue
- Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2010-12
- Volume
- 36
- Issue
- 4
- Pages
- 595-599
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W2911878849
- Cited by
- 29
- References
- 5
- Citations by year
- 2026: 1 · 2025: 10 · 2024: 3 · 2023: 4 · 2022: 2 · 2021: 2 · 2020: 3 · 2017: 3
- FWCI
- 0.34
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Computational Linguistics (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Arts and Humanities › History › Biographical and Historical Analysis
- Keywords
- information theory, noisy channel model, speech recognition, statistical machine translation, syntactic analysis, coding theory, algorithm evaluation, science and engineering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
