Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 362

The DOI matches its Crossref record; 155 of the 155 references it cites were found in a registry or an index.

Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations

Hickman, Louis, Thapa, Stuti, Tay, Louis, Cao, Mengyang, Srinivasan, Padmini·2022·Organizational Research Methods

Abstract

Recent advances in text mining have provided new methods for capitalizing on the voluminous natural language text data created by organizations, their employees, and their customers. Although often overlooked, decisions made during text preprocessing affect whether the content and/or style of language are captured, the statistical power of subsequent analyses, and the validity of insights derived from text mining. Past methodological articles have described the general process of obtaining and analyzing text data, but recommendations for preprocessing text data were inconsistent. Furthermore, primary studies use and report different preprocessing techniques. To address this, we conduct two complementary reviews of computational linguistics and organizational text mining research to provide empirically grounded text preprocessing decision-making recommendations that account for the type of text mining conducted (i.e., open or closed vocabulary), the research question under investigation, and the data set’s characteristics (i.e., corpus size and average document length). Notably, deviations from these recommendations will be appropriate and, at times, necessary due to the unique characteristics of one’s text data. We also provide recommendations for reporting text mining to promote transparency and reproducibility.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/1094428120971683

Cites

155 of 155 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 155 references

Research works printed but not linked: 43

  1. 10.1002/(SICI)1099-1379(199711)18:1+<513::AID-JOB905>3.0.CO;2-8 · doi:10.1002/(sici)1099-1379(199711)18:1+<513::aid-job905>3.0.co;2-8
  2. 10.1145/2812809 · doi:10.1145/2812809
  3. *Alhaj Y. A, 2018, Proceedings of the 2nd International Conference on Telecommunications and Communication Engineering, 397, 10.1145/3291842.3291900 · doi:10.1145/3291842.3291900
  4. 10.1109/IALP.2015.7451548 · doi:10.1109/ialp.2015.7451548
  5. *Arif S. M., 2017, Proceedings of the International Conference on Computing, Mathematics and Statistics (iCMS 2015), 93
  6. *Ayedh A., 2016, Algorithms, 9, 1
  7. 10.1109/KBEI.2017.8324994 · doi:10.1109/kbei.2017.8324994
  8. *Bao Y., 2014, International Conference on Intelligent Computing, 615
  9. 10.1109/TELFOR.2016.7818923 · doi:10.1109/telfor.2016.7818923
  10. Bird S., 2009, Natural language processing with Python: Analyzing text with the natural language toolkit
  11. *Buyl T., 2019, Journal of Management, 45, 1372
  12. 10.3115/v1/P15-2071 · doi:10.3115/v1/p15-2071
  13. 10.18653/v1/P16-1187 · doi:10.18653/v1/p16-1187
  14. *Deniz A., 2017, 2017 International Conference on Computer Science and Engineering (UBMK), 655
  15. 10.1109/BRACIS.2014.20 · doi:10.1109/bracis.2014.20
  16. 10.1109/IranianCEE.2019.8786652 · doi:10.1109/iraniancee.2019.8786652
  17. 10.1109/ISECS.2010.9 · doi:10.1109/isecs.2010.9
  18. *Farrar D., 2019, Proceedings of the 10th International Workshop on Software and System Traceability (SST’19) at the International Conference on Software Engineering
  19. 10.1002/(SICI)1099-1379(199711)18:1+<583::AID-JOB908>3.0.CO;2-T · doi:10.1002/(sici)1099-1379(199711)18:1+<583::aid-job908>3.0.co;2-t
  20. *Gomes S. R, 2017, 2017 4th International Conference on Advances in Electrical Engineering (ICAEE), 482
  21. *Hardaya I. S, 2017, 2017 3rd International Conference on Science in Information Technology (ICSITech), 144
  22. Hart R. P., 2000, DICTION 5.0: The text-analysis program
  23. 10.1109/ICITISEE.2018.8720957 · doi:10.1109/icitisee.2018.8720957
  24. *Hidayatullah A. F., 2015, Proceeding of the Electrical Engineering Computer Science and Informatics, 2, 127
  25. 10.1609/icwsm.v14i1.7306 · doi:10.1609/icwsm.v14i1.7306
  26. 10.1109/SMAP.2017.8022661 · DOI
  27. 10.1016/j.jvb.2014.05.001 · doi:10.1016/j.jvb.2014.05.001
  28. 10.1037/0021-9010.76.3.465 · doi:10.1037/0021-9010.76.3.465
  29. 10.1109/ICCMC.2017.8282676 · doi:10.1109/iccmc.2017.8282676
  30. 10.1177/014920639402000410 · doi:10.1177/014920639402000410 · Paper page
  31. *Palmer I., Kabanoff B., Dunford R. (1997). Managerial accounts of downsizing. Journal of Organizational Behavior, 18(1997), 623–639. https://doi.org/10.1002/(SICI)1099-1379(199711)18:1+<623:AID-JOB909>3.0.CO;2-T · doi:10.1002/(sici)1099-1379(199711)18:1+<623::aid-job909>3.0.co;2-t
  32. Pennebaker J. W., 2015, Linguistic Inquiry and Word Count: LIWC 2015 operator’s manual
  33. Pennebaker J. W., 2015, The development and psychometric properties of LIWC2015
  34. 10.1037/0033-2909.124.2.262 · doi:10.1037/0033-2909.124.2.262 · Paper page
  35. 10.18653/v1/E17-2069 · doi:10.18653/v1/e17-2069
  36. Schwartz H. A., 2013, Second Joint Conference on Lexical and Computational Semantics, Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity, 296
  37. 10.18653/v1/D15-1197 · doi:10.18653/v1/d15-1197
  38. *Suadaa L. H., 2016, 2016 4th International Conference on Information and Communication Technology (ICoICT), 1
  39. 10.1145/2540988 · doi:10.1145/2540988
  40. *Sun F., 2014, 2014 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr), 122
  41. 10.1037/0021-9010.74.2.259 · doi:10.1037/0021-9010.74.2.259 · Paper page
  42. 10.1109/ISACV.2017.8054932 · doi:10.1109/isacv.2017.8054932
  43. 10.1145/3090354.3090371 · doi:10.1145/3090354.3090371

Cited by

See all 362

Metadata

Authors
Hickman, L., Thapa, S., Tay, L., Cao, M., & Srinivasan, P.
Venue
Organizational Research Methods
Published
2020-11-23
Volume
25
Issue
1
Pages
114-146
ISSN
  • 1094-4281 (ISSN-L)
  • 1552-7425
OpenAlex
W3109416014
Cited by
362
References
155
FWCI
86.60
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations. https://app.cochat.ai/papers/10.1177/1094428120971683
The paper
Hickman, L., Thapa, S., Tay, L., Cao, M., & Srinivasan, P. (2022). Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations. Organizational Research Methods. https://doi.org/10.1177/1094428120971683

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.