Sentiment Polarity Detection for Software Development
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
The role of sentiment analysis is increasingly emerging to study software developers’ emotions by mining crowd-generated content within social software engineering tools. However, off-the-shelf sentiment analysis tools have been trained on non-technical domains and general-purpose social media, thus resulting in misclassifications of technical jargon and problem reports. Here, we present Senti4SD, a classifier specifically trained to support sentiment analysis in developers’ communication channels. Senti4SD is trained and validated using a gold standard of Stack Overflow questions, answers, and comments manually annotated for sentiment polarity. It exploits a suite of both lexicon- and keyword-based features, as well as semantic features based on word embedding. With respect to a mainstream off-the-shelf tool, which we use as a baseline, Senti4SD reduces the misclassifications of neutral and positive posts as emotionally negative. To encourage replications, we release a lab package including the classifier, the word embedding space, and the gold standard with annotation guidelines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Russell, J. A. (1980) · Journal of Personality and Social Psychology · doi:10.1037/h0077714 · Cited by 16,334
Lazarus, R. S. (1991) · doi:10.1093/oso/9780195069945.001.0001 · Cited by 12,691
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · cited 1×
“As an evidence of the reliability of the coding schema and procedure, we computed the weighted Cohen ’ s Kappa among the pairs of raters (Cohen 1968).”
MethodRecursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 1×
“As an evidence of the reliability of the coding schema and procedure, we computed the weighted Cohen ’ s Kappa among the pairs of raters (Cohen 1968).”
Method
Cited by
Socio-Technical Grounded Theory for Software Engineering
Hoda, R. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3106280 · Cited by 204
Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642596 · Cited by 160
Sentiment Analysis for Software Engineering: How Far Can Pre-trained Transformer Models Go?
Zhang, T., Xu, B., Thung, F., Haryono, S. A., Lo, D., & Jiang, L. (2020) · 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme46990.2020.00017 · Cited by 129
“…informal short texts SentiStrength-SE [5] Lexicon and rule-based Jira issues comments SentiCR [6] Supervised learning Code review comments Senti4SD [26] Supervised learning Stack Overflow Emotxt [23] Supervised learning Stack Overflow and Jira DEVA [24] Lexical approach Jira issue comments [19]…”
BackgroundMethodInfluentialRaman, N., Cao, M., Tsvetkov, Y., Kästner, C., & Vasilescu, B. (2020) · Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results · doi:10.1145/3377816.3381732 · Cited by 70
Metadata
- Authors
- Calefato, F., Lanubile, F., Maiorano, F., & Novielli, N.
- Venue
- Empirical Software Engineering
- Published
- 2017-09-19
- Volume
- 23
- Issue
- 3
- Pages
- 1352-1382
- ISSN
- 1382-3256 (ISSN-L)
- 1573-7616
- OpenAlex
- W2885935106
- Cited by
- 171
- References
- 16
- FWCI
- 10.50
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sentiment Polarity Detection for Software Development. https://app.cochat.ai/papers/10.1007/s10664-017-9546-9
- The paper
- Calefato, F., Lanubile, F., Maiorano, F., & Novielli, N. (2018). Sentiment Polarity Detection for Software Development. Empirical Software Engineering. https://doi.org/10.1007/s10664-017-9546-9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.