CoChat
Sign Up
Cited by 63Sentiment Analysis and Opinion Miningscientometrics and bibliometrics researchComputational and Text Analysis Methods

Aspect-based Sentiment Analysis of Scientific Reviews

Open access
Chakraborty, Souvic, Goyal, Pawan, Mukherjee, Animesh·2020·Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020
DOI 10.1145/3383583.3398541
Open this paper in CoChat — free accountOpen PDF

Abstract

Scientific papers are complex and understanding the usefulness of these papers requires prior knowledge. Peer reviews are comments on a paper provided by designated experts on that field and hold a substantial amount of information, not only for the editors and chairs to make the final decision, but also to judge the potential impact of the paper. In this paper, we propose to use aspect-based sentiment analysis of scientific reviews to be able to extract useful information, which correlates well with the accept/reject decision.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3383583.3398541

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Aspect-based analysis of peer reviews can help identify aspects of the manuscript that were insufficiently or vaguely covered by the reviewers.

    “In particular, aspect-based analysis of peer reviews (Chakraborty et al., 2020; Kennard et al., 2022) can help identify aspects of the manuscript that were insufficiently or vaguely covered by the reviewers.”

    From What Can Natural Language Processing Do for Peer Review? (2024) · cites this paper as Chakraborty et al., 2020

    The Introduction (p. 2) says the method identifies aspects and sentiments in review sentences, and the Conclusion (p. 9) reports analysis of aspect-level disagreements. These findings support analyzing what reviewers discuss, but our copy does not say the method identifies manuscript aspects that reviewers covered insufficiently or vaguely. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 24

  1. [1][1] Mohammad Al-Smadi, Omar Qawasmeh, Mahmoud Al-Ayyoub, Yaser Jararweh, and Brij Gupta. 2018. Deep Recurrent neural network vs. support vector machine for aspect-based sentiment analysis of Arabic hotels'reviews. Journal of computational science 27 (2018), 386-393. -
  2. [3][3] Seyed-Ali Bahrainian and Andreas Dengel. 2013. Sentiment analysis and summarization of twitter data. In 2013 IEEE 16th International Conference on Computational Science and Engineering . IEEE, 227-234. -
  3. [2][2] Stefanos Angelidis and Mirella Lapata. 2018. Multiple instance learning networks for fine-grained sentiment analysis. Transactions of the Association for Computational Linguistics 6 (2018), 17-31. -
  4. [5][5] Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, et al. 2018. Universal sentence encoder for English. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations . 169-174. -
  5. [7][7] Yang Gao, Steffen Eger, Ilia Kuznetsov, Iryna Gurevych, and Yusuke Miyao. 2019. Does My Rebuttal Matter? Insights from a Major NLP Conference. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . 1274-1290. -
  6. [6][6] Daniel R Dooly, Qi Zhang, Sally A Goldman, and Robert A Amar. 2002. Multiple-instance learning of real-valued data. Journal of Machine Learning Research 3, Dec (2002), 651-678. -
  7. [10][10] Binxuan Huang, Yanglan Ou, and Kathleen M Carley. 2018. Aspect level sentiment classification with attention-over-attention neural networks. In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation . Springer, 197-206. -
  8. [9][9] Jeremy Howard and Sebastian Ruder. 2018. Universal Language Model Fine-tuning for Text Classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 328-339. -
  9. [11][11] Dongyeop Kang, Waleed Ammar, Bhavana Dalvi, Madeleine van Zuylen, Sebastian Kohlmeier, Eduard Hovy, and Roy Schwartz. 2018. A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) . 1647-1661. -
  10. [13][13] Michail Kovanis, Raphaël Porcher, Philippe Ravaud, and Ludovic Trinquart. 2016. The global burden of journal peer review in the biomedical literature: Strong imbalance in the collective enterprise. PloS one 11, 11 (2016). -
  11. [12][12] Chetan Kaushik and Atul Mishra. 2014. A Scalable, Lexicon Based Technique for Sentiment Analysis. International Journal in Foundations of Computer Science & Technology 4, 5 (Sep 2014), 35âĂŞ56. https://doi.org/10.5121/ijfcst.2014.4504 - · DOI
  12. [14][14] Langford and Guzdial. 2015. The Arbitrariness of Reviews, and Advice for School Administrators. Commun. ACM 58 (03 2015), 12-13. https://doi.org/10.1145/2732417 - · DOI · Paper page
  13. [16][16] Prem Melville, Wojciech Gryc, and Richard D. Lawrence. 2009. Sentiment Analysis of Blogs by Combining Lexical Knowledge with Text Classification. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Paris, France) (KDD '09) . ACM, New York, NY, USA, 1275-1284. https://doi.org/10.1145/1557019.1557156 - · DOI
  14. [15][15] Zeyang Lei, Yujiu Yang, Min Yang, and Yi Liu. 2018. A Multisentiment-resource Enhanced Attention Network for Sentiment Classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . 758-763. -
  15. [17][17] Bo Pang, Lillian Lee, et al. 2008. Opinion mining and sentiment analysis. Foundations and Trends® in Information Retrieval 2, 1-2 (2008), 1-135. -
  16. [19][19] Sandipan Sikdar, Matteo Marsili, Niloy Ganguly, and Animesh Mukherjee. 2016. Anomalies in the peer-review system: A case study of the journal of High Energy Physics. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management . 2245-2250. -
  17. [18][18] Azzurra Ragone, Katsiaryna Mirylenka, Fabio Casati, and Maurizio Marchese. 2013. On peer review in computer science: analysis of its effectiveness and suggestions for improvement. Scientometrics 97, 2 (01 Nov 2013), 317-356. https://doi.org/10.1007/ s11192-013-1002-z -
  18. [20][20] Sandipan Sikdar, Matteo Marsili, Niloy Ganguly, and Animesh Mukherjee. 2017. Influence of reviewer interaction network on long-term citations: a case study of the scientific peer-review system of the journal of high energy physics. In 2017 ACM/IEEE Joint Conference on Digital Libraries (JCDL) . IEEE, 1-10. -
  19. [21][21] Youwei Song, Jiahai Wang, Tao Jiang, Zhiyue Liu, and Yanghui Rao. 2019. Targeted Sentiment Classification with Attentional Encoder Network. Lecture Notes in Computer Science (2019), [93âĂŞ103. https://doi.org/10.1007/978-3-030-30490-4\_9](https://doi.org/10.1007/978-3-030-30490-4_9) - · DOI
  20. [22][22] Tun Thura Thet, Jin-Cheon Na, and Christopher S. G. Khoo. 2010. Aspect-based sentiment analysis of movie reviews on discussion boards. J. Information Science 36 (2010), 823-848. -
  21. [24][24] Peter D Turney. 2002. Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th annual meeting on association for computational linguistics . Association for Computational Linguistics, 417-424. -
  22. [26][26] Min Yang, Wei Zhao, Jianbo Ye, Zeyang Lei, Zhou Zhao, and Soufei Zhang. 2018. Investigating Capsule Networks with Dynamic Routing for Text Classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . 3110-3119. -
  23. [25][25] Ke Wang and Xiaojun Wan. 2018. Sentiment analysis of peer review texts for scholarly papers. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval . 175-184. -
  24. [27][27] Yasuhisa Yoshida, Tsutomu Hirao, Tomoharu Iwata, Masaaki Nagata, and Yuji Matsumoto. 2011. Transfer learning for multiple-domain sentiment analysisâĂŤidentifying domain dependent/independent word polarity. In Proceedings of the TwentyFifth AAAI Conference on Artificial Intelligence . AAAI Press, 1286-1291.

Cited by

Sorted by citations

Authors
Chakraborty, S., Goyal, P., & Mukherjee, A.
Affiliations
Indian Institute of Technology Kharagpur (IN)
Venue
Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020
Publisher
ACM
Published
2020-08
Pages
207-216
ISSN
1552-5996 (ISSN-L)
DOI
10.1145/3383583.3398541
OpenAlex
W3033536844
Cited by
63
References
3
Citations by year
2026: 5 · 2025: 7 · 2024: 10 · 2023: 19 · 2022: 12 · 2021: 10
FWCI
3.84
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM/IEEE Joint Conference on Digital Libraries (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, public-domain)
  • Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
Tata Consultancy Services (15)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Keywords
aspect-based sentiment analysis, peer review, scientific review, reviewer comments, research impact assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review