A Deep Neural Architecture for Decision-Aware Meta-Review Generation

Abstract

Automatically generating meta-reviews from peer-reviews is a new and challenging task. Although close, the task is not precisely summarizing the peer-reviews. Usually, a conference chair or a journal editor writes a meta-review after going through the reviews written by the appointed reviewers, rounds of discussions with them, finally arriving at a consensus on the paper's fate. In essence, the meta-review texts are decision-aware, i.e., the meta reviewer already forms the decision before writing the meta-review, and the corresponding text conforms to that decision. We leverage this seed idea and design a deep neural architecture to generate decision-aware meta-reviews in this work. We propose a multi-encoder transformer network for peer-review decision prediction and subsequent meta-review generation. We analyze our output quantitatively and qualitatively and argue that quantitative text summarization metrics are not suitable for evaluating the generated meta-reviews. Our proposed model performs comparably with the recent state-of-the-art text summarization approaches. Qualitative evaluation of our model-generated output is encouraging on an open access peer reviews dataset that we curate from the open review platform. We make our data and codes available11https://www.iitp.ac.in/~ai-nlp-ml/resources.html# decision-aware-meta-review.