Beyond Black Box AI generated Plagiarism Detection: From Sentence to Document Level

Abstract

The increasing reliance on large language models (LLMs) in academic writing has led to a rise in plagiarism.Existing AI-generated text classifiers have limited accuracy and often produce false positives.We propose a novel approach using natural language processing (NLP) techniques, offering quantifiable metrics at both sentence and document levels for easier interpretation by human evaluators.Our method employs a multi-faceted approach, generating multiple paraphrased versions of a given question and inputting them into the LLM to generate answers.By using a contrastive loss function based on cosine similarity, we match generated sentences with those from the student's response.Our approach achieves up to 94% accuracy in classifying human and AI text, providing a robust and adaptable solution for plagiarism detection in academic settings.This method improves with LLM advancements, reducing the need for new model training or reconfiguration, and offers a more transparent way of evaluating and detecting AI-generated text.