LLMarking: Adaptive Automatic Short-Answer Grading Using Large Language Models

With the advancement of educational technology, automatic assessment systems are becoming increasingly essential, particularly for grading short-answer questions. However, due to the inherent ambiguity and complexity of language, automatic grading of short-answer questions remains a challenge. Traditional grading methods are often time-consuming and subjective, highlighting the need for efficient, objective, and feedback-driven solutions. This paper proposes an innovative approach to automatic short answer grading (ASAG) utilizing large language models (LLMs). We introduce a specialized design for crafting questions and corresponding answers named Key Point Scoring Framework (KPSF) which significantly enhances the model's performance in ASAG tasks and improves the flexibility and objectivity of assessments. Moreover, we incorporate Prompt Dynamic Adjustment (PDA) that continuously refines the grading process, effectively handling ambiguous student responses while ensuring reliable results. To evaluate our approach, we develop a multidisciplinary dataset and incorporate real-world dataset from actual exams. The experimental results demonstrate that our ASAG approach provides educators with a highly efficient, flexible and accurate tool for short-answer assessments, indicating a significant advancement in automatic grading technology.