To eat or to feed: can large language models provide useful feedback in translation education?

This study explores the integration of Large Language Models (LLMs) into Translation Quality Evaluation (TQE) tasks in translation education, addressing the growing challenge of providing expert feedback on student translations amidst increasing student numbers and limited teaching resources. With the advancements in deep learning and the proliferation of machine translation (MT), automatic TQE has gained importance, leading to the development of various automatic evaluation metrics such as BLEU, ROUGE, METEOR, and others. However, these metrics often overlook nuanced aspects of high-quality translation, such as cultural appropriateness and stylistic fidelity, which are better captured by human evaluation. This study proposes an innovative approach by employing LLMs, especially GPT- 4, to generate constructive TQE feedback or artificial intelligence (AI) generated translation feedback. Through similarity evaluation analysis, feedback relevance analysis and prospective user study, the research shows that AI feedback aligns significantly with expert feedback, offering a viable solution to the feedback bottleneck in translation education. The findings indicate that the feedback generated using the designed LLM TQE Feedback Generation Pipeline in this research has a high degree of overlap with expert feedback and is specific. Moreover, students’ acceptance of our pipeline is positive due to the flexibility, promptness, and accuracy of LLM.