Teacher Evaluative Judgment in Material Creation with GenAI

6,287

The recent introduction of Generative AI (GenAI) into educational settings has led to
significant interest in its ability to streamline teaching material creation, saving time and
offering enhanced customization (Karpouzis et al., 2024; Lan & Chen, 2024). Teachers
are increasingly utilizing GenAI tools to generate texts, quizzes, and explanations,
indicating that GenAI can contribute to the effectiveness of educational resources
(Koraishi, 2023; Mittal, 2024). However, while many teachers see the potential benefits
of GenAI, there remain gaps in the knowledge and effective evaluation of those
applications. During the summer of 2024, the PI tested one such implementation and
managed to produce an evaluation (pending publication). That success prompted the
need to create a broader framework based on teacher’s experiences and evaluation
with reference to pre- and post-GenAI chatbot literature.

The proposed research aims to address this need by formulating this framework for
material creation to guide teachers in evaluating and using GenAI effectively. This
framework will empower teachers to overcome challenges such as limited experience
and insufficient understanding of the full potential of GenAI, as identified in existing
studies (Edmett et al., 2024; van den Berg & du Plessis, 2023). By focusing on the
critical role of human agency in evaluating GenAI-generated content, this project aligns
with XJTLU’s vision of enhancing modules with GenAI. The resulting framework will not
only help address challenges, but also unlock broader possibilities for GenAI in material creation, offering practical solutions for teachers to make informed evaluative
judgments.

The proposed project aims to answer the following questions:
RQ1: How is teachers’ evaluative judgment exercised for material creation and GenAI
usage?
RQ2: How do teachers use the framework?
RQ3: What is the impact of the framework on the effectiveness of materials?
In addition, the project aims to produce an evaluative framework for critical and effective
use of GenAI for material creation and evaluation. This project will also involve a series
of professional development workshops in which teachers would practically explore
creation and evaluation of materials using GenAI.