TY - GEN
T1 - Experiments in automatic assessment using basic information retrieval techniques
AU - Hasan, Md Maruf
PY - 2011
Y1 - 2011
N2 - In Information Retrieval (IR), the similarity scores between a query and a set of documents are calculated, and the relevant documents are ranked based on their similarity scores. IR systems often consider queries as short documents containing only a few words in calculating document similarity score. In Computer Aided Assessment (CAA) of narrative answers, when model answers are available, the similarity score between Students' Answers and the respective Model Answer may be a good quality-indicator. With such an analogy in mind, we applied basic IR techniques in the context of automatic assessment and discussed our findings. In this paper, we explain the development of a web-based automatic assessment system that incorporates 5 different text analysis techniques for automatic assessment of narrative answers using vector space framework. The experimental results based on 30 narrative questions with 30 model answers, and 300 student's answers (from 10 students) show that the correlation of automatic assessment with human assessment is higher when advanced text processing techniques such as Keyphrase Extraction and Synonym Resolution are applied.
AB - In Information Retrieval (IR), the similarity scores between a query and a set of documents are calculated, and the relevant documents are ranked based on their similarity scores. IR systems often consider queries as short documents containing only a few words in calculating document similarity score. In Computer Aided Assessment (CAA) of narrative answers, when model answers are available, the similarity score between Students' Answers and the respective Model Answer may be a good quality-indicator. With such an analogy in mind, we applied basic IR techniques in the context of automatic assessment and discussed our findings. In this paper, we explain the development of a web-based automatic assessment system that incorporates 5 different text analysis techniques for automatic assessment of narrative answers using vector space framework. The experimental results based on 30 narrative questions with 30 model answers, and 300 student's answers (from 10 students) show that the correlation of automatic assessment with human assessment is higher when advanced text processing techniques such as Keyphrase Extraction and Synonym Resolution are applied.
KW - Computer Aided Assessment (CAA)
KW - Information Retrieval (IR)
KW - Intelligent Text Analysis
KW - Natural Language Processing
UR - http://www.scopus.com/inward/record.url?scp=81755166832&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-24788-0_2
DO - 10.1007/978-3-642-24788-0_2
M3 - Conference Proceeding
AN - SCOPUS:81755166832
SN - 9783642247873
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 13
EP - 21
BT - Knowledge, Information, and Creativity Support Systems - 5th International Conference, KICSS 2010, Revised Selected Papers
T2 - 5th International Conference on Knowledge, Information and Creativity Support Systems, KICSS 2010
Y2 - 25 November 2010 through 27 November 2010
ER -