Experiments in automatic assessment using basic information retrieval techniques

Md Maruf Hasan*

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

2 Citations (Scopus)

Abstract

In Information Retrieval (IR), the similarity scores between a query and a set of documents are calculated, and the relevant documents are ranked based on their similarity scores. IR systems often consider queries as short documents containing only a few words in calculating document similarity score. In Computer Aided Assessment (CAA) of narrative answers, when model answers are available, the similarity score between Students' Answers and the respective Model Answer may be a good quality-indicator. With such an analogy in mind, we applied basic IR techniques in the context of automatic assessment and discussed our findings. In this paper, we explain the development of a web-based automatic assessment system that incorporates 5 different text analysis techniques for automatic assessment of narrative answers using vector space framework. The experimental results based on 30 narrative questions with 30 model answers, and 300 student's answers (from 10 students) show that the correlation of automatic assessment with human assessment is higher when advanced text processing techniques such as Keyphrase Extraction and Synonym Resolution are applied.

Original languageEnglish
Title of host publicationKnowledge, Information, and Creativity Support Systems - 5th International Conference, KICSS 2010, Revised Selected Papers
Pages13-21
Number of pages9
DOIs
Publication statusPublished - 2011
Externally publishedYes
Event5th International Conference on Knowledge, Information and Creativity Support Systems, KICSS 2010 - Chiang Mai, Thailand
Duration: 25 Nov 201027 Nov 2010

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume6746 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference5th International Conference on Knowledge, Information and Creativity Support Systems, KICSS 2010
Country/TerritoryThailand
CityChiang Mai
Period25/11/1027/11/10

Keywords

  • Computer Aided Assessment (CAA)
  • Information Retrieval (IR)
  • Intelligent Text Analysis
  • Natural Language Processing

Fingerprint

Dive into the research topics of 'Experiments in automatic assessment using basic information retrieval techniques'. Together they form a unique fingerprint.

Cite this