Sherlock: A Semi-automatic Framework for Quiz Generation Using a Hybrid Semantic Similarity Measure

• Published in: Cognitive computation Vol. 7; no. 6; pp. 667 - 679
• Main Authors: Lin, Chenghua, Liu, Dong, Pang, Wei, Wang, Zhe
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.12.2015; Springer Nature B.V
• Subjects: Actors; Algorithms; Artificial Intelligence; Automation; Big Data; Biomedical and Life Sciences; Biomedicine; Clustering; Computation by Abstract Devices; Computational Biology/Bioinformatics; Games; Knowledge; Linked Data; Model accuracy; Motion pictures; Neurosciences; Ontology; Semantic web; Semantics; Similarity; Similarity measures; Subject specialists; Text analytics; Semantic similarity; Linked data; Quiz generation; Educational games; RDF
• ISSN: 1866-9956; 1866-9964; 1866-9964
• DOI: 10.1007/s12559-015-9347-7

In this paper, we present a semi-automatic system (Sherlock) for quiz generation using linked data and textual descriptions of RDF resources. Sherlock is distinguished from existing quiz generation systems in its generic framework for domain-independent quiz generation as well as in the ability of controlling the difficulty level of the generated quizzes. Difficulty scaling is non-trivial, and it is fundamentally related to cognitive science. We approach the problem with a new angle by perceiving the level of knowledge difficulty as a similarity measure problem and propose a novel hybrid semantic similarity measure using linked data. Extensive experiments show that the proposed semantic similarity measure outperforms four strong baselines with more than 47 % gain in clustering accuracy. In addition, we discovered in the human quiz test that the model accuracy indeed shows a strong correlation with the pairwise quiz similarity.