Evaluating Tracking Accuracy of an Automatic Reading Tutor
Abstract
In automatic reading tutoring, tracking is the process of automatically following a reader through a given target text. When developing tracking algorithms, a measure of the tracking accuracy–how often a spoken word is aligned to the right target text word position–is needed in order to evaluate performance and compare different algorithms. This paper presents a framework for determining the observed tracking error rate. The proposed framework is used to evaluate three tracking strategies: A) follow the reader to whichever word he/she jumps to in the text, B) follow the reader monotonically from left to right ignoring word skips and regressions (going back to a previous text word), and C) the same as B but allowing isolated word skips. Observed tracking error rate for each of the three tracking strategies is: A: 53%, B: 56%, and C: 47%, on 1883 utterances from 25 children.
BibTeX
@workshop{Rasmussen-2011-122077,author = {Morten Højfeldt Rasmussen and Jack Mostow and Zheng-Hua Tan and Børge Lindberg and Yuanpeng Li},
title = {Evaluating Tracking Accuracy of an Automatic Reading Tutor},
booktitle = {Proceedings of ISCA International Workshop on Speech and Language Technology in Education (SLaTE '11)},
year = {2011},
month = {August},
pages = {17 - 20},
}