Reliability and Validity of the KAT™ Engine

Summary

The correlation of the KAT engine with human scorers has been shown to be as high or higher than that between two independent readers in dozens of studies with over 200 prompts of every type--from the persuasive, opinion, argument and creative kinds of special interest in language arts to expository essays in 4th-12th grades, college, graduate and professional knowledge and reasoning.

The correlation and agreement rates of the scores produced by the KAT engine are better the more expert and reliable the human scores. The KAT engine's scores have been shown to reflect progress in writing skills and knowledge as a result of instruction much more sensitively than human scores.

The Summary Street® tutorial application powered by the KAT engine in which students read articles and summarize them produced highly significant improvement in a state assessment of reading and writing compared to students in random control classes.

Most of these results have been reported in prestigious refereed professional journals and conferences.

General Overview of WriteToLearn™ and Its Components (pdf)
Evidence for Reliability, Validity and Learning Effectiveness (pdf)