Diagnostic Assessment of Reading Comprehension: Development and Validation

Abstract:

Currently available tools for the assessment of reading comprehension have limited utility for profiling students’ strengths and weaknesses in the components of reading comprehension, are unduly affected by students’ ability to decode the words in texts, are generally non-informative for teachers in guiding instruction, and are particularly limited in assisting teachers of language minority students. Specifically, we will develop a diagnostic assessment of reading comprehension (DARC) for children in grades 3 through 5 that focuses on the critical components of comprehension. They include: 1) decoding and word reading fluency; 2) central comprehension processes (remembering the text, accessing relevant background knowledge, making text-based inferences, and integrating text-based information with world knowledge), and 3) sensitivity to linguistic variation in the text. Pilot work in all three areas has established the feasibility of this approach. The specific aims of the current proposal are a) to further develop assessments (subtests) that measure the key components of comprehension; b) to establish reliability and validity for the individual components; c) to establish reliability and validity for the assessment as a whole; and d) to provide norms on the DARC for children in grades 3-5 for both monolingual English speakers and for English Language Learners (ELLs). Validity will be evaluated using item response methods and confirmatory factor analysis to compare performance on DARC sub-scores to other targeted assessments of the hypothesized comprehension processes, and by relating DARC scores to performance on standardized and state-prescribed reading assessments as well as textbook specific end-of-chapter assessments.

During years 1&2, we will develop texts and items and pilot them for the underlying components of comprehension, and will conduct initial studies to determine that subtests are developmentally sensitive, that they are usable with English-language learners and relate to other reading assessments that target similar constructs, where such assessments exist. These small scale validity studies are designed to demonstrate that individual subtests indeed reflect the specific components of comprehension processing that they are designed to tap. During year 3 we will engage in a larger-scale validity study using a broader sample of students and relating performance on the DARC to performance on the Gray Oral, the state assessments, and textbook end-of-chapter assessments of text-based reading in the content areas. In year 4 we will norm the assessment and equate multiple forms. All reliability, validity, and norming studies will be carried out in multiple locations drawing on the experience of our team to conduct multi-site assessment research on reading and language using samples that range in size from 600 to 1200.

Collaborating Organizations:

Center for Applied Linguistics, Diane August, Co-Project Director