Panel #3: “RTI and LD Identification”
Identification of Learning Disabilities in an RTI Framework
By Jack M. Fletcher, Ph.D., University of Houston
RTI Leadership Forum
December 8, 2010
This paper addresses the identification of children with learning disabilities (LD) in a service delivery model based on a Response to Intervention (RTI) framework. I will discuss the alignment of identification and the RTI framework under IDEA 2004 and with research on the definition of LD, will discuss the nature of the comprehensive evaluation, and will address other viewpoints for evaluation and eligibility.
RTI AND IDENTIFICATION
Misconceptions of RTI models are that the primary goal is to identify LD, that failure to respond to instruction equates to special education eligibility or LD, and that evaluation procedures fundamentally change. These misconceptions are inconsistent with IDEA 2004 and with the research base on RTI and on LD. Most of the eligibility provisions of IDEA are universal and not specific to RTI models, including a) parental consent, b) comprehensive evaluation within a specified timeline, c) consideration of exclusionary criteria involving other disabilities and/or factors that represent alternative explanations of low achievement, and d) consideration of data addressing appropriate instruction in general education, including progress based on repeated assessments during this instruction. There are no requirements for specific forms of evaluation, including standardized tests. Educational need must be documented and the child may not be considered eligible for the LD special education category based on a single criterion.
The only requirements for identification specific to an RTI model involve a) specification of instructional strategies used in the RTI process and to increase rate of learning, and b) documentation of parental notification that the child is participating in an RTI process and of parental rights to request an evaluation at any point. Some states have adopted requirements for documentation that the data-gathering processes are implemented with integrity and that the interventions are delivered appropriately in terms of quality, fidelity, and duration, which are major issues for RTI models. Regardless, by implementing an RTI service delivery model, the type of child likely to emerge as having LD will show intractability to instruction and the context of the evaluation will be different from more traditional models in terms of what is known about the child and what may be needed to determine LD and special education eligibility (Fletcher & Vaughn, 2009).
RTI AND LD
The identification issues that emerge are not specific to an RTI model. A foremost consideration is the definition of LD. It is instructive to review the recommendations of a consensus group of researchers convened by the U.S. Department of Education Office of Special Education Programs as part of the LD Summit in 2001 (Bradley, Danielson, & Hallahan, 2002). Three primary criteria emerged, the first two of which are inclusionary: a) the student demonstrates low achievement; b) there is insufficient response to effective, research-based interventions; and c) exclusionary factors such as intellectual disability, sensory deficits, serious emotional disturbance, language minority status, and lack of opportunity to learn should be considered. These criteria were not provided to help support or implement an RTI model. Rather, they were developed as criteria for LD regardless of the identification model employed and are implicit in the requirements of IDEA 2004. Any identification model has to deal with these three issues in the comprehensive evaluation, but RTI models are well poised to assess these criteria.
Comprehensive evaluations represent data-gathering processes by qualified personnel who must observe the child and who may or may not use standardized tests depending on the question. In an RTI model, a comprehensive evaluation represents an effort to understand why the child has not responded to quality instruction, but eligibility for special education cannot be established solely on the basis of data collected as part of the RTI process. Any identification model should consider the three sets of criteria outlined above, which are implicit in IDEA.
Evaluating Instructional Response
instructional program must be supplemented with assessments of fidelity and monitoring of the child’s progress, typically with some type of curriculum-based assessment that permits charting of growth toward grade-level benchmarks. Data from growth, level of performance, or both can be used to determine whether the child is learning adequately or needs more intensive intervention (Fuchs & Deshler, 2007). Although the measures are highly reliable and valid indicators of progress, they have been studied more frequently with elementary school children and their use with secondary students is not as well understood. Comparisons across methods for indicating inadequate response generally show poor agreement across different criteria and measures. These issues reflect the slightly lower reliability (and higher measurement error) of progress-monitoring assessments, the attempt to distinguish children toward the end of the achievement distribution where there is less margin for error, and variations in methods, cut points, and other issues. Moreover, not all domains of achievement can be directly or adequately measured with probe assessments (e.g., reading comprehension, written expression), although curriculum-based measures certainly predict these domains.
Establishing Low Achievement
Measuring achievement levels with norm-referenced tests does not solve issues related to the reliability of ascertaining children’s position in the achievement distribution, but do provide additional criteria in relation to the issue of whether the student is underachieving in one of the eight domains of IDEA, thus addressing the need for multiple criteria. Such assessments also provide guidance for intervention. At the same time, there is no point in assessing all eight domains of IDEA if the nature of the achievement problem is easily established. This component can be brief and based on hypotheses about the nature of academic impairment.
In terms of other disabilities and contextual factors that lead to low achievement and inadequate response to instruction, assessments proceed as they would if the child was not in an RTI framework. IQ and adaptive behavior assessment would be needed if there was concern about an intellectual disability or an autism spectrum disorder. The assessment of limited English proficiency and speech and language requires different measures and professional expertise. Behavior rating scales from parents and teachers should be completed routinely as screening measures for comorbid disorders (e.g., ADHD) and other contextual factors that may explain low achievement, especially in formulating a treatment plan. However, not every child needs to be assessed for every potential problem; in an RTI model, there should be hypotheses about the basis for the achievement problem that will lead to assessments specific to the child and to an intervention plan that is individualized.
Altogether, evidence that an identified disorder leads to adaptive impairment (i.e., educational need) must also be considered since disability determination always has these two prongs. In an RTI model, adaptive impairment is determined first (i.e., evidence that the child does not achieve at grade level despite quality instruction). In other identification approaches, the assessment of educational need has been somewhat subjective and partly responsible for the confusion that emerges when an interdisciplinary team denies eligibility despite a diagnosed disorder that sometimes, but not always, interferes with school performance.
ISSUES WITH IDENTIFICATION EMERGING FROM RTI MODELS
Most of the issues with identification in the context of RTI are universal and reflect the dimensional nature of LD (i.e., there are no qualitative diagnostic markers that indicate presence or absence of LD) and measurement issues involving the reliability of tests and ascertainment of cut points (Francis et al., 2005). RTI models do not “fix” these long-term problems, although their implementation may lead to new approaches to these problems. Instructional response likely exists on a continuum where the severity of academic impairment discriminates between adequate and inadequate responders (Vellutino et al., 2006). Whether the issue is defining low achievement or inadequate response to intervention, reliance on cut scores to indicate group membership (e.g., LD or not LD) will lead to arbitrariness in the identification model. There is little difference in children who cluster around the cut point, and measurement error will cause children to move above or below a cut point on repeated assessments (Francis et al., 2005). There is nothing unusual or unique about this problem and the same issues could plague medical disorders that are essentially dimensional (e.g., obesity or hypertension). The difference is that these medical disorders are not usually expressed in terms of absolute cut points, but understood in terms of the probability of a particular outcome at a particular level based on multiple criteria. Another alternative would be to use confidence intervals around a cut point that take into account measurement error, as is done with intellectual disabilities. Regardless, LD is not an all-or-none phenomenon and the field needs to move beyond cut points toward informed decision making based on multiple criteria where the reliability and efficacy of decision making by interdisciplinary teams becomes a major focus. In addition, we need a better understanding of what constitutes adaptive impairment in achievement and how low achievement and LD are related to more functional outcomes. Moving toward Bayesian models that express LD as a probability statement instead of a category and that take into account the underlying prevalence of positive identifications in the assessment environment may be a stronger approach than the current preoccupation with test scores and cut points. In the end, the decision will be categorical, but dealing with this question with cut scores on tests that don’t account for the dimensional nature of LD and the measurement error of the assessments are long-term problems for any identification model.
RTI models also carry issues that have a specific impact on identification, including what constitutes inadequate instructional response, how to assess high-quality instruction, the amount of intervention needed to indicate intractability, and more research on the role of curriculum-based measures in the identification process (which measures at what grade levels for what domains and what cut points). These issues need to be embraced through research (Fuchs & Deshler, 2007).
The most commonly proposed alternative to identification based on RTI is an approach incorporating an assessment of strengths and weaknesses in cognitive processes. In a recent consensus paper, Hale et al. (2010) argued for a third approach to identification that “identifies a pattern of psychological processing strengths and deficits, and achievement deficits consistent with this pattern of processing deficits” (p. 2). The paper indicated that patterns of strengths and weaknesses (PSW) were essential for identification and treatment planning especially after two tiers of intervention, indicating that there was a research base supporting both uses of cognitive discrepancies. The paper also outlined multiple issues with identification of LD in a “stand-alone” RTI model, including a) the lack of compliance with statutory requirements for an assessment of psychological processes, b) multiple issues with the implementation and research base of RTI models, c) higher achieving children with PSW who struggle and may not be detected, d) issues with methods for identifying inadequate responders, and e) the absence of a “true positive” for LD.
In examining the premises of the consensus paper, the point at which referral would be made (i.e., before Tier 3) would likely represent a base rate condition in which the number of true positives is higher than in traditional referral settings, so the capacity of cognitive tests to improve on identifications derived from base rates alone is unknown and likely difficult. Whether cognitive process assessments can be used for treatment planning and differentially interact with different interventions has long been disputed. A recent review by Pashler, McDaniel, Rohrer, and Bjork (2009) did not identify evidence that interventions based on Group × Treatment interactions (e.g., learning styles, Aptitude × Treatment interactions) were differentially related to outcomes. Moreover, the cost of such assessments is significant. Cognitive processes are related to LD, but there is little evidence that assessment of these processes adds information beyond that obtained from assessments of achievement, the latter clearly related to intervention.
There is no requirement or even possibility of adhering to statutory regulations in any route other than the regulations that implement the statute. The IDEA regulations for LD have never required an evaluation of cognitive processes. In fact, the guidance to IDEA 2004 provided by the Department of Education (http://idea.ed.gov/explore/view/) is quite explicit in not supporting this identification model or these types of assessments. Note that the statutory definition is focused on the “manifestations” of the “disorder of psychological processes,” which is why the IDEA regulations specify 8 domains (listening, speaking, reading, etc.) in which LD can occur.
The concern about higher performing children who are missed by RTI models ignores the fact that children can be referred at any time for evaluation for any kind of assessment. However, the field has yet to establish reliable criteria for identifying these children, especially when the intercorrelations of cognitive and achievement tests and the measurement difficulties in establishing discrepancies at higher levels of achievement are considered.
In terms of the multiple issues with stand-alone RTI models, RTI is not a diagnostic enterprise, but rather a service delivery model that yields data relevant to identification for special education (Fletcher & Vaughn, 2009). The issues with cut points and determination of responder status, and related issues, are not unique to RTI models, but also affect the use of cognitive tests in determining LD status, discrepancies in cognitive skills, and what constitutes a strength or weakness, which has received little evaluation. Most important, the “true positive” issue is one that affects any identification model. An RTI model can absolutely specify criteria for LD and evaluate the reliability and validity of the approach and the accuracy of decision making relative to these criteria. However, because of the dimensional nature of LD, such decisions are always relative to the definition and criteria used to establish LD. These designations are inherently arbitrary for any dimensional disorder. A PSW approach solves none of the identification problems identified in this paper and may make them more significant because of the use of discrepancy scores.
The RTI framework is not an identification model for LD, but it yields data relevant to identification, particularly in determining instructional response. Contemporary definitions of LD use multiple criteria and include assessments of instructional response and low achievement as inclusionary criteria. Other disabilities and contextual factors that have an impact on achievement are also assessed as exclusionary criteria that explain inadequate instructional response and low achievement. A comprehensive evaluation is required to assess these factors. As such, RTI models lend themselves to implementation of this definition of LD; other identification models must collect data on instructional response or add assessments that are not strongly related to identification or treatment. Many of the issues related to identification in an RTI model are universal, affecting any identification model and reflecting the dimensional nature of LD and the measurement error associated with measures used to operationalize LD identification criteria. Researchers and practitioners may need to more fully embrace the dimensional nature of LD and begin to move away from fixed cut points and categorical decision making based on tests toward decision-making processes that account for measurement error and are related to functional outcomes and adaptive consequences associated with LD and its identification and treatment.
Bradley, R., Danielson, L., & Hallahan, D. P. (Eds.) (2002). Identification of learning disabilities: Research to practice. Mahwah, NJ: Erlbaum.
Fletcher, J.M., & Vaughn, S. (2009). Response to intervention: Preventing and remediating academic deficits. Child Development Perspectives, 3, 30-37.
Francis, D.J., Fletcher, J.M., Stuebing, K.K., Lyon, G.R., Shaywitz, B.A., & Shaywitz, S.E. (2005). Psychometric approaches to the identification of learning disabilities: IQ and achievement scores are not sufficient. Journal of Learning Disabilities, 38, 98-110.
Fuchs, D., & Deshler, D.K. (2007). What we need to know about responsiveness to intervention (and shouldn't be afraid to ask). Learning Disabilities Research & Practice. 20, 129-136.
Hale, J., Alfonso, V., Berninger, V., Bracken, E., Christo, C., Clark, E., … Yalof, J. (2010). Critical issues in response-to-intervention, comprehensive evaluation, and specific learning disabilities identification and intervention: An expert white paper consensus. Learning Disability Quarterly, 33(3), 223–236.
Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2009). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9 (3), 105-119.
Vellutino, F. R., Scanlon, D. M., Small, S., & Fanuele, D. P. (2006). Response to intervention as a vehicle for distinguishing between children with and without reading disabilities: Evidence for the role of kindergarten and first-grade interventions. Journal of Learning Disabilities, 39(2), 157-169.
Copyright © 2010 by Jack M. Fletcher, Ph.D. Used with permission.
Back To Top