RTI and Universal Screening

September 11, 2008 1:00 PM - 2:00 PM • Joseph R. Jenkins, Ph.D., Marjorie Lipson, Ph.D.

About this Talk

Join Joseph Jenkins, Professor of Special Education at the University of Washington, and Marjorie Lipson, Professor of Education at the University of Vermont, as they discuss the importance of universal screening and how it is used under an RTI framework while answering your questions about RTI and Universal Screening.



Transcript

Mary Jo Rose
What measures are recommended for universal screening 3 times a year of preschool through 5th grade students in Math?
Joseph R. Jenkins, Ph.D.

A good resource for math screening, "RTI and Math Instruction," can be found on the  RTI Action Web site.

Research on first grade math screens has been recently reported in:  Fuchs, L.S., Fuchs, D., Compton, D.L., Bryant, J.D., Hamlett, C.L., & Seethaler, P.M. (2007). Mathematics Screening and Progress Monitoring at First Grade: Implications for Responsiveness to Intervention. Exceptional Children, 73(3), 311-330.  To predict later math disabilities, the strongest screens consist of CBM Concepts and Applications probes along with Computation probes (MBSP: Monitoring Basic Skills Progress: Basic Math Kit, Second Edition ISBN: 978141640382).

Liliana Isoe
Within special education in the high school, how could we monitor the students progress? I assume the progress monitoring should be at their functioning level, for example a 15 year old reading in the third grade should be monitor with this grade range.
Joseph R. Jenkins, Ph.D.

With oral reading or maze, the convention is to monitor progress on the highest level of text (i.e., graded text) on which students read with 90% accuracy or better.

Michael Patterson
Who provides the best universal screening today? How important are scales scores like those used by Northwest Evaluation Association (NWEA)?
Joseph R. Jenkins, Ph.D.

NWEA states that their use of equal interval (Rasch) scaling allows for accurate assessment of students' achievement levels. This scaling approach has good potential for use in screening and progress monitoring. Although I'm not familiar with the NWEA tools, the reliability and validity data reported on their website are impressive.

Marjorie Lipson, Ph.D.

As Joe has indicated, the NWEA assessments (I am most familiar with MAPS) can provide accurate achievement data for school districts. Especially for districts with large numbers of struggling students, the computer-based assessments facilitate more frequent assessment. On the other hand, the authenticity of the tasks is sometimes questionable. Although these assessments can be helpful in making monitoring judgments (how are we doing) for schools and districts, the scaled scores do not necessarily provide "value-added" for screening in an area like reading because the various components may not be identified discretely enough.

For example, I could miss a "difficult" item on the MAPS test because I did not understand the task - although I may have read the materials accurately. In other words, the domain of reading isn't just about more or less difficult items, but rather about areas of strength and difficulty. That said, these assessments can be used to identify students who may require a closer look since they do tend to be correlated with other achievement measures.

Jill Crawford, MSN, CPNP
Who is doing the universal screening and what tools are being used?
Joseph R. Jenkins, Ph.D.

Ordinarily, schools determine the individuals who conduct screening (e.g., classroom teachers, special educators, speech and language professionals, para-educators). Various tools are used for screening (e.g., teachers' ratings of students, performance on readiness, standardized achievement, state standard tests, or various commercial screens like those provided by DIBELS, Edcheckup, Aimsweb). Informal assessments (e.g., letter naming, letter naming and/or letter-sound fluency, phonological blending and segmenting, word identification fluency, informal reading inventories) can also be used.

Jill Crawford, MSN, CPNP
IS universal screening now mandatory and at what ages?
Joseph R. Jenkins, Ph.D.

Universal screening is only mandatory if your State or School District requires it.

Yael Cohen
I am already seeing "RTI" be used when it should really be special education. How do we get around that?
Joseph R. Jenkins, Ph.D.

In IDEA 2004, RTI is posed as an alternative to ability-achievement discrepancy approaches in identifying students with LD. In general, RTI has two functions:

  1. Prevent long-term academic failure through early primary and secondary prevention; and
  2. Identify those students whose unresponsiveness to primary and secondary standard interventions (that are validated and well implemented) may indicate a disability.

In RTI, special education is a tertiary intervention (more intense and individualized intervention) provided to students who are insufficiently responsive to primary and secondary intervention.

Melissa Sinunu
What is the official definition of a Universal Screener?
Joseph R. Jenkins, Ph.D.

Universal screening refers to a process in which all students are assessed to identify those at risk for failure (e.g., achieving an unsatisfactory outcome on a standardized or state mandated reading test). In practice, universal usually refers to a grade level (e.g., all students in a specific grade are screened).

The purpose of screens is to predict an outcome months or years in advance. The first step in creating a screen is to define the future outcome the screen seeks to predict (e.g., unsatisfactory reading ability). "Future" is relative, ranging from several months to several years and marked by specific points in the school curriculum (e.g., end of Grade 1, 4, 8, 12). Reading screens attempt to predict which students will score poorly on a future reading test (i.e., the criterion measure). Some schools use norm-referenced test scores for their criterion measure, defining poor reading by a score corresponding to a specific percentile (e.g., below the 10th, 15th, 25th, or 40th percentile). Others define poor reading according to a predetermined standard (e.g., scoring below "basic") on the State's proficiency test. The important point is that satisfactory and unsatisfactory reading outcomes are dichotomous (defined by a cut-point on a reading test given later in the students' career). Where this cut-point is set (e.g., 10th or 40th percentile) and the specific criterion reading test used to define reading failure (e.g., a state test or SAT 10) greatly affects which students a screen seeks to identify.

The second step in creating a reading screen is identifying early predictors of later reading outcomes. To be effective, a reading screen must be sensitive to different levels of reading development. In kindergarten, children develop phonemic awareness, letter and sound knowledge, and vocabulary. In first and second grades, they grow in phonemic spelling, decoding, word identification, and text reading. At higher grades, they gain in ability to comprehend increasing difficult texts. Thus, screening measures valid for beginning first-graders (e.g., word identification fluency) differ from those valid for kindergarten (e.g., letter naming fluency) or second-grade students (e.g., oral reading skill).

The third step in designing a reading screen is to determine a cut-point on the screening measure(s) that identifies students at-risk for failing the future criterion test. To identify a cut point researchers work backwards - first selecting students who failed the criterion (later) reading measure, then identifying the score on the screening measure that best distinguishes those students from the students who passed the criterion measure. This cut-score is then used to screen subsequent groups.

Marjorie Lipson, Ph.D.

Joe’s definition of universal screening is very comprehensive. As he has pointed out, the nature of the screening measure would vary depending on the age/grade of the students. As a practical matter, I would add that, in addition to predicting future performance, universal screening serves to identify students who are not presently meeting the standards and who, as a result, require immediate attention. This would generally mean additional (diagnostic) assessment, monitoring, and/or intervention. The McKenna & Walpole (2005) visual below is helpful is envisioning how this might look.

McKenna & Walpole (2005)
Figure from McKenna, M.C., & Walpole, S. (2005). How well does assessment inform our reading instruction? The Reading Teacher, 59(1), 84–86. http://dx.doi.org/10.1598/RT.59.1.9. Reprinted by permission of the International Reading Association.

When RTI is being used from the very earliest grades, prevention is the ideal. In the meantime, most schools have students who are already struggling. In these cases, universal screening is used for two purposes:

  1. To identify students who are presently at/above, near, or below the standard or cut point on the criterion measure and,
  2. To demonstrate the nature and size of the school/district problem (i.e., what percentage of students fall into each category).
Scott Fry
In schools that have successfully implemented universal screening, who is conducting the universal screening measures (gen. ed. teachers, reading specialists, etc.)?
Joseph R. Jenkins, Ph.D.

Ordinarily, schools determine the individuals who conduct screening (e.g., classroom teachers, special educators, speech and language professionals, para-educators).

Lews Bonney
What is the role of school psychologists in universal screening?
Joseph R. Jenkins, Ph.D.

School psychologists can be of assistance in assuring that screening assessments are properly administered and scored. In addition, many school psychologists have the skills required to identify screening cut points that are associated with target outcome measures used in their school district.

Bobbie Miller
What are some examples of universal screeners commonly used at the elementary level?
Marjorie Lipson, Ph.D.

This question was answered, in part, by the response to Anonymous ("What universal screening instruments (K-5 and 6-8) would you recommend?").

To go further, there are several very commonly used universal screening instruments for the early elementary grades. Perhaps the most widely used at this time is DIBELS. It is quick, easy, and is supported by excellent technology. However, it is also controversial because many educators worry that some subtests establish an inappropriate instructional target. At the same time, not all subtests of the DIBELS appear to be equally valid or reliable. The Oral Reading Fluency subtest, a one-minute test of oral reading that results in a WCPM score is likely to be useful to many. Teachers/schools who already have a text-leveling system and procedures for collecting records of oral reading have often opted to use these as screening tools instead.

Another widely-used assessment is the Phonological Awareness Literacy Screening (PALS) which includes a wide array of subtests useful for primary-grade teachers. Many schools/districts use a test of developmental spelling as well (see, for example, Barr, Blachowiz & Buhle, 2004 for the Illinois Snapshot of Early Literacy – ISEL.

Lisa Ray
What assessments do you recommend for higher-level language/thinking for students? At this time my school is currently switching to RTI instructional practices, but are very far from using this effectively to "respond" to students who are not responding.
Marjorie Lipson, Ph.D.

Although there are assessments that tap higher-level language/thinking for students, they are not "efficient" screening instruments. These assessments are most likely to provide revealing information when they involve reading longer texts (informal inventories such as the QRI-4; DRA, grades 4-8 for example, provide challenging informational texts and demanding responses).  As a result, they are difficult to use universally.  On the other hand, teachers may not need to assess all students in this way.  A combination of challenging state assessments (where appropriate) and an efficient screener like the Gates MacGinitie Reading Tests, can flag those students who require further assessment.

We cannot know exactly why a student did poorly on these assessments (see Valencia & Buly, 2005)  There are several reasons why a student may not perform well on challenging reading tasks and diagnostic assessment is more feasible if you can limit the number of students through effective screening.

Lisa Ray
What are you using for screening of reading at the secondary level?
Joseph R. Jenkins, Ph.D.

There has not been much attention given to developing screens for the secondary grades. By the time most students have reach secondary school, those who are at risk have already been identified by an earlier screen or by poor performance on an achievement test. For schools that administer standardized achievement test or state standards test, scores from these assessments can used to identify students at risk.

Bobbie Miller
Can you talk about developing local norms for CBMs and/or benchmarks?
Joseph R. Jenkins, Ph.D.

I'm convinced screening instruments should be "locally validated" according to the future measure used to judge adequate/inadequate reading outcomes. Schools (and states) can validate the screening cut points on CBM and on other types of screens that predict adequate/inadequate reading levels on the local criterion outcome measure (e.g., state standards test). Because these outcome measures (and the score used to distinguish between adequate and inadequate performance) differ by state and school district, schools should evaluate screening measures (and screening cut points) in relation to the specific (local) outcome measure.

Cyndi
I understand the concept of relating the progress monitoring tool to the universal screener. Would you suggest using Item Response Theory (IRT) to achieve this alignment?
Joseph R. Jenkins, Ph.D.

Using IRT to align screens with outcome measures has the advantage of ordering and reducing the number items that students must respond to on a screen. Tindal’s (University of Oregon RTI model) EasyCBM monitoring system is IRT based.

*Gerald Tindal is the Director of Behavioral Research and Teaching (BRT) in the College of Education at the University of Oregon.

Anonymous
What universal screening instruments (K-5 and 6-8) would you recommend?
Marjorie Lipson, Ph.D.

Schools and school districts may want to think about a multi-layered approach to assessment in the same way that they think about their multi-layered interventions. The universal screen should check the status of students’ (e.g.) reading achievement and performance. The cut score should indicate students who appear to be at little or no risk and students who are clearly at risk. As well, they will want to consider students who are not clearly below standard but who appear to be very vulnerable. Then, teachers/schools should take a closer look at those in the latter two categories.

In our school improvement projects here in Vermont, we mandate a closer look at all students who fall below the 40th percentile. Generally this closer look involves additional (and multiple) assessments. Sometimes, however, a review of existing information and recommendations is sufficient. Others suggest coupling the screening results with a 5-week period of good classroom instruction plus close monitoring before further action is taken (Compton, Fuchs, & Fuchs, 2006). The important point is that we take care with the selection of our screening instruments and that we do not rely too heavily on just one tool.

Now, what do I recommend? I will speak to reading only. Careful consideration should be given to the developmental demands of various ages/grades. For most schools, screening tools and procedures should be considered separately for K-2; 3-5; and 6-8. In the earliest grades, there are four key elements to consider: phonological awareness, alphabet knowledge, concept of word, and grapheme-phoneme correspondence or decoding. One tool that examines all of these is the Phonological Awareness Literacy Screening (PALS) (see Invernizzi et al, 2005). Many schools use DIBELS at the earliest grades, although it appears that only the Oral Reading Fluency subscale is essential for predicting subsequent reading comprehension (see "The Relation Between DIBELS, Reading Comprehension, and Vocabulary in Urban First-Grade Students," Reading Research Quarterly, 42, 546-567.).

Importantly, by grade 2, vocabulary becomes an exceedingly important factor and there are not enough good screening tools for this essential element. The PPVT-4 can be used, but not repeatedly. As an alternative, we have found the Gates MacGinitie Reading Tests very useful in grades 3-5. As a universal screening instrument it has the merit of including separate subtests for vocabulary and comprehension and of assessing the major outcome of interest – can students read and understand at an appropriate level for their age/grade. If the answer is no, then we pursue the matter by looking at likely factors such as phonics ability, fluency, etc. Fluency checks assessing WCPM and assessments of comprehension using longer passages and informal inventories all provide excellent instructional direction and allow teachers to monitor performance over time.

In grades 6-8, the Gates MacGinitie Reading Tests are useful, but schools may want to add information from on-going assessments of fluency (where appropriate) and also content-area reading checkpoints. The assessment, SSSMART (Reading in the Content Areas: Science, Social Studies, Math) includes survey tests with informational text and authentic content tasks. For longer, diagnostic assessments of reading comprehension and strategy use, the informal Developmental Reading Assessment, Grades 4-8 is also useful. Some schools use these relatively labor-intensive assessments 3 times a year as both screening and monitoring tools.

Marcia Dean
What is the rationale for conducting benchmarking three times a year, especially for those students who are performing at or above the benchmark?
Marjorie Lipson, Ph.D.

The three times a year practice makes a great deal of sense in the earliest grades when students' performance can be so volatile. Fuchs and their colleagues, for example, note that waiting 5 weeks (while students receive good classroom instruction) and re-assessing cuts the number of "false positives" in half (Fuchs & Fuchs, 2007).  Many students simply need the opportunity to learn in a high-quality environment. Frequent assessment can help us to re-consider early decisions and inform instruction.

Although "false positives" are more common in the early grades than "false negatives," this too can happen. A student who has appeared to meet the standards can begin to falter. This is actually fairly common during grades 3-4, as the benchmarks can change dramatically (shifting from criterion measures that focus on word-level competence to comprehension on text-level measures).

As students move into the intermediate and middle grades, universal screening of all students can be daunting – especially if the screening measures are appropriately challenging. Although we do not expect older students’ achievement to change as dramatically, we still need to assure ourselves that students are making good progress. Many schools create an “assessment matrix” or “assessment plan” that includes multiple sources of information about all students. Students who have met the standards complete some aspects of this matrix as part of an on-going monitoring system. In some cases, the identical screening measure is administered three (3) times a year.

Common examples include a phonics inventory (e.g., McKenna & Stahl, 2003) or a developmental spelling inventory such as the Words Their Way inventory by Bear, Invernizzi, Templeton, and Johnston (2004) or the Developmental Spelling Assessment by Ganske (2000). Other assessments are more curriculum based and may include mid-year comprehension assessments using leveled texts, written summaries that are scored with agreed-upon rubrics; or fluency assessments using classroom curriculum materials.

Layla Martinez
What sorts of adjustments should we be making for students whose native language is not English?
Marjorie Lipson, Ph.D.

I encourage you to look at the transcript of an earlier NCLD chat "Challenges of Early Literacy for English Language Learners" (March 29, 2005). This should be helpful in thinking about the special challenges of these students. In addition, you may want to read Lori Helman's terrific article in The Reading Teacher, "Using literacy assessment results to improve teaching for English-language learners." In that article, she demonstrates how data from early screening instruments (in this case PALS) can reveal the gaps between English-only and English Language Learners (ELL) students. More importantly, she uses cases of English-language learners to demonstrate how screening information is used to make decisions about core instruction and also about additional support. She highlights the ways in which early identification can be helpful in tailoring instruction for English learners in the early years.

Helman's examples involve a context with Spanish speaking students. In our state, there is no dominant language, but some schools have very large numbers of ELLs and also a wide array of languages. We have had good success with a vigorous combination of: early ELL support from a specialist; intensive high-level classroom instruction that makes continuous use of assessment data; and additional intervention (double-doses) when necessary. With this combination, many students are meeting the state standards by grade 4.

However, English Language Learners need continuing support. Like Valencia and Buly (2005), we have found that a close examination of screening data shows that many ELLs are hovering right on the cut-score. Their acquisition of skill is more precarious than some other students and requires continuing monitoring. Many of these students continue to meet high standards for oral reading accuracy and for fluency, but as these students move into grades 4-8, they often need additional intervention in the areas of vocabulary and comprehension. This highlights the important of screening tools that are multi-faceted and developmentally appropriate.

a worried parent on Long Island (NY)
My son's school (he's in 5th grade) did some screening at the end of last year and the beginning of this year, and I was just told that he didn't 'achieve benchmark' the way most of the other students did. I understand that the school is trying to do something different (not rushing to refer kids for special education testing) and I'm all for trying to provide my son the help he needs without going through a huge evaluation. My worry is that if he is behind and needs the kind of help that only special education can provide, I don't want to wait! Can you explain how this benchmarking process works, how screening can be helpful, and how I will know when it would be best to formally request a comprehensive evaluation? Thank you!
Marjorie Lipson, Ph.D.

In using Response to Intervention, your son's school has several goals. One of them is, indeed, to avoid the rush to special education identification. The other is to provide appropriate support and intervention for students. This may lead to a comprehensive evaluation, but that may not be required. It's a good thing to be very involved in this process (RTI calls for the active involvement of parents).

If the screening process has identified your son as below the benchmark, you should ask the school a couple of questions. First, find out what they plan to do next. What does the core instructional program look like and how will it be differentiated to increase the intensity of instruction for your son. His progress will be monitored and you should also ask when the next assessment(s) will occur. You will want to be informed of the results from that next assessment. If your son has not made good progress at the next checkpoint, the school will probably be thinking about introducing even more intense intervention procedures. Ask to be informed of all these decisions.

Julie Coffee
How is this incorporated in the everyday classroom where so much pressure is put upon us to complete all of the standards before achievement testing. My class needs this so much, but I don't know where to begin.
Joseph R. Jenkins, Ph.D.

Universal screening for RTI is a school-wide endeavor. RTI requires collaboration between classroom teachers and other professionals, and can't be merely added as another task for classroom teachers. Some screening measures are brief enough that teachers can and will want to administer them. Other screens may require help from colleagues (para-educators, Title 1 teachers, special educators, school psychologists). In RTI, school level teams work together to determine how screens are administered and how the results are used.

Blanche Deren
Please describe the follow-up process after a screening has been implemented.
Marjorie Lipson, Ph.D.

You are asking a question that gets at the heart of the entire RTI process. Michael McKenna and Sharon Walpole (2005) have created a model of assessment-driven reading instruction (see response to Melissa Sinunu). What happens next depends a great deal on what the screening shows. In all cases, however, the idea is to ensure that the student is receiving the most appropriate instruction possible. This may involve only classroom instruction (with monitoring) or it might involve classroom instruction plus additional intervention. As progress is monitored, additional adjustments would occur until a determination was made that the child should be considered for the most intense intervention through special education.

Alicia
What if the parent(s) want their own testing done during the RtI process? What if they don't agree with the school's assessment findings?
Joseph R. Jenkins, Ph.D.

Parents always have the right to have independent tests done. RTI does not remove this right. When parents disagree with the schools assessment results, they should meet with school representatives to try to discuss the different results. Most differences in test results can be reconciled through discussion.

If parents suspect that their child has a disability and requires special education, they are entitled to due process under federal law.

Cathy Roller
What are the advantages and disadvantages of Informal Reading Inventories for screening?
Joseph R. Jenkins, Ph.D.

The advantage of Informal Reading Inventories is that they provide more information than do typical screening measures, especially in regard to students' reading strategies and comprehension. The drawback of IRIs as universal screens is that they require substantial time to administer. An IRI could be a good follow-up assessment for students who are on the risk borderline, but again time is the issue.


Many thanks to Drs. Joseph Jenkins and Marjorie Lipson for sharing their expertise and to the many participants for their thought-provoking questions.

For more information on Response to Intervention, visit the RTI Action Network Web site.


Related Reading from RTINetwork.org:

 


Additional Online Resources: