RTI Action Network
Print

Implementing a Combined RTI/PBS Model: Social Validity

By: Juli L. Pool, Ph.D., Evelyn S. Johnson, Ed.D., and Deborah R. Carter, Ph.D.Published: June 8, 2010
Topics: Implementation Planning and Evaluation, Leadership


Recent Entries

Recent Comments

Contributors

Topics

Acquiring teacher buy-in and positive teacher perception are vital to the long-term sustainability of any systems change implemented in schools. In our last blog, we discussed some challenges with teacher perceptions and buy-in for the implementation of a comprehensive RTI/PBS model at Silver Sage Elementary. We (the researchers) did not want to come into this school and begin an implementation process that would just fade away when our two-year grant ended. What would be the point in implementing a new system of data collection, using decision rules, providing tiered interventions, and generally asking for the staff to accept a massive amount of system changes for it all to go by the wayside when the project ended? If the process is not sustainable when we leave, we have basically caused a lot of upheaval in their lives, and ours, for no reason.

Therefore, of particular interest to us are the ways that the comprehensive model and process is viewed by the teachers and staff and the degree to which they accept it. Teacher buy-in starts (and ends) with whether or not they see the process as being valid. Wolf (1978) describes social validity as validating our work on three levels: (1) the social significance of the program, (2) the social acceptability of the procedures, and (3) the social importance of the effects and outcomes. It is important to monitor social validity because the practice of doing so ensures that the culture and climate of the school, its unique needs and resources, and teacher’s values are all a part of our work. We continually check to see if what we are doing is meaningful and acceptable to the school staff by conducting quarterly focus groups and collecting social validity survey results. We know that social validation is a critical component of a tiered process that directly impacts the fidelity and sustainability of implementation (Gresham, 2004). When goals and procedures are acceptable to those implementing the process (e.g., teachers, staff, administrators), there is a much higher likelihood that they will be implemented (and sustained) as intended.

 

As we shared in the last blog, the teachers and staff at Silver Sage began the project with high expectations and enthusiasm. However, once the project got underway and the process of change began, perceptions took a negative turn. In our February focus groups, the following themes emerged as obstacles to implementation: (a) reluctance to "get on board," and (b) practical considerations (i.e., procedures). Despite these identified obstacles, however, both groups (leadership team members and other staff) indicated they feel that implementation is, or will be, beneficial and student outcomes will improve because of this process. In the following paragraphs we will discuss teachers’ perceptions as they relate to Wolf’s three levels of social validity.

 

Reluctance (Social significance of program). Teachers have shown a sense of resistance or hesitancy to the process as well as wariness about the changes required that has raised issues about buy-in. The focus groups and the social validity surveys have revealed that the teachers have taken a passive stance toward the process of "time will tell." Teachers are "waiting" for time to show whether this is just a passing initiative that is tied to their current administrator or true systems change. At the same time, teachers have shown a more active distrust of and/or distaste for the procedures. For example, using data-based decision rules to identify students who would benefit from Tier 2 or Tier 3 support has been met with concerns about removing teachers’ perceptions from the equation.

 

Practical considerations as obstacles to implementation (Social acceptability of the procedures). The teachers at Silver Sage feel that implementation is challenging due to the RTI process being too time consuming and requiring an overwhelming amount of data. Over and over, they have referred to the process as requiring "loads/mounds/gobs" of data before a student receives the help they need and of being "oppressive." They have indicated that they feel that there is a lack of communication from leadership and across teams and that they are unsure of what data is needed and how much is sufficient to bring to the Problem Solving Team meeting. Because of these challenges, many said that they have avoided the process (and consequently, are hesitant to embrace it now).


Implementation as beneficial (Social importance of effects and outcomes).
One positive theme that emerged was that the teachers do feel that implementation of the model is, or will be, beneficial to the school and the students. For example, many teachers were glad to have a shared vocabulary and an increased structure for school-wide behavior support.


Figure 1
shows the three general themes and the issues within each theme that have arisen through focus groups and social validity surveys.

Untitled1
Untitled2
Untitled3



Figure 1. Focus Group General Themes

As procedures have been put into practice and become more routine, there have been more positive comments from the teachers. They have also begun to embrace the process and take a more active role in the decision-making meetings. This shift in perception can be attributed to student outcomes. The teachers have indicated that they are seeing positive changes and student improvement. The data collection procedures, while cumbersome, give the necessary information for decision-making. The teachers work as a team to review the data and decide on intervention needs of each student. The team decision-making process, as well as the student outcomes, drives the positive change in perception.


One teacher, who was particularly vocal and reluctant to get on board with the project in the beginning, has become our "go to" example of how perceptions are improving and buy-in is increasing. From the beginning, she dug in her heels, generally refused to take part in the process, and became the voice for all the teachers that had reservations and concerns. In team meetings, she seemed stand-offish and unreceptive towards us and the whole implementation process. After the universal screening at the beginning of the school year, she was one of the teachers that was very hesitant to identify students as needing a Tier 2 intervention, giving reasons why each student scored the way they had (e.g., "He has had a tough week at home"). About a month ago, she came to us and asked if she could see the data for a student because she felt that the student needed more targeted intervention. She wanted to see data! We have taken this as a symbol that change in perceptions is beginning to happen.


We will continue to monitor the three levels of social validity in order to make sure we are addressing teacher reluctance (e.g., Why are they reluctant? What can we do to help move them forward? Are we clarifying misunderstandings?) and perceived obstacles to implementation (e.g., too much data needed, time-consuming, overwhelming expectations of teachers). As we move into the second year of implementation, more emphasis will be put on the procedures implemented and how to ensure sustainability.


In our next blog, we will share where our project is headed as we wrap up the first year of implementation and move into the second year. We welcome any questions, comments and suggestions!


References


Gresham, F. M. (2004).  Current status and future directions of school-based behavioral interventions. School Psychology Review, 33, 326-343.


Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203-214.

Back To Top


Read what others had to say...





Visit www.rtinetwork.org for more information on this topic.
Copyright © 1999-2017 National Center for Learning Disabilities, Inc. All Rights Reserved.