PBIS - Positive Behavior Interventions and Supports
Program Evaluation Report
Group 4: Latasha McCord Kecia McGouirk Kristy Slaughter
FRIT 8435 Spring 2013 Georgia Southern University May 6, 2013
Group 4 - Page 2
Table of Contents
I. II.
III.
IV. V.
VI.
VII.
Executive Summary......................................................................................... Introduction to the report............................................................................... a Purpose of the evaluation......................................................................... b Audiences for the evaluation report......................................................... c Limitations of the evaluation..................................................................... d Overview of report contents...................................................................... Focus of the evaluation.................................................................................... a Description of the evaluation object.......................................................... b Evaluative questions used to focus the study............................................ c Information needed to complete the evaluation...................................... Brief overview of evaluation plan and procedures.......................................... Presentation of evaluation results................................................................... a Summary of evaluation findings................................................................ b Interpretation of evaluation findings......................................................... Conclusions and recommendations................................................................. a Criteria and standards used to judge evaluation object............................ b Judgments about evaluation object (strengths and weaknesses)............. c Recommendations..................................................................................... Appendices...................................................................................................... a Description of evaluation plan/design, instruments, and data analysis and interpretation..................................................................................... b Literature Review......................................................................................
3 4 4 4 4 5 5 5 7 7 8 8 8 9 13 13 14 14 15 15 18
Group 4 - Page 3
Executive Summary
The information in this evaluation report combined a literature review (www.pbis.org) with a case study of a central Georgia high school to determine the effectiveness of a schoolwide PBIS program in a public high school setting. It was found that the school adequately implemented the program and was able to decrease problem behaviors as well as increase academic achievement. However, this particular school does show a need for some minor adjustments in order to maintain the success of the program. The PBIS program implemented at the district level of ninth through twelfth grades in this report will demonstrate areas of improvement in students tardies, attendance, disciplinary behavior, and academic achievement. The study found that tardies and disciplinary referrals decreased somewhat during the year of PBIS implementation when compared to the same time periods the year before implementation. Absences also decreased overall in the second and third quarters of implementation. Furthermore, the number of first-time participants increased nearly 25 percent after the first quarter incentive was awarded, demonstrating that students who had not received a reward the first time improved their behavior and were able to participate in the second quarter reward. It is worth noting that the overall number of participants did not increase significantly in spite of the increase of first-time participants. For this reason, it is the recommendation of the evaluators that further study be conducted on what incentives may be motivational for high school students. The study should include a survey of student interests.
Group 4 - Page 4
Introduction
The purpose for this evaluation is to investigate the Positive Behavior Interventions & Supports (PBIS) program in a central Georgia high school to determine if the program is serving its intended purpose: to have a positive influence on student behaviors including being on time to school and to class, not receiving any disciplinary referrals, improving attendance, and passing all of their classes. The audience for this evaluation includes administration, teachers, students, parents and the Board of Education. These individuals will assist with the results of student attendance, grades, behavior from disciplinary reports, date of incentives, pros/cons of incentives and the outcome from both disciplinary reports and student participation utilizing SWIS (School Wide Information System) and PowerSchool. The utilization of such data will help answer stakeholders questions such as how well the program is functioning and what changes may be needed in order to improve it. The limitations of this evaluation include the short timeframe available for the study and the availability of access to peer-reviewed texts by experts in the field. There is an abundance of general research about positive behavioral interventions and supports, but there is less research on specific studies regarding the effectiveness of the program on academics and behavior. An additional limitation of the study was smaller amounts of literature on schools and districts implementing PBIS programs. This program evaluation is limited to one school system in central Georgia. Its results are intended to demonstrate the effectiveness in this one setting. The current report also reviews only available quantitative data. For a more thorough analysis,
Group 4 - Page 5
further qualitative data should be collected including surveys of all stakeholders. Students should be surveyed to determine if they feel the incentives that are offered are indeed motivating them to modify their behavior. Teachers, parents, and other stakeholders should be surveyed to see if they feel there has been a change in attitudes or behaviors since the inception of the program. This report contains a thorough description of the object of the evaluation, a description of the evaluation plan and procedures, a summary of the findings along with detailed charts, interpretations of the data, and the recommendations of the evaluators. Included in the Appendices are the charts containing the complete evaluation plan, as well as some additional diagrams that may help in the understanding of the PBIS program.
Focus of the Evaluation
Teachers and administrators have been utilizing the most effective ways to interact with students, support their learning, and help them become lifelong learners for years. Unfortunately, there is not a quick fix for every challenge that arises in schools across the country. However, certain practices have been shown to be more effective when used correctly. Positive Behavioral Interventions and Supports, or PBIS, (www.pbis.org) defines, teaches, and supports appropriate student behaviors to create a positive learning environment where problem behaviors are decreased and academic performance is increased (Appendix B). The object of this evaluation is the Positive Behavior Interventions & Supports (PBIS) program in a high school environment to determine if the program does indeed have a positive
Group 4 - Page 6
influence on student behaviors. PBIS is a data-driven, research-based intervention with the goal of improving behavioral and academic outcomes. PBIS comes directly from the 1997 reauthorization of the Individuals with Disabilities Education Act (IDEA). PBIS is used interchangeably with SWPBS, which is short for School-wide Positive Behavior Supports. (Appendix C). PBIS is an implementation framework that is designed to enhance academic and social behavior outcomes for all students by emphasizing the use of data for informing decisions about the selection, implementation, and progress monitoring of evidence-based behavioral practices (Appendix D). PBIS includes allowing choice and decisions about selfcontrol, personal goal setting, and additional independent choices. Encouraging and allowing self-determination and personal goals for all individuals in the school pushes students to become their own advocates and provides further reason for expanding their abilities and confidence. With the idea of making individual decisions, a student helps plan his or her own behavior goals that can be reached successfully and celebrated among peers and staff members through school-wide incentives. Several school systems around the state of Georgia have begun implementing PBIS programs into their school day. Students are rewarded for desirable behaviors such as getting to school on time, not having any tardies to class, receiving good grades on Report Cards, and not receiving any disciplinary referrals. For this report, the focus will be on using the quantitative data available from one school system in central Georgia. The school district is a rural school system located in the heart of Georgia near both Macon and Atlanta. The system includes six schools-- three elementary, two middle, one high school, and one alternative school-- and serves just over 4,000 students. The high school that is the focus of this evaluation serves 1086 students. The population is 68%
Group 4 - Page 7
Caucasian, 27% African-American, and 5% other ethnicities. Fifty-six percent of the students are classified as economically disadvantaged. The questions being evaluated include:
Do PBIS (Positive Behavior Interventions & Supports) programs help schools to increase desired behaviors including being on time for school and class, not receiving disciplinary referrals, not having excessive absences, and improving academic grades?
Is participation in the PBIS program growing, or are the same students receiving rewards each time? In order to evaluate the program, data concerning participation in the PBIS program was
collected from the high school in the district. The data includes the numbers of students who were eligible to participate in each of the three quarterly rewards during the 2012-2013 school year to determine if the program has grown in the course of the year. Numbers of students participating for the first time each quarter were also analyzed to determine whether the same students were being recognized each quarter or if more students were qualifying for rewards. For comparison, data was also collected and analyzed from the current year and the previous school year concerning students tardies to class and to school, absences, disciplinary referrals, and classes passed. Data was collected from the School Wide Information System (SWIS) and from PowerSchool records.
Group 4 - Page 8
Overview of Evaluation Plan and Procedures
To assess the impact of the PBIS Program in a high school setting, information detailing students tardies, absences, disciplinary referrals, and passing grades was collected through emails and system data analysis. Since the program has not undergone a formal evaluation in this particular school district, much of the assessment of the effectiveness of PBIS was circumstantial and based upon literature research and from the previous years data prior to PBIS going into effect. Time constraints and cooperation with administration played a factor in this evaluation. Due to anonymity reasons, administrators asked that we not copy and paste actual SWIS (School Wide Information System) and PowerSchool data in our report. Instead, we created our own bar charts based on the numbers received from administration. Therefore, quantitative data relating to PBIS was discreetly obtained from administration and examined through the generated reports from SWIS on students disciplinary referrals and generated reports from PowerSchool on students tardies, absences, and grades. In all, the quantitative data associated with PBIS was thoroughly assessed in this program evaluation.
Presentation of Evaluation Results
The results of this evaluation demonstrate that there was only a slight increase from the first quarter to the second quarter (from 596 to 619 students) in the total number of students receiving awards for having no tardies to class or to school and having no disciplinary referrals (see Figure 1 below). This represents less than a five percent increase. Furthermore, there was
Group 4 - Page 9
no increase in the total number from the second quarter to the third quarter. The number of students meeting the second tier goal, which also included having no absences for the quarter, actually decreased by nearly 30 percent each quarter. The numbers of students meeting the third tier goal, which includes all of the criteria for the first two, but also includes passing all of their courses, also showed a significant decline each quarter.
2012-2013 School Year
Number of students meeting Goal #1: Zero tardies to class Zero tardies to school Zero disciplinary referrals for the quarter Number of students meeting Goal #2: Met criteria for Goal #1 and Zero absences Number of students meeting Goal #3: Met criteria for Goal #2 and Passing grades in all classes Figure 1
1st Quarter
2nd Quarter
3rd Quarter
596
619
618
235
153
116
217
141
102
If the data in Figure 1 were all that was available, then it would be a logical assumption that PBIS was ineffective. However, upon further evaluation, several important factors can be observed. For example, 152 of the 619 participants in the second quarter were being rewarded for the first time. This indicates that there was nearly a 25 percent increase in first-time participants from the first quarter. An additional 66 students were first-time participants for the rewards in the third quarter. This substantial increase in new participants indicates that the PBIS rewards may have been a motivating factor for students to improve specific aspects of their behavior.
Group 4 - Page 10
Data analysis concerning specific target behaviors from the year prior to implementation of PBIS also supports the idea that some modest improvements may have occurred in behavior patterns during the implementation year. The figures below illustrate a slight improvement in almost every area each quarter compared to the previous year. Tardies to school, tardies to class, disciplinary referrals, and absences were consistently reduced, with the exception of the first quarter of implementation when student absences increased slightly (Figure 5). However, this irregularity may have occurred because students were not yet fully aware of the PBIS system and the rewards that were to come. One final chart demonstrates that the number of students passing all of their classes increased slightly over the previous year as well.
Number of Tardies to School per Day on Average
Tardies to High School Grades 9-12
50 40 30 20 10 0 1st Quarter 2nd Quarter 3rd Quarter 2011-2012 2012-2013 42 38 35 33
27 24
Figure 2
Group 4 - Page 11
Tardies to Class
Number of Tardies to Class per Day on Average
Grades 9-12
20 15 10 5 0 1st Quarter 2nd Quarter 3rd Quarter 2012-2013 2011-2012 12 10 18 15
14
12
Figure 3
Disciplinary Referrals
Grades 9-12
1,000 900 800 700 600 500 400 300 200 100
Number of Referrals
746 540 458
686
717
630
1st Quarter
Figure 4
2nd Quarter
3rd Quarter 2011-2012 2012-2013
Group 4 - Page 12
Absences Grades 9-12
2,000 Number of Absences 1723 1682 1,500 1438 1470 1789 1758
1,000 1st Quarter 2nd Quarter 3rd Quarter 2011-2012 2012-2013
Figure 5
Passing All Classes Grades 9-12
Number of Students Passing all Classes 1,000 900 800 700 600 500 1st Quarter
Figure 6
874 893 715 722
821 836
2nd Quarter 3rd Quarter 2011-2012 2012-2013
Group 4 - Page 13
Conclusions and Recommendations
The quantity of tardies, referrals, absences, and failing grades that occurred in the 20112012 school year compared to the 2012-2013 with PBIS in place determine that PBIS programs do seem to assist with the increase of desired behaviors. The preferred behaviors included being on time for school and for class, not receiving disciplinary referrals, not having excessive absences, and improving student grades. Additionally, the specific level of performance expected in order to show the criteria met are at least a 10 percent decline in tardies to class and to school, disciplinary referrals, absences, and failing grades. Though not every criteria met the 10 percent mark every quarter, the evaluators feel that the improvements are significant enough to continue the program for another year. Furthermore, in evaluating the growth of the participation and verifying if the same students are in receipt of rewards each time, the number of students awarded with PBIS incentives during the school year should definitely be considered. In order for this criteria to be fully successful, we felt there should be at least a 10 percent increase in the number of students eligible for rewards during the school year. This goal was greatly exceeded for first-time participants throughout the course of the year. Based upon the evaluation of program data relating to students problem behaviors and academic achievement of the current PBIS program, this evaluation study concludes the following: Program Strengths
Shows sufficient improvements in targeted areas
Group 4 - Page 14
Increases problem awareness Places a greater focus on students Places more emphasis on preventions Removes potential triggers of bully behavior Teaches effective, efficient, and relevant alternative social skills Establishes a consistent and fair reward system
Program Weaknesses
The principles and concepts of behavior management associated with PBIS can be complex and difficult to implement on a school-wide basis
Secondary and tertiary level interventions may need improvement Universal implementation of meaningful incentives for students who display desired behaviors
Time and labor intensive May be costly depending on the rewards offered
Recommendations It is the recommendation of this evaluation study that the implementation of the PBIS program justifies that the school district continue to do what is already showing some improvement. Throughout the current assessment it was deemed vital to pay close attention to the smallest changes that might produce larger effects, in particular with regard to the targeted aspects of student behavior. In addition, it was noted that it is highly important to avoid defining a large number of goals when considering implementing the PBIS Program. The current
Group 4 - Page 15
expectations concerning tardies, discipline, absences and grades are sufficient. Furthermore, other recommendations include collecting and utilizing additional qualitative data such as surveys of stakeholders for decision-making, and adapting any initiative which involves families, students, faculty, and fiscal structure. This thorough evaluation demonstrated that a new program should not be added without thorough research which includes defining the expectations of the program. As with any new program, the establishment of policy clarity should be considered before investing in any implementation.
Appendices Appendix A
Below is an evaluation plan of the PBIS program (Positive Behavior Interventions & Supports). PBIS is a data-driven, research-based intervention with the goal of improving student behavioral and academic outcomes. The school system under evaluation started using PBIS in the 2012-2013 school year. The evaluation compares current and previous school years PBIS results with grades 9-12 and to determine whether or not the program seems to be effective.
Planning the Evaluation Reporting:
Evaluation Questions
Audience for the Report
Report Content
Report Format
Reporting Schedule
Context for Presenting Report PowerPoint presentation to administration, faculty and the Board of
1. Do PBIS (Positive Behavior Interventions & Supports) programs help
Administration Teachers Parents Students Board of
Results of student attendance, grades, and behavior from disciplinary
Presentation to Faculty, Student/Parent Letter, Mass Email, Posted on
Quarterly
Group 4 - Page 16
schools to increase desired behaviors including being on time for school and class, not receiving disciplinary referrals, not having excessive absences, and improving academic grades? 2. Is participation in PBIS program growing, or are the same students receiving the rewards each time?
Education
reports using SWIS (School Wide Information System) and PowerSchool
District Website & Presentation to the Board of Education
Education. Onepage letter to students and parents, which entails the findings.
Administration Teachers Parents Students Board of Education
Date of incentives, pros/cons of incentives and the results of student participation using SWIS (School Wide Information System)
Presentation to Faculty, Student/Parent Letter, Mass Email, Posted on District Website & Presentation to the Board of Education
Quarterly
PowerPoint presentation to administration, faculty and the Board of Education. Onepage letter to students and parents, which entails the findings.
Summarizing the Evaluation Plan:
Evaluatio n Questions
Informati on Required
Design
Informati on Source
Method for Collecting Informati on
Sampling
Informati on Collectio n Procedur es Obtaining quarterly reports of SWIS data on student disciplinary referrals. arranged and conducted with Assistant Principal (AP), IC, and teachers.
Schedule
Analysis Procedure s
1. Do PBIS (Positive Behavior Interventio ns & Supports) programs help schools to increase desired behaviors including being on time for school and class, not receiving disciplinary referrals,
Number of disciplinary referrals issued each quarter for current school year and previous school year. Notice of decrease in student behavior issues and increase in students grades by teachers.
Interrupted Time Series
Administra tor, Instruction al Coach (IC), Teachers, and SWIS Reports
SWIS Data Reports
District High school students
Quarterly SWIS reports & in April
Descriptive stats from SWIS data reports. Summarize Integrate all results to describe PBIS outcomes.
Group 4 - Page 17
not having excessive absences, and improving academic grades? 2. Is participatio n in PBIS program growing, or are the same students receiving the rewards each time? Numbers of students participatin g in rewards each quarter Interrupted Time Series SWIS Reports SWIS Reports District High school students Obtaining quarterly reports of SWIS data Quarterly SWIS reports Descriptive stats from SWIS data reports
Reporting Procedures:
Evaluation Questions 1. Do PBIS (Positive Behavior Interventions & Supports) programs help schools to increase desired behaviors including being on time for school and class, not receiving disciplinary referrals, not having excessive absences, and improving academic grades? 2. Is participation in PBIS program growing, or are the same students receiving the rewards each time?
Interpretation Procedures Is there at least a 10% decrease in students tardies to class, tardies to school, disciplinary referrals, absences, and failing grades? If not, what solutions are recommended?
Audience(s)
Content
Format
Schedule
Administration Teachers Parents Students Board of Education
Results of student attendance, grades, and behavior from disciplinary reports using SWIS (School Wide Information System) and PowerSchool. Utilizing data will help answer questions: What is the program doing well? What changes are needed? Date of incentives, pros/cons of incentives and the results of student participation using SWIS (School Wide Information System). Utilizing data will help answer questions: What is the program doing well? What changes are needed?
Presentation to Faculty, Student/Parent Letter, Mass Email, Posted on District Website, & Presentation to the Board of Education
At the end of each quarter- October, January, March, and May
Is there at least a 10% increase in the number of students eligible for rewards during the school year? If not, what solutions are recommended?
Administration Teachers Parents Students Board of Education
Presentation to Faculty, Student/Parent Letter, Mass Email, Posted on District Website, & Presentation to the Board of Education
At the end of each quarter- October, January, March, and May
Group 4 - Page 18
Appendix B
http://www.pbis.org/common/cms/media/4Elements.jpg
Appendix C
Group 4 - Page 19 http://www.pbis.org/common/cms/media/ContinuumTriangle.jpg
Appendix D
http://www.cber.uconn.edu/wp-content/uploads/2011/11/PBIS-keynote-overview-1hr-Nov-9-2011POST.pdf
