you're welcome there we go so you are started you are um being recorded folks aren't seeing it just yet and we are starting to populate we've got already 20 people attending 20. they get seats at the front that's right all right there we go [Music] all right so we should be live all right well hello everybody um we are going to have a slow roll for a minute here and let everybody else come in through zoom we can see the numbers as they as they tick up we also know there was only about nine minutes between that that closing or that last session and right now so uh glad to have you here and we're gonna get started in about a minute looks like we should probably have one or two hundred folks joining us today just a great number and perfect um let me start off by um introducing myself i'm brian meyer i'm uh co-director for the midwest pbis it today be with you throughout the day on the getting started strand and uh you know this first session for using data for decision making and uh my co-presenter today is heather heather can you say hi hi everyone welcome oh heather george and i'm co-direct the national center on pbis as well as florida's pbis project and i'm super excited to have all of you here and um we've got a great strand set up for you let's go through some housekeeping stuff and uh that can give folks a few more minutes to log in here so uh for this strand today we're gonna spend the next 75 minutes focused on getting started in data you can help us help you by focusing on our expectations for the forum so these are to be responsible be kind and be safe really focus on the language you're using um for the in chat especially um i think that's where you demonstrate uh especially being uh kind would be great um you can find your sessions you're already here of course you guys know how to navigate passable already on if this is the first session that you're jumping into on the left-hand side you see the video on the right-hand side you'll see chat polls files and people and you can navigate our content there as well um questions can be entered in a poll you'll see that as well um and you also might put some questions in chat too we'll talk about that more um after this introduction be careful you might click and navigate away from the system if you do just go back to your sessions enter it again and you'll be back with us and if you have any technical issues it's hard for us to manage that with a couple hundred friends on um chat so um you can use that help desk link at the top and we have some folks standing by live to assist you and it looks like we're about three minutes in so let's talk about our norms what we're doing uh that's a little unique for this um for these strands are uh heather and heather and i invited um our colleagues diane brian and therese to um be chat panelists they have a lot of great experience uh with this work and so they're going to monitor chat in pathable or in our application here and as we get engaged with different ideas and content they're going to jump in and help answer questions and give some of their thoughts and ideas so you can help us by when you use chat especially if you're posing a question it would be great if you could include your district name and your state and maybe your position because that makes a difference if you're asking a question from a state team perspective or district team if you're a clinician or if you're a building administrator those can help us especially in the context of diane brian and teresa responding to you okay um so those are the ways and the norms we like to put in place for using chat let's talk about pbis the big ideas um i think it was don who kicked off this idea of using the bicycle as a as a metaphor for this work and it's amazing um you can have all of the parts that you need for your school and all the things that excite you like you see that bicycle on uh in pieces there you need to have a framework something to bring it all together and over the course of the day today we're gonna get into these topics this idea of um coaching the work or steering the bike um the interventions kind of where the the rubber meets the road doing the work with the kids um having the team who's sitting down who's making decisions uh the framework which is or the frame of the bicycle which is uh database decision making that'll make diane happy she likes any time you can say the word data school leadership are the gear are the uh the gears on the back and changing the the pace at which we move sometimes and then implementation what's driving all of this the pedals i love the metaphor it's perfect um and it's a good way for us to sort of get going today also you'll see um this idea that we've shared in our or sort of getting back to school guides which is um how can you create safe positive predictable learning environments um and what are the key components there so connect with your youth and your staff screen to figure out what you need provide support based on what you learned from that screening teach teach teach and be direct with your instruction then monitor so that's another way of looking at the work of pbis i don't know if anybody saw this on october 19th this was published from the us department of education and it was a formal publication multiple technical assistance centers provided input into it coming out of the office of special education programs and the publication is called supporting child student social emotional behavioral mental health needs and this is a milestone publication um it is worth every 100 pages if you can dig into it and read it um if you are seeing it for the first time take a look at these recommendations the us department of that is suggesting um each one of these are at the foundation of the work that we've done at the center for the past 15 to 20 years and i want to highlight especially that in order to help students socially emotionally behaviorally you should establish an integrated framework to organize your school and that's what the work is today when we're talking about getting started with pbs it's the framework it's how are you going to put what's the glue that's going to hold all of this together um i can't say enough how exciting this document is um there's so much in there that that you should break down um and whether you're new or experienced it's definitely worth the time also talk about kind of where you are um in terms of this work are you the start are you in the middle is this a re reboot opportunity do you know pbis or you knew it 20 years ago and now you need to dig in again and look at the fidelity of the work something to know about our experience in education is this idea of about a four-year cycle for um innovations in education the enthusiasm the implementation and then the drop off until something new and we see that time and time again whatever your priorities are in the school give it a four year timeline and see where they'll be four years from now and we've said for a long time that in pbis it takes three to five years to implement this with fidelity well that's right in that sweet spot rob horner um started shifting some of that language many years ago and said don't do anything in your schools unless it's going to be important to you for the next 10 years i like that framing as well and what we've known even though there is that for your life cycle throughout the past 20 plus years pbis has been continually relevant and the reason is because the framework at its core is simple and straightforward this is the most concise definition i can write about what is pbis so it organizes your school to achieve outcomes through understanding your data providing continuum of practices supporting staff through systems and prioritizing equity throughout you can't you can't get fewer words together and while it might be easy to define in this case it doesn't mean that school change is easy so that's what we're gonna again dig into today so what do those components mean that i just kind of quickly rattled off when we're talking about outcomes it's your outcomes what's important to you and your school community social competence academic and academic achievement is sort of the broad category of what we would start with and you're going to define that specifically what are the outcomes that you are currently getting and or that you want to get when we talk about data we're talking about supporting decision making across all of these components using data to see where you're at using data to progress monitor using data to see what you've accomplished practices are how adults interact with students and that's where i kind of hit that little triangle there that idea of a of multiple tiers what are we doing for all students what are the practices that we layer on top for students who need additional doses of support what are the practices then that we further individualize for students who need intense support for any specific social emotional behavioral skills and finally systems those are the components of our school structure that best support adults other language that's been that's become very um uh commonly heard across the country is the idea of a multi-tiered system of supports or mtss and if we're in the u.s and you look at 50 states there's about 50 different ways that this is being tweaked and identified in each state um i like these six components taken from the macintosh and goodman publication in 2016. yours might be a little bit different but they're probably similar enough so when we talk about mtss no 80 of the research that went into mts has come out of this foundational work in pbis and pbs then becomes the social emotional behavioral mtss framework so you take these six components and this is how they would layer on under systems how we support our adults what we're talking about is leadership and coordination and then we're talking about the process of providing professional development coaching and content expertise two of those components of mtss when we talk about data the mtss components are evaluating the fidelity of your system and your practices progress monitoring and then universal screening what do we need to focus on for our whole community which students need additional doses or more intense supports and then under practices that is this three-tiered continuum of culturally relevant evidence-based interventions what are we doing for all what do we layer for some what do we individualize for a few so some big messages big principles when we're talking about these components of pbis equity you can see equities in the middle it crosses everything equity is not just the outcomes that you get and are they equitable it's the processes and inputs we put in it's the representation of our decision making groups so equity should be evident across all of this work this is for all students all families all educators it's always important to include your staff when you're talking about these outcomes the framework helps you differentiate support that's this idea of tiered interventions we want to make sure that the students are central to all decision making so this is for all and how is it impacting the students student outcomes should reflect equitable learning opportunities and equity should be part of the ultimate criteria for the decisions that you're making when it comes to those practices you're better off picking a few and implementing them well than having a smorgasbord of practices integrate and align your practices how many of you are working on pbs but you also have restorative practices and second step curriculum taking place in a bullying prevention initiative and oh we're going to have a book study on on racism in america you're doing all of these things align and integrate one framework pick the framework and use that and bring those important ideas into that framework data should inform decision making for all and invest in those systems to support the high fidelity of implementation across time let's don't make this a four year project make sure this is existing for at least 10. if you do this well we have evidence clear evidence through the last 20 plus years in randomized control trials showing outcomes for schools that implement fidelity have increased academic performance increased social emotional behavioral competence increased social and academic outcomes for students with disabilities reduction in bullying behaviors decreased rates of student reported substance abuse we know schools that involve fidelity have reduced exclusionary discipline practices we know that schools implement with fidelity have improved teacher outcomes perception of teacher advocacy i love that one the parents will feel and the community will feel that your teachers are more effective because of how you've organized your school so this is the threshold it's not just hey pbs is on our to-do list but we're doing it to fidelity and if we can get there when you compare yourself to other schools that are similar these should be the outcomes that you see okay so let's help organize this in sort of a logical rollout and show you where where data fits in all of this this is one way that we like to organize it so start with teaming who's on the team who's making decisions is it shared leadership [Music] have that team that's representative of your community decide what is the vision for our school and how are we going to define that through three to five expectations once you have the vision now let's task analyze it what does that vision look like in my in the classrooms what does that look like on the way to school before the bell rings what does that look like in our after school programs if you've defined your vision and you've task analyzed it how students can show it through their social emotional behavioral skills now we need to teach so what's the lesson plans and the curriculum you're going to use to teach the skills to achieve the vision it's once we've achieved those components that those are in place now we can talk about that system for feedback how are you going to give students specific praise for feedback about what you've taught about what's taught and connected back to the school-wide division [Music] how about when students have behavioral errors or need more support or intentionally say not interested in following the rules today we need a process then by which adults define that behavior the same way and have a consistent way of responding to the behavior [Music] and then finally what's the data that informs all of these processes so that's the process if you're familiar with how we measure fidelity in pbis at tier one we use something called the tiered fidelity inventory and you can see how all of those components those 15 components of tier one map onto that process today we're going to start with database decision making and that's what we're going to dig into so you can see i'll move all those components to the side we'll sort of leave them up as we present throughout the day we're going to be prioritizing database decision making and if you're familiar with the tiered fidelity inventory it's items 1.12 13 14 and 15 especially okay if you haven't done so already now is a great time for you to go over to the polls tab and while we do this i'm going to stop sharing and heather will pull up her screen go over to the polls tab and answer these four questions about where your school or if you're a district administrator in general your district where are they at with these ideas so we'll give you a minute to do that heather let me go ahead and stop sharing all right good we can see your screen heather okay and while you guys are completing the polls we're going to be looking at the results and just know that we'll be reporting back on those results um towards the end of this session too so and i know it might take a little bit um to answer them we're going to keep talking because we have a lot of information to cover um to give you guys a heads up this is something that we have typically done in two full sessions um we've realized you know what getting started has so much information and it's so important to hear directly from the voices of folks that we have invited exemplars across the three sessions within this getting started strand to be able to answer questions directly to you in the chat box while we're presenting um they may even have opportunity depending on time because we've got a lot of people that are in this session to um to turn their microphones on to answer answer direct questions but rather than taking time like for one person um providing the exemplar um situation you're going to have the opportunity to pick their brain while we're going and talking specifically about data today because the exemplars are going to change across each of the three sessions that we have in this strand so you'll be able to pick their brain ask specific questions and if you're not putting questions in trust me they're going to be start to uh possibly ask you guys questions in the chat box so um we hope that you guys will really take advantage of this great opportunity so let me talk to you um about our first presenter and therese um would you like to hop on so they can see you but here's some background information about therese and she's going to be one of the people behind the scenes answering your questions hey everybody my name is therese sandomierski i actually work with heather i'm a technical assistant specialist with florida's pbis project i started with the project back in 2002 as a graduate assistant and came on full-time as a technical assistant specialist in 2006. since then i've provided training and technical assistance to districts in almost every corner of our state depending how you think about the different corners so i've been able to see a whole variety of schools with a whole variety of data systems at all different starting points in their journey towards implementation so thank you all for having me and i hope you enjoy this session thanks terese and our next exemplar brian hi good morning everyone and i'm also with the florida pbis project as a technical assistant specialist working mostly with districts in the southern florida tip of the state and i've been with the project since 2010 prior to that with the florida rti project where i served as a regional coordinator and helped with statewide implementation of our florida problem solving process across our school districts um and so i'm very happy to be here and welcome and i hope you enjoy the presentation sorry thanks brian and diane didn't realize i was muted no worries good morning um it's good to have everybody here with us in the session i am diane lemaster with the midwest pbis network um as a technical assistance coordinator and trainer i have been involved with pbis for 20 plus years implementing at a building level at a district level three tiers of supports for students so lots of lots of years of experience my specialty within the network is all things swiss and tips so i am a national um trainer and facilitator of swiss and schoolwide information system as well as tips the problem solving process and model so that it's uh kind of i'm that go-to person within our network for all things data and so it's this is a perfect place for me to be and i'm thrilled to be here and hope you enjoy the session thank you awesome thank you and welcome diane so and welcome exemplars and so again everybody um if anything comes up any question whatsoever um during this session please make sure to put that in the chat box um because we really want to be able to use this time to to pick their brain to hear firsthand directly from them some of the issues barriers they have come up with some of the enablers they've been able to work through and i guarantee if you've had that question somebody else has had that question too um so i think this is a great opportunity just as a reminder if you do put something and you have a specific question you put that in the chat box please make sure if you're coming from a district to identify your district name and add the state it's from because we all don't know which say independent school district is coming from what state and then maybe your position and that will help them determine which of the three will be answering your questions as we're going i'm also going to try to monitor the chat box while we're talking about using data for decision making um so that and they may interrupt me occasionally too if i miss something but essentially sometimes folks wonder when they're talking about those three circles that brian covered with you they wonder you know which one do i start with do i start with the practices do i start with the data do i start with the systems piece and i think for folks that have gotten fluent in the pbis framework and the implementation particularly at tier one they there's all different ways that you can start and there's no one right way but one of the things we do always like to say is well what do your data say so before you go and invest in establishing the the pbis framework and looking at establishing a core um across all of for your school campus you want to say like what do our data tell us and that starts to help you determine and generate an action plan so one of the things i'm curious for you whether you're coming from a state level you're coming from a district and you're thinking of many schools potentially implementing or that are starting to implement within your district or you're at that school level and you're thinking how do i get this going right now within my school so one of the things i'd like to hear from you guys if you can put it quickly in the chat box is right now does your team review or analyze data and i'm curious um again this should probably be a poll but we didn't think of it that way but to think about are we analyzing looking at our data at every single team meeting um are we doing it at least two times a year are we doing it at the beginning so i'm just curious um what you guys have to say about that and i did see that somebody said they love data so that's great i think you're in the right session so looking at the chat all right so there's at least three times a year there's some folks that are saying not regularly and you know what it's important to have an honest assessment of that because we can't ever figure out how do we improve um if we don't know exactly where we are so talking about understanding data you know there's there's various purposes for it so there's evaluation purposes for that and that there's the summative it helps us with our overall decisions you know should we um you know do this is it working do we need to abandon it we need to modify it then there's times we go in and provide there's those formative types of data which can help us determine um is it implemented as intended is it improving the outcomes as intended um and then accountability you know determining are people actually adhering to procedures policies practices from there's there's all different ways that we can use data for our evaluation purposes i think overall it's important for us to know when we're looking at tier one we're thinking of that overall kind of screening process and it doesn't necessarily mean instantly we're going to be doing a social emotional screener however if you did which is um a great thing that is something that is a tier one data process because you are starting to gather data across all students across your entire campus um but the data can also help us determine for those students which ones are not responding well and help us in that diagnosis to determine who needs more intensive supports data also can help us in that progress monitoring piece to say hey based on what we've come up with is it working is it making a difference rather than waiting towards the end of the year to say did we do better than last year or not so good at that point it's too hard to make changes live in that process then you got to just gear up and hopefully prevent for the next school year but it's so important that we understand why and what purpose the data are being collected and it needs to be meaningful to you it needs to be meaningful for the folks that are helping to contribute in the data collection and overall who are completing the data so if you did have screeners it's important for the kids to know why is this information being collected what's going to come out of this again having that meaning that purpose behind it can help it make it less of a paperwork compliance task so looking at that overall continuum of assessment you know we're looking at school-wide screening and so like i said i might may not be talking a formal screener at this point yet screeners do fall into this category but i think another way to also look at is who is responding to what we have set up on campus and who's not responding and sometimes our basic just office discipline referral data can give us some indications with that but also we have to know not all of our kids um generate office discipline referrals not all of our kids have externalizing behavior issues not all teachers send kids to the office the takeaway is that not to say we need to send more kids to the office we get accurate data the take away is how are we um gathering and figuring out how are kids doing out there across the board based upon our expectations that we've got set up how effective are we in providing our supports across campus what supports um are indicated and what can students um do why can they do that why can't some students do that and it starts to help you in that problem solving process that other continuum starts to move into that progress monitoring piece of saying how effective if there's an intervention that we're doing say in the cafeteria how effective are we in that process rather than again waiting towards the end of the year and then evaluations that final how did we do overall so some of the examples of data collected you know is the fidelity of systems so overall when we're talking about um our tier one across the board that data when we're looking at our system um you could use the benchmarks of quality which is sus is just tier one there's the tfi which is the tiered fidelity inventory which barney talked to you um a little bit already there but that tiered fidelity inventory can assess all of your tiers from not only tier one but tier two and tier three if you chose and there's other tools as well that can help you determine have we established a system and do we continue to support that system with fidelity as it was designed but then there's also the piece of practices so just because we set up a system we actually have to have some evidence-based practices that we're using on our school campus so you'd want to know like are we actually implementing these um interventions that we've committed to or that we've you know paid a lot of money for on our school campus are we implementing them as they were designed and as intended and so there's different types of ways to assess that as well and then another way of looking at our data is to look at our overall outcome data it could be your office discipline referrals that are happening per week that are happening per month at the end of the school year but you also want to look at your attendance data you want to look at the number of students that are actually being acknowledged that are doing the correct thing on campus um do you have data around that look at grades for students you know if many students are failing their classes that's a classic sign that it's a systems issue sometimes as hard as we try for our lesson plans and we think it's a fantastic lesson plan plan the kids don't get it or the way we tested did not match in how we had taught them um also are students achieving their goals are we seeing that we're regaining our instructional time there's also parent surveys um i know a lot of our schools and districts are good at maybe um completing like climate surveys they've got other um types of surveys satisfaction are people happy with the supports they're getting are they happy with their their children going to your schools but then you could also look at have focus groups direct observation that you're seeing so looking at this list of the different types of data to be collected don't worry if you're not collecting all of them or don't worry or panic thinking oh my gosh we do collect all of this we don't look at all of it and don't worry most people have not always done that and so the key is identifying what is it that we do have data on what questions do we have that we need to be um asking ourselves and determining where are we doing well what are we not doing so well and then that helps us on our next step some examples of that discipline referral data again these look pretty fancy but if you were interested in having these fancy graphs and entering some of the discipline referral data that do occur on your school campus the swiss system if you go to that link that is something that is available for for your school to check out um but you know is it the average referrals per day per month also looking at you know what types of problems are occurring so if you are using some of your office discipline referral data what are the problems that are being sent to the office what kinds of problem behavior are students exhibiting because that information can help us determine where we need to go um in determining what practices we need to focus on um in what location are most of our problem behavior areas or problems coming from you know you think about my gosh there's so many kids there's a lot of minutes throughout the school day where do we even focus our time but if you're able to determine and identify where most of the problems are occurring in what location you can um you know focus in on that and provide that intervention in that particular area or that particular time of day in that area or it might be that you're realizing it feels like there's an awful lot of problem behavior but maybe it's just a small group of kids that are actually generating most of those referrals well rather than providing massive intervention across all maybe we need to provide more intensive intervention for these groups of kids we also could look at referrals by day per week um i think sometimes referrals per day per month like right before holidays sometimes things can get a little um a little wild or even right after the holidays for things and looking at referrals by grade again you're trying to use your data to determine where are we having issues and where do we need to focus more of our supports and then disproportionate outcomes um typically are not going to be addressed unless equity is documented and sup and a supported priority so one of the documents that i wanted to show on the right here about identifying disproportionate discipline that is hyperlinked so if you are looking at either the the handouts the pdf handouts that have been uploaded you can click on that and that will hyperlink you directly to that resource that's in the um live binder on the florida project so talking about disproportionate outcomes um this is theresa's alley so if you guys have specific questions she is the person to be putting in the chat box and asking your questions right now on this one i'm sure diana brian would be happy to jump into um but i therese i think uh might be salivating over this one maybe it's a good thing maybe her her camera's off right now but she is somebody to definitely um pick her pick the brain over but it's so important that we're looking because we can see are we um are we being equitable in the amount of in the discipline that we are providing across our campuses um for students and we need to sometimes look and take this hard look and say okay if we are not let's what do we need to do differently what's going on here and again we're using data to help us determine what's working and then what's not working so well so that we have a better idea and what to do when we start pbis and start to identify our practices and how and why we're going to focus in those areas talking about data sharing with staff students and community um this one i it's so important that we don't overlook this because again the last thing we want to do and we all know there's been lots of things in our work environments um regardless of where you're working school or district that you've had to complete something you've had to do something and there's a deadline and there's people sending you reminders to get it done but you're not really sure why you're doing it but you know your boss is telling you we have to get this done and then sometimes you wonder a couple weeks couple months go by you're like what was that for did they do anything with that what are they going to be doing with that it's so important that our students know as well what you're doing on campus it's so important that your staff your faculty if you're asking for information and you're asking them to collect data and you're asking them to let you know what's happening out there and you're observing things across campus and you're proposing to make changes on campus it's important that they know why it's important that they get a read and get some feedback of this is what we found overall this is why we're going to be making these changes that's an example of database decision making but it's also an example of how you keep your folks at the table because they're going to be invested in this process in helping you implement so i think it's important for folks to know how are we progressing towards our goal it might be that you guys are simply in that data gathering piece right now to figure out what areas of your school climate that you're wanting to improve and where what's really needed i think it's important that you share the fidelity checks with them if there's interventions that and trainings that you've had them go through and you're going through and providing observations it's important that they get feedback on how things are going you know adults like knowing too if they're doing something correct um as much as we say like i don't need to be rewarded you gotta admit at times adults do like to be validated if even if they know they were doing something correct they do like to be validated kids like to be acknowledged too if they've done something correct again if we want the likelihood of this um behavior to occur again and if it's a positive behavior we need to acknowledge folks about that when we're sharing data with um folks we also need to get a read from folks out there does this look accurate so i remember sitting in a high school in the high school was trying to get their faculty on board to get excited about tier one pbis and they hadn't even gone through training yet but they wanted to open up the conversations to get a gauge of where their whole faculty and staff were and the faculty were saying we don't need to do anything else um the administration needs to do their job and they need to discipline these kids and the administration was saying teachers just need to teach and they need to stop sending the kids to the office he's like i'm backlogged so he decided at a faculty meeting i'm going to have an open conversation um and before the new school year starts out i want to get an idea from the teachers where they think the number of tardies that we had across campus because their current system actually had when you're tardy to class in high school this high school said you had to be sent to the office and you had to go get a tardy slip so think about it you're late to class already so you've missed some instruction and now you're going to be even more late by being sent to the office and getting a tardy slip and then you enter late again and the administration was saying this is not working however how do they get their folks their teachers on board to help them on trying something new and so from this information they said how many parties do you think we had last year and this was a school of a thousand kids and you know they had people saying like 2 000 tardies you know 10 000 tardies and then one person said okay i'm throwing it out there 11 000 parties and people were like oh my gosh that's crazy uh the administrator said nope i actually have over 16 000 documented tardies and we know not every single tardy was accurately documented last year he says i've equated this to what this means at the lost instructional time if it's just five minutes per tardy that gets missed of instructional time and there were years and there were lost of instructional time across this entire student body that was enough to stop the teachers and they said well this is really we need to change this isn't working so think about of the data um that are being sent it's important to have conversations with folks over that but then it also helps them determine and realize we're going to do something with these data um these data actually can be helpful for us it might not look good right away but the goal is okay we're looking at what's not looking good and what we need to do um for improvement and that'll help us track to determine again progress monitor are we making a difference as we're going so provide feedback to folks let them know where you are in the process i think another piece of data i'm sharing is what you heard today this morning and even yesterday of the voice the voice of the students the voice of other district folks the voice of your teachers the voice of your parents are they giving you feedback on what's working what's not working again just because it might not be a data point that's still important data all right and then fidelity of implementation so um you know do we have um schools implementing with fidelity is your school implementing with fidelity and what it means is that it's implemented as it's intended so that tfi that we've got um established that tfi can help us determine and build that system on our school campus um for tier one and we're not just guessing at what we want to do so this was a graph for example that in florida i had administrators that would say you know what i'm implementing okay we know we're just okay and we're having some positive impact and that's all right and and i started to hear that a lot from principles over the years and i thought you know what i would like to know at least in my state if you're implementing with fidelity do you actually have better outcomes and this was from long ago you can see we started collecting these data back in 2009 and sure enough you had 34 fewer office discipline referrals across years if you implemented with fidelity and i'm not even saying high fidelity at this point you know where you're having to get like 100 on the benchmarks equality or the tfi i'm talking about you're getting at least a 70 on either one of those so another thing that we had found is that um the average was 41 fewer days of out-of-school suspension if you implemented with fidelity and you think about that it's like great data that started to get a lot of administrators in florida on board so this is one example of our annual report you can go there you can click on that and see all kinds of information but again sometimes to help us get started is to hear data from elsewhere of what what worked and the data that they used and sometimes that will get things going um in your area so the dsfi we've talked briefly about the tfi which i'll be talking or might be mentioning that again um in this session but the dsfi is specific for district systems fidelity inventory and you can click directly on this icon and it will take you straight to um this this tool um and it is essentially to help you set up your system at a district level um for supporting pbis and so if you're at a school level that'd be something to think okay how can i help get my district on board what are the some of the things that they might need to consider it helps also to organize what we're doing around training what we're doing around providing coaching supports for folks the last thing we want to be doing is accidentally competing and we want to make sure that we're providing resources to folks overall so that when they do go through professional development they are getting the supports done in the classroom and know what to do with it so the tfi that we talked about again you can click right on that it'll take you right there but brian had given you um earlier today that overall snapshot of where we are are we implementing correctly um but essentially with the fidelity it means like there is a purpose in why we're setting this up and so i think when you're looking at this graphic and you're seeing teaming and leadership involvement at the very top typically when we're getting going on a school campus you would go in a a uh clockwise kind of direction so the next thing would be okay we've got some folks together as a team we're going to develop our vision and expectations we're going to define rules we're going to set up our system for teaching we're going to set up our system and how we're acknowledging students how we're acknowledging staff who are implementing um correctly and or even providing some error correction for folks but then we're also going to come up with some different strategies of how we're responding to problem behavior differently and we're going to use our data as we're going to progress monitor and for that continual improvement process but what you're going to see is that we're going to mix this up a little bit today because as you can see database decision making i just mentioned that as the very end however you're going to see data-based decision making is part of this process all the time so we need data to determine what are we doing at tier one why do we want to implement tier one what is it that um we need to select as part of our tier one practices but then we need data to determine are we implementing that correctly um how is how is our system going to change to support these practices um that we've set up and then you're using that data in that continual improvement process along the way so and it doesn't hurt to always revisit your vision and expectations throughout the process so even when you do get um this up and established i always say to folks go back and double check because even though you came up with the system of how you were acknowledging kids or you came up with your schoolwide expectations it doesn't mean it's time to edit it we evolve over times our needs of our students sometimes change over time um our needs of our teachers and based upon the infrastructure and the system we've set up on our school campuses change over times and we need to adapt with that um and respond accordingly and that's that piece where the data are always helping us along the way if you have not had a chance to check out the pbis apps um website so um the pbs apps this is something you can go there if you click on the the pbis assessment you can enter either the benchmarks equality or the tfi um directly on there that piece is free um and it would give you instant graphs but there's also a lot of other information on the pbs apps website um to help with the teaching aspect to give you different on things to think about and using data for decision making as well so using that data to guide intervention and instruction so this problem-solving process so when we talk about tier one we know we're talking about typically what's in the green category um on the triangle but what's deceiving about that is that makes it think like oh it's only kids in the green that get tier 1 supports and therefore others get other things if you're in the yellow or the red you get other things and what's important to know is that tier 1 means all so when we're talking about using data to guide intervention and instruction we need to know how things are working across all at tier one which means all students it means all adults which could be the teachers the staff um how are all of our families responding you know are they coming to um our school events are they you know following the pickup line procedures um at you know dismissal time um but you're talking about all but then there's times we know that some folks need a little bit more support for one reason or another so that's where that problem solving process comes in again you're defining what's the problem looking at that analyzing it coming up with the hypothesis and basically saying what are we going to do try it out and then you're going to use your data to evaluate it and kind of say to this work then that next piece is let's take it out and let folks know but from this process of kids getting say needing tier two supports or needing tier three supports they're still part of the tier one process because they are part of the all the difference is the data we're showing that they need more intensive supports that's all so that's the piece that i think is is hugely important um classrooms fall into the same category so if you're thinking about how do i implement tier 1 get started across my entire school you got to think about all of your classrooms what are all classrooms going to be implementing and supporting across the board yes some classrooms based on um who might be in the classroom or the levels of um years of instruction for the teacher or their training um might vary and which might mean there's different supports that you have to provide for some of your different classrooms but all classrooms there may be an expectation for how you want teachers to be supporting students and you're using these data to help you determine how are we going to um action plan what is it that we need to do so if you've got few students that say they're having more most difficulty they're part of your tier one they continue to have access to tier one they continue um but they need more supports looking at that data to determine what else do they need you know is it more social skills is it um problem solving like conflict resolution stuff is it um friendship making you know those types of things that's using your action plan for that and using your data to determine who you're identifying in that process how you're going to provide them support and then how is that support going how are the students responding to that intervention same is true is if you have um a couple of teachers you've decided to send them to additional training um you're going to be looking with that action plan and just say how is it working are they um being able to take the new information and apply it within their classroom and are you starting to see some positive outcomes what other supports might be needed so that's that piece where you're looking at your data and you're going through that four-step problem-solving process saying how can identify the needs you know what is it that we're working towards how can i understand the barriers let's analyze what's going on why aren't we having um stronger responses to this providing sufficient information we've got to implement and then we've got to evaluate to determine how effective we are in anything that we do this is an example of some of the key questions and infrastructure across that continuum of support where we're looking at decision rules of moving in and out of tiers because again tier one means all people okay nobody is exempt in this process so if you're looking and starting at the bottom you're saying like is it working for all so whatever it is that you're doing on your campus is it working for everybody is everybody having positive outcomes if they're not then you start to think about okay what services are being delivered are they evidence-based are they with fidelity who's receiving those services um but it's so important that we have an idea of using our data to determine is our system working overall you know what are the decision rules that we have to determine that it's not working what are the expectations that we have overall for our students for our teachers that starts getting into the practices piece that also starts getting into how you're setting up your system to best support um pbis on your school campus or across your district and some of this stuff may if this your brand new to pbis might start to make more sense as we get into the practices piece and get into on the systems piece in the next two sessions one of the things you can also i think it's always important for data regardless of what data that you're looking at you essentially want to know our behavior is improving our behavior is holding steady our behavior is getting worse okay so that can start to help you determine right there what would our decision rules be so if behaviors are getting worse well what is it that we expect overall that could be some of your basic decision rules right there um we expect you know that uh all students would not be sent to the office or maybe they're only sent to the office one time throughout the entire year let's just say well that's for a basic baseline right there that you can look at so then you can say okay then how many of our students are actually um following this or are we having bigger issues so it's that's going to help you start to mine down even further you know why is this happening before you can start to even come up with and say what is it that we need to do differently so questions for consideration is to what data are needed to assist us in making meaningful and efficient decisions are there additional data that can help us and if yes what type and what will we need in order to begin using data regularly for decision making during our team meetings because when you're using the problem solving process across the continuum of support i want you to think about tier one is about what does everybody need and that's gonna be the big takeaway and like i said i started to dabble a little bit and talking about tier two and tier three and you'll start to hear that come up a little bit throughout this entire strand but really the focus of this strand is about getting started at tier one and so i want you guys to think about what do you all need across your school or across all of your schools if you're district level or across all of your districts if you're state level and then go through and see these questions what data are needed to assist us in making these decisions are there additional data that can help us why would they help us and what do we need to begin using data regularly so consider the database decision making capacity of your system that you have right now a bunch of you guys did say that you are engaging in some type of data decision making throughout the school year um but do your data analysts have full access to the data or do they have to request and do they have to know what they need to be requesting to make decisions within five to ten minutes a start of the meeting are your teams reviewing up-to-date precision problem statements developed by your data analysts are they going to help us identify that if then um statement so you know we haven't taught students uh what the procedures are in the cafeteria chances are that's probably why kids are bumping into each other and then why we have fights again are they starting to look through and start to come up with some ideas of why there's problems occurring now your team is able to develop a complete action plan to help address a specific problem that is coming up in 10 minutes or less so just think about that if you're not there's other kinds of resources we can help connect you to but you've got to start first with some type of data um and to be gathering people around the table um talking about that data and saying what are we going to do next in order to start developing an action plan so brian i'm going to turn it over to you here um how things were on the poll i'm gonna take a look at the chat um i've seen the exemplars have been typing away like crazy so um i'm gonna mute um i'm going to share i was able to pull up the results um and while i do that i just want to highlight and that last slide that heather was sharing she's you know the prompt is what are your teams doing if they don't have those systems for analyzing data instead and i always like asking that question when we're working with schools and districts a lot of time teams get together and they enjoy each other's company which is great they persevered on what's not going well in the school maybe they're planning parties and celebrations um and those those have components of importance but teacher time is so precious so the efficiency and the effectiveness of our team meetings really needs to be attended to and those prompts on that previous slide i think help with that they speak to this idea of when we sit down we already have our data analyzing can we spend our team time talking about what to do about the data for example um and is it accurate and are we able to make decisions i would say you know when a problem is presented can we get to a solution in 10 minutes um before my work with pbs i would have thought that that was not realistic and now we actually have some pretty good research models to show how to do that which is pretty awesome so you were asked to look at these um polls let me make sure i can pull it up set to share one more time give me one second here let me share [Music] see if that comes through all right a little hard to see let me try to make it bigger all right so it's hard to see on your screens i think so i'm looking at a secondary view myself so let me tell you what i'm seeing what we're looking for in these questions these four questions about data and decision making is we're trying to grow the blue essentially so do you have a leadership team that reviews accurate and up-to-date data behavioral data monthly and right now about a quarter of you um the um uh the thing the thinking here is do we have a process where we're organized and each month at our at least at least one of our monthly meetings we're able to break down the data and make quick decisions about it when i first started doing this work my very first school team had we had 20 participants we were very small residential school we had 20 team members which is almost undependable and we would just bring the data we would sort of swim in it in our meetings and we'd spend a lot of time kind of looking at data it wasn't the best use of our time we didn't know it then um but analyzing in advance actually made made a big difference so for that first question 24 the next question decision making our leadership team develops database action plans to address behavioral needs and monitors each plan monthly so that's at about 17 of you said that you have a decision-making process in place the next question which is the fidelity our leadership team regularly assesses fidelity of teachers discipline practices and of our pvas implementation like the system itself and we're about five percent so i find this to be very common i'm sure of the hundreds of folks who are listening in today many of you probably have implemented pbs in some level of fidelity in the past or your school has been doing it for a long time for example and then if you had called heather or myself and said we're having trouble the first question we probably would ask you is what's your current fidelity implementation and the reality is most folks don't know when they're in your situation because they're not actually measuring it we have a way to answer the question are we implementing pdis the way we're supposed to and that's this tiered fidelity inventory so uh this question sort of answers itself in a way it's item 1.14 on the tier fidelity inventory and if you're doing the inventory you would at least be able to answer this one positively so that's a great uh a great way to start um maybe before you even get into the where do we go next let's take a baseline and figure out where we're currently at with what we're doing for our tiered pbs supports and then finally an evaluation plan and i thought this was again about a quarter about a quarter of you said yep we have this in place and um for that group um i would want to dig a little bit deeper and to say you you have a plan a plan that's been informed for at least once a year that speaks to your system for supporting the social emotional behavioral wellness of your students and it's in your school's climate and culture and that plan includes action steps what are our goals for next year where do we want to get to um so the quality of the evaluation plan may be something to consider and even in this case even if you said it's fully in place for any of these you can always look at that quality of that item and consider but what else can we do to make this more effective and efficient for us so that's where we're currently at with the fidelity the uh polls i'm gonna share my other screen and let's now transition maybe to sort of action planning and question and answer and while i share that slide it's probably just going to come up as it'll be easier for me if i just show it as a the whole powerpoint deck um this might be a time for therese or diane or brian to maybe if you want if you come off mute and maybe you could talk a little bit about what are some common questions that you heard in chat or what questions might might we we want to speak to as a group we have about 10 minutes left here let me switch over my screen hey brian this is terese hi therese i thought that we had some really really good conversations going on in the chat area um i think maybe one of the biggest conversations at least from my perspective was a conversation about data consistency and do we have consistency do we use data if it's inconsistent and that sort of a thing do you or other have any recommendations or thoughts about that issue um yeah so that that's that that is very common among schools um when we talk about discipline data for example you might find that when you're getting your staff organized how do we see behavior similarly how do we document the behavior we see you might be a school that has an increase in documented discipline problems for example um so being consistent with staff might mean oh actually we're going to see the numbers kick up because now we're documenting what we see a little bit more um and i think my i'm here to hear what heather says but my concept my regular message around this is data integrity and data consistently should be a continuous focus of your pbis teams for as long as they exist so you might put a substantial work into that this year getting your teachers to say what do we consider disrespectful behavior and what's not disrespectful and let's be consistent with how we document it for example but you can't just do that one time and expect five years from now your teachers are still documenting it consistently you would need to come back to that on a regular basis and show teachers of their own data and ask those questions heather how how would you respond to that question about the data consistency and suggestions for uh focusing on that yeah i mean i think the big thing is um it's a great goal to have and it's something we always want to strive and just know that we're also human and there's always going to be that human error that occurs um and so the folk the key that's going to be is are you looking at that data and when there seems to be things that don't quite match whether if it's an office discipline referral say that's not filled out completely rather than i think us going oh you need to fill it out again we actually need to follow up um with that person who filled it out and just say hey you know we need to talk about how it needs to be filled out and hopefully then they're able to see oh this is why it's filled that needs to be filled out this way because you guys are using it for certain things and so hopefully that that connection is is made but just know that it is going to be a continual improvement process um because there are so many people involved with that data piece and um and i know that there's a ton of kids and there's a ton going on um throughout the day so you know they're and we are human so keep keep in mind that as long as you continue to look at it and there's a way that people don't start getting scared of data and thinking like um you know data is that that bad four-letter word um type of thing but that data actually um we need feedback too and it's just it's that performance feedback and we need to know when we're doing it correctly but we also need to know if we filled something out incorrectly um too so i again i mean i'd like you to say like yes work to make sure that it's consistent um and i think that's always a goal to have but just know at times that there's still going to be some errors hopefully it's a small standard deviation in that because because we are human as long as we're continuing to keep up with it and heather that consistency is key um we saw early on in the chat some questions and comments around major and minor problem behaviors and you know that a few comments came up about some people are just tracking minors and some people are just tracking majors and some are doing both and so it's that consistency um getting everyone on that same page you know that might go back to the development of that t-chart that really spells out very clearly office managed or major behaviors versus minor classroom teacher managed behaviors and getting everybody on that same page and having that consistency and that common understanding makes such a huge difference in the the you know the data that you're collecting if you want to really great clean data and we just have to have everybody on that same page so it really kind of goes back to one of those components of the development of a t-chart another thing that i'm seeing in chat was this kind of conversation we're all engaged in and you know is does pbs use a root cause analysis uh for for uh database decision making i think i would take a step back from all of that and ask the question of your schools do we have a database decision making process and what is our process and are we doing that process with fidelity so in the slide i'm going to pull it up here the side that heather shared um define analyze and evaluate force that process and if a school is doing this they're relying on their data and they're coming up with a plan and implementing it at their team meetings or planning to a minute and then implementing it then evaluating it at work that school is in great shape most schools without a formal process that they are focusing on and providing some pd for fidelity they have problems and they talk about solutions and they just talk the problems aren't defined with precision there's no data necessarily there's a feeling of boy it feels like the parents are pretty unhappy with our recent decision about our bus schedule after schools okay that's a primary problem statement but people actually encourage you look at the precision of those uh statements turn that into data that you can specifically tie and you can connect it to and what the process you're gonna use so this one on the screen i think is a great example you might have heard of team initiated problem solving that's another database decision making process and they have something very similar to this as well again whatever you pick i would say are you providing training to your teams on using data for decision making and then are you staying with them and helping them do it in a way that that is a fidelity we have a couple more minutes here anybody else brian therese diane any other questions that have come up i'm looking through some of the the chat here as well to see if anything else pops out i just see brian a recent um a comment about um clip charts our school is still a clip chart school so i don't know brian you're pretty good at addressing that you want to say something about that we talk about that a lot during team training uh yeah how about how about we save that for practice okay yeah which is coming up next because i know i i trust in teachers i know why they they decide to use clip charts they probably have the best of intentions they're trying to do what they think is right for students and let's honor teachers and say okay if we can do better than that let's teach them what that might be first so we can give them some good replacement strategies but i think that that's part of what we'll get into next with practices is how now should adults take advantage of this data that we're organizing and use that data to inform what practices will help students achieve the social emotional behavioral success we want so uh i like that setup diane i think you're trying to get on with just three minutes left here is what i really what i really think um so anyway um you know what though i we didn't mention i don't think we mentioned uh like a sort of discipline or shaming strategies or things like that in our in our section that's coming up next um but feel free to bring that out and chat for those of you who are going to be here after the break and uh we'll jump into that toward the end so i'm looking at the time here it's uh almost a quarter to the hour this would be a great time if you haven't already if you could look on the pathable page you'll see the survey link oops i'm scrolling way too far let me find it i'm going to put up a slide as well and we would greatly appreciate it how's that going to work i'm just making a little too big i'm trying to look at my screen down here hopefully you can see the scan me i'm not sure i don't know how well that'll work um i'll zoom in on that how about that but it's also on impactable so in the files tab you can click on the evaluation link um on the card itself under our names under c1 you can see the survey your feedback is really important we'd like to do that for each of the sections today so how did you feel about what you learned today regarding data this big overview idea just getting you started and uh let us know and then what we're going to do is we're all going to take a half hour break i think it's a half hour i have to double check um it is okay i think i should have looked at that i have looked at that many times a half hour break we're going to come back at a quarter past the next hour for wherever you are in the world and um we're going to talk practices next okay heather anything else before we wrap up yeah i just want to say the very last slide um on the handout is there's actually three different links there so we've got the national center obviously but then there's the florida project as well as midwest pbis so different things that we had talked about um from from both brian um diane and the rest of uh brian and terese um you can find it at one of these three spots and um our contact information again if you have a specific follow-up you want to talk to any of the exemplars their emails were located on the slide that introduced them and then brian's and and my contact information is on that very first uh intro slide as well but we're looking forward to seeing you guys in c2 in a half hour and um we'll have even additional exemplars so a huge thank you to therese brian and diane for your expertise um and answering questions along the way and sharing your trade secrets with folks and we have it all recorded so which is good enjoy your break everyone thank you everybody the next one thanks thanks everybody i'm gonna go ahead and close this all out thank you all right thank you guys thank you thanks everyone thanks lauren