hi there i'm matthew lavry director of research for the south carolina education oversight committee or eoc over the next couple of years south carolina's educational accountability system will transition to a new value-added model for the student progress indicator on its school report cards we developed this new model called the added value growth model to reward year-to-year growth that places each student on the path toward standard proficiency designed for educators this video will give a high level overview of the added value growth model how we developed it and how it can help school and district educators to support students growth towards proficiency on the grade level targets historically south carolina has valued student growth and has included a measure of student progress in its accountability system several years ago the state used a value table to award points to schools when a student moved to a higher achievement level this criterion-referenced model was built on the criteria or the cut scores for student proficiency it encouraged schools to move students closer to meeting or exceeding the grade level standards some students start the year closer to one of those cut scores than others and unfortunately models like these were criticized for encouraging schools to prioritize the students who start out closer to the next achievement level who were sometimes referred to as bubble kids to prioritize the learning of each individual student the state moved to a norm-referenced value-added model in which the year-to-year increase in each student's achievement scores is compared to the norm or the increase that is typically observed for students like them statewide a school whose student's sc ready scores went up more than the norm for students like them is said to have a high value added while a school whose student scores went up less than the norm is said to have a low value added this norm-referenced model encouraged schools to focus on academic growth for all students which meant that it addressed the problem that was inherent in value tables unfortunately it also lost its direct connection to the criteria for grade level proficiency with this norm referenced model schools couldn't necessarily tell whether their students were growing enough to meet grade level standards just that they had grown more or less than average at the eoc we started to wonder if we could develop a growth model that drew on the best of both of these previous models we wanted a model that was built on the criteria for proficiency that would move students toward meeting or exceeding the grade level standard and we also wanted a model that encourages growth for all students the added value growth model is a criterion-referenced value-added model that provides individualized growth targets for each student in grades four through eight that if met would move them towards proficiency on the grade level standard and challenge them to continue to excel to explain how the model works let me explain a bit about how we developed it and that story begins with trying to understand how much growth is typical for students in south carolina we used historical sc ready scores from 2017 2018 and 2019 to understand how much student scores tend to increase from year to year on our tests by matching students who took the essay ready in 2017 or 18 and then who took the test again in 2018 or 19 we were able to use more than half a million student growth records to estimate what typical growth looks like for our students as we examined these data we found that average growth differs by grade level and differs by prior year score notice that students who score near the bottom on last year's test tend to show higher gains on this year's test similarly students who score at or near the top on last year's test tend to show much lower gains this year this meant that we couldn't use the same growth targets for everyone we need different growth targets for students in different grade levels and with different scores on prior year tests also not all students with the same prior year score will show the same growth in our data students who scored about a 389 on the fourth grade ela sc ready typically gained about 46 points on their fifth grade test but some students gained about 200 points and other students lost about 70 points this gave us the idea to use conditional growth percentiles to set growth targets to say that 46 points is typical growth for a student who scored 389 on the fourth grade test means that a 46 point gain is as good or better than 50 percent of students with that prior year score this is how we define the 50th conditional growth percentile also note that we will often use the simpler term growth percentile when discussing the added value growth model but we always mean the conditional growth percentile or the growth percentile among students with similar prior year scores growth in the 60th growth percentile would mean that the number of points gained on the current year test is as good or better than 60 of students with the same score on the prior year test for students scoring 389 on the fourth grade test the 60th growth percentile is about a 57 point gain the 70th growth percentile is about a 70 point gain and the 80th growth percentile is about an 86 point gain that means that if a student who scored a 389 on the 4th grade ela sc ready scores a 475 or better on the 5th grade test that student would have scored as well or better than 80 percent of students with similar prior year scores historically have we developed the added value growth model to give each student two individualized growth targets based on their prior year scores the first is the median annual target or the mat and it is set to reflect typical gains for students with similar prior year scores that is to say that the mat is set at the 50th conditional growth percentile since the existing norm referenced value-added model recognizes schools for student progress that is higher than normal for similar students the new model will also award at least one point for every student at a school who meets or exceeds their mit since that target represents normal growth for students like them unfortunately our analyses with historical data suggest that typical growth is insufficient to move students towards proficiency to illustrate that let's use the example of the average family adam average is a typical high achieving student who scored at the exceeds achievement level on the third grade ela sc ready if adam makes typical growth for students with similar third grade achievement then his score will go up 63 points giving a score of 642 on the fourth grade test which still scores at the exceeds achievement level in mathematics adam also scores at the exceeds achievement level and typical growth for students with similar prior math scores is 13 points which means that he would earn a 609 on the fourth grade test which also still exceeds the grade level standard annie average typically meets the grade level standard median gains for students with her third grade sc ready scores are 71 points in ela and 26 points in mathematics giving her these scores on the fourth grade tests which also meet grade level expectations their brother alberto approaches grade level expectations typical gains for students with alberto's third grade scores are 60 points in ela and 36 points in mathematics which means that he still scores at the approaches achievement level in fourth grade ashley average scores at the does not meet achievement level typical growth for students with her third grade scores would give her scores in fourth grade which also land in the does not meet achievement level now if we assume that the average children continue to make growth every year that is typical for students with prior year scores like theirs then we can use their fourth grade scores to estimate their fifth grade scores then we can use their fifth grade scores to estimate their sixth grade scores and then their seventh grade scores and finally their eighth grade scores so after five years of median growth very little changes none of the children in the average family have improved their achievement level and annie who met the mathematics grade level standard in third grade slipped into the approaches achievement level by the time she got to eighth grade in fact analyzing more than a hundred thousand historical third grade essie ready scores for each of the tests demonstrated that after five straight years of growth at the 50th growth percentile only 15 percent of students move to the next higher achievement level by 8th grade on the ela esee ready and five straight years of median growth led 32 percent of students to drop to the next lower achievement level by 8th grade on the mathematics essay ready although the added value growth model includes a median annual target for every student and will award at least one point in the accountability system for students who meet or exceed that target we also know that the mats alone are insufficient to change the landscape of student achievement in south carolina and that's where the second growth target comes in each student with a prior year sc ready score will also have an added value target or avt that is progressive based on prior achievement level and is designed to move students towards proficiency added value targets have been designed based on our analysis of historical data students whose prior achievement is at the exceeds level will have avts based on the 55th growth percentile because that was the lowest growth percentile at which no student who scored it exceeds in the third grade dropped to the meat's achievement level on either test by the time they got to the end of eighth grade similarly students with prior achievement at the meets level will have avts based on the 60th growth percentile because that was the first growth percentile at which no students who scored at the meets level in third grade fell to the approaches level by 8th grade to generate avts that are as appropriate as possible for all students we divided the does not meet and approaches achievement level in half at the median observed score because our analyses suggested that students with the lowest prior achievement needed to demonstrate gains in the 80th growth percentile to eventually meet the grade level standard students with prior year scores at the does not meet 1 achievement level will have avts based on the 80th growth percentile prior achievement at the does not meet 2 achievement level leads to avts based on the 75th growth percentile approaches 1 at the 70th and approaches 2 at the 65th growth percentile to understand how avts work let's visit another family as an example the added value family has six children each of whom score in a different achievement level on the third grade sc ready each of the students in the av family will make their added value target for five straight years so that we can see how their achievement shifts under this new model because alicia scored at the exceeds achievement level in third grade her avt for fourth grade is based on the 55th growth percentile thus she's expected to gain 70 points in ela which leads to a score of 649 on the fourth grade test still at the exceeds achievement level alicia is expected to gain 24 points in mathematics with an avt based on the 55th growth percentile and that will lead her to a fourth grade score of 620 also still exceeds anthony scored at the meets achievement level in third grade meaning that he has avts based on the 60th growth percentile or 84 points in the la and 42 points in math thus anthony earns a 578 and a 5 30 on the fourth grade test both of which still meet the grade level standard because amelia scored at the approaches to achievement level in third grade her added value targets are based on the 65th conditional growth percentile if she meets those avts she earns these scores on the 4th grade test notice that emilia now has met the grade level standard in english language arts although she still scores at the approaches 2 level in mathematics her score is now much closer to means and one more year of similar growth will get her to proficiency aiden scores at the approaches one level so his avts are based on the 70th growth percentile meeting those targets gives aiden these scores in 4th grade both of which are now at the approaches 2 level because ava scored at the does not meet 2 level in 3rd grade her avts are based on the 75th growth percentile which lead to fourth grade scores that move her to the approaches one level as the lowest scoring sibling alex has added value targets based on the 80th growth percentile which when met move him into the does not meet 2 achievement level note that the added value targets for 5th grade are based on 4th grade scores since none of the av family children have scored at the does not meet one achievement level in fourth grade none of them will have an avt based on the 80th growth percentile the avts based on the growth percentiles shown here lead to these fifth grade scores and achievement levels emilia now meets the grade level standard in both ela and mathematics and her other siblings have all moved closer to the goal again the avts for sixth grade are based on the fifth grade achievement levels leading to these sixth grade outcomes targets based on those prior year scores lead to these scores and achievement levels in grade seven which in turn lead to these avts and these outcomes in grade eight thus after five straight years of meeting their individualized added value targets all of the siblings in the added value family meet or exceed the grade level standard except for alex who meets the grade level standard in ela and scores in the upper portion of the approaches achievement level in mathematics if we apply five consecutive years of added value growth to the third grade sc ready scores earned by students statewide in 2017 and 18 we can see the potential benefit of this new model for south carolina in this visualization each of these figures represents five percent of students in the state and the color codes represent the four achievement levels of exceeds meets approaches and does not meet for example in 2017 and 18 combined about 17 and a half percent of third grade students statewide scored at the exceeds achievement level in ela and about 25 percent of the students scored at the exceeds achievement level in mathematics if all these students meet their avts then the distribution of achievement in fourth grade looks like this and if they all meet their avts again in fifth grade the distribution of achievement looks like this subsequent years of added value growth leads to this picture in sixth grade this picture in seventh grade and this picture in eighth grade five years of added value growth changes the picture of achievement in south carolina from what we see on the left to what we see on the right moves all students towards proficiency maintains or improves initial achievement levels and exceeds the state's goals for student achievement which suggests that the added value growth model is a better fit for south carolina than our current model and better supports the goals of educational accountability here in our state one of the features of the added value growth model that we're very excited about is the potential to work with interim and benchmark assessments that educators give throughout the school year to gain insight into whether students are growing enough in their academic achievement to meet or exceed their avts on the state summative test at the end of the school year we used conditional growth percentiles to set the added value targets because these kinds of growth percentiles can be used and calculated for virtually any assessment system that has a long enough history of student scores to analyze we're already working with the vendors of state approved interim and benchmark assessment systems to help them develop ways to support their clients success with this new growth model we encourage educators in south carolina to talk about the added value growth model with their interim benchmark assessment providers to see how they can help you monitor whether your students are making sufficient growth throughout the school year to hit their annual targets finally you might be wondering how this new model will be scored and how schools will be rated with it keep in mind that the method of scoring for the added value growth model and the method of converting those scores to school ratings has not yet been determined that work happens this year using the additional data that we were able to collect from the previous school year we have determined some basic features of that scoring system however which among many other things ensures that the growth metric is not overly correlated with the proportion of pupils in poverty served by a school students who fall short of historically observed median growth will not earn positive indicator points for their school students who meet or exceed their mit will learn at least one indicator point for their school students who meet or exceed their added value target will earn substantially more indicator points for their school how many points will depend on the growth percentile on which their abt is based with students whose avts are based on higher growth percentiles earning more indicator points than students whose targets are based on a lower growth percentile finally as with many of our other indicators the mean of the indicator points earned by all students at a school will determine the school's rating points which in turn will be compared to cut scores for various ratings 2023 report cards will simply report the proportion of students at a school who meet or exceed their mit and the proportion of students at the school who meet or exceed their avt this year points and ratings will still be based on the existing norm-referenced value-added model the added value growth model will be both reported and scored on the 2024 report cards now we recognize that one short overview video probably isn't quite enough to adequately describe this model and its implications for practice we plan to post additional videos in the coming months to provide more information so that stakeholders can be well informed as these changes to accountability roll out in our state in the meantime don't hesitate to reach out to eoc staff with any questions that you may have about the added value growth model as always thank you all for all that you do for south carolina students