The Truth behind the PSSA Data

When I started working at Centennial School District with Conor Corey in August of 2017, with instruction to continually improve the math scores district wide. We made sure to set a few short and long term goals never really considering many of the hurdles that we would face along the way and certainly not the amount of success that has been achieved. We were excited that we finally had an opportunity to use Khan Academy combined with our teaching system that we had developed and perfected over years of using this program. It was all or nothing for us on a system and belief that had yet to be proven in mainstream public education.

That September, we started training the elementary staff on what Khan Academy is and the results that we have seen using it in the School District of Philadelphia. With all the new district initiatives that come in September, we didn’t want to put too much pressure on the teachers to use it right away. Our goal was to completely set up infrastructure and allow all stakeholders to use this learning platform when they were ready. We used Khan Academy’s first National Academic Competition called LearnStorm to motivate the students and get them acclimated to using the program. This competition was designed to invigorate students and focus on the importance of Growth Mindset. Throughout this competition, the Centennial community combined for the most active minutes of any district in the country resulting in 2 out of 3 elementary schools placing in the top 25 nationally.

Starting in November, we used MAP/PSSA scores, Khan Academy, our own teaching experience, and other factors to focus in on groups of students across the district who were demonstrating foundational gaps that we believe prevented growth and confidence about their mathematical ability. We also identified students that were using Khan Academy so frequently that they were passing their current grade level with ease. We started remediating and accelerating concurrently with these students multiple times per week. An example of this can be seen below:

These students started putting in a lot of time on Khan Academy during school, and were putting in even more time AT HOME! This had a domino effect on the surrounding students and teachers. As more and more students began using the program, more teachers became open to using it in the classroom. This is when Khan Academy took off exponentially. Students in grades 2-5 were using the program on a daily basis. Teachers were assigning homework in the Khan Academy application and were implementing it in centers. Teachers were even using it to master their own grade level content! Soon the middle schools became interested in using the program with the 6th-8th grade students. By the end of December we had all 600+ students from one of our middle schools using the program simultaneously, at least one day a week, for 30 minutes. With this type of support from teachers across Centennial School District we were actually looking forward to standardized testing.

However, during the summer before the 2017-18 school year, the state decided to make some dramatic changes to the state tests. The state reduced the amount of multiple choice questions from 60 to 40. Initially, there was some positive feedback following this announcement about the decrease in time and questions on the PSSA (Pennsylvania State Standardized Assessment). We did not feel exactly the same way. For us, we know that many of our struggling math students also struggle in literacy and comprehension. Eliminating 20 multiple choice questions resulted in an increased weight of the open response questions which put our ESL (English Second Language) students at a further disadvantage. Effectively, they took away 20 chances for students to prove what they know and place higher importance on a part of the test that is reviewed by different eyes for each student and extremely subjective in nature. As might be expected, this change made the test more difficult than it had been in previous years. However, our students had completed over 1.3 million minutes on Khan Academy, and they had put in a lot of hard work and were ready to show what they had learned.

To say we were anxious to get the scores back over the summer was an understatement. We had staked our reputation and careers on our system and this program working on a wide scale. Finally, the scores came back in early July. The results on the surface, to our administration, were not good. The math scores had basically dropped across the board. But after years of experience, we knew the importance of looking at all the data before rushing to a conclusion. Working in Philadelphia for the majority of our careers had prepared us to alway focus on growth rather than a subjective grade level standard. We also knew our students and teachers worked too hard to just be told that “math scores are down”.

Being the math nerd I am, I kept track of all data from the 2017-18 school year. When we compared Khan Academy data for student in grades 3-8, PSSA from 2017 and 2018, and MAPS/FastBridge testing we found amazing results. We had closed the gap! Students that spent an average of 30 minutes a week on Khan Academy were 2.5 times more likely to be proficient on the state tests. If a student used Khan Academy for around 1000 minutes a school year, there was over a 50% chance they would meet state standards. This held true in all grades from 3-8. If we looked at the data comparing MAP, we found that middle school students who met 30 minutes a week achieved 1.2 times their predicted MAP growth. Elementary students with the same usage achieved 99% of their predicted MAP growth. In contrast, if they used Khan Academy less than 15 minutes a week, they fell short of their predicted growth by 15% and 20%, respectively. Arguably, the most interesting insight derived from this data was that there was no difference in these trends when you filtered by student demographic, ESL, and/or economically disadvantaged. This may not seem like a big deal but to many that review student data on a large scale, this is a stat that is rarely seen and sought after for every implementation strategy.

Following this collection of data, it seemed nobody wanted to see it or cared. The scores went down and that is all that anyone wanted to see. So we decided to take a different approach and contacted Khan Academy directly, who at this point had become very familiar with our story and our efforts. The research and efficacy team at Khan Academy not only corroborated our findings but began publicizing the efforts and success of the teachers and parents in Centennial School District. Just this past week, Sal Khan himself even mentioned our findings and Centennial School District on CBS News as seen in the video below:

https://www.cbsnews.com/video/khan-academy-founder-announces-mastery-learning-features/

Khan Academy also posted this on their blog:
https://www.khanacademy.org/about/blog/post/177799922435/school-district-reports-test-scores-rise-with

And issued a press release:
https://www.prnewswire.com/news-releases/khan-academy-introduces-new-mastery-learning-features-300708027.html?tc=eml_cleartime

The 2017-18 school year was truly amazing. The level of success that Centennial School District achieved could not have been realized without the support and tireless efforts of the principals, teachers, teaching aids, building engineers, cafeteria staff, and many others within the district. We want to thank them for all that they do for the students and us each and everyday. We would also like to thank the Centennial School Board and Administration for their belief in us and letting us captain this ship. They allowed us to evolve this program using our model and vision and removed many hurdles along the way. Khan Academy has become a part of the culture at Centennial in only two short years.

The bar is set high for the 2018-19 school year. Due to the success of Khan Academy, Centennial School District has decided to pilot Eureka Math across all grades. Eureka Math is a mastery based curriculum that integrates seamlessly with Khan Academy. In addition, LearnStorm 2018 has started and our students and teachers are already hard at work. We are coming to defend the title and are bringing 6000+ students with us!

If you would like to follow us through this school year, you can find us on Twitter and Instagram below.

Twitter:

@JoshBGerloff & @CCorey223 & @OERKhanAcademy

Instagram:

@JoshBGerloff & @OERKhanAcademy

Share this: