Episode 3: Scaling STARI

Speaker A: Welcome back to SERP Stories, a podcast where we pull back the curtain on research, practice, partnerships, transforming education. I'm your host, Dr. Kayla Jones. This episode focuses on what it takes to scale a promising practice and how our team at CERB has navigated the unexpected along the way. Scaling, starring our strategic Adolescent Reading intervention was never just about getting bigger. It has always been about getting better, with a clear focus on improving learning outcomes for the students who need it most. The goal of SCALE was not only to reach more classrooms, it was also about learning, listening, redesigning over and over and over again alongside those educators who implement STAARI in their classrooms. Here's our story of how a small nonprofit is navigating the scaling of a tier 2 middle school reading intervention through a decade of challenges, pivots, and yes, grow. Leading this charge was and still is Dr. Margaret Troyer, Serb's director of literacy Research and Development.

Speaker B: One of the lessons that I learned in my first year of working at SERP is that the work is never done at serp. We're always in a process of continuous improvement.

Speaker A: And that mindset, continuous improvement, has guided every phase of Starry's growth. Margaret's first year at SERP was 2018, but that wasn't the first time she heard about Starry.

Speaker B: I came on board, um, when the data analysis from the 1314 school year was being done, and I was a grad student.

Speaker A: Margaret was a doctoral student at Harvard and worked alongside Dr. Jimmy Kim, who you may remember hearing about in a previous episode. He led the data analysis of the first Starry study. If you hadn't heard our last episode, stop, go back and listen, then come back. But to summarize, I mentioned that those study results were analyzed, and they were remarkable. The first randomized controlled trial of STAARI had found that students made more than a year's worth of additional reading growth, outperforming, uh, peers in decoding, fluency, vocabulary, and comprehension. But beyond the numbers, STAARI changed students interest in comprehending text. Here's a seventh grader, Jasmine Starry has

Speaker B: helped me become a better reader because

Speaker A: talking with others and hearing the opinions of others can help me learn more about what I'm reading about. Once the study wrapped, SERP made the Starry materials available as free downloads, a reflection of our core mission to generate solutions that anyone can access and use. The team built a website with basic information about the program and a download center with the student workbooks, fluency passages, and teacher lesson plans. Just like that, STAARI curriculum materials were no longer limited to middle schools and our research study, but were available to anyone, anywhere, anytime. With grant funding coming to an end and a website that cost very little to maintain, STAR was released into the wild, ready to scale. Or so we thought.

Speaker B: A lot of our early adopters of Starry were reading specialists who were tasked with teach these kids to read without being given a curriculum. So they were googling and they found Starry. And because the PDFs are free to download, they downloaded, they printed, and they started using it with their students.

Speaker A: But the team didn't stop there. They submitted the study findings to the federal what Works Clearinghouse, the gold standard for education research in the United States. Fast forward a few years. STAARI earned a rare distinction. It met what Works Clearinghouse's standards without reservations. That meant that the team's research was rigorous enough to remove all doubt. STAARI worked. Let me reintroduce Dr. Suzanne Donovan, SERP's founder and ED, who we heard from in previous episodes.

Speaker C: Because the STAARI research project as part of the Reading for Understanding grant had gotten positive results and because it's so hard for any reading intervention for middle grade students to show a positive result, it was a big deal.

Speaker A: It was a big deal to get to this point. Not just for Starry to earn Tier 1 evidence of effectiveness from the what Works Clearinghouse, but to make a comprehensive research based middle school intervention freely available for anyone to download and use. But just because it was a big deal didn't mean SERP would have the funding to support the intervention. The team celebrated milestones, but also grappled with a challenge that is all too familiar to those who develop products through R and D grants. How do you turn a temporary investment into a sustainable solution? Also, when something gets released into the world for use in new settings and contexts, there are new questions, considerations, possibilities, and constraints. As Starry was being downloaded in various places, teachers and administrators raised additional questions. How will schools make sense of the program? How will it fit into existing schedules? How will teachers and coaches be trained? How will schools get the companion materials like the novels and timers used in staari? And will it even work beyond Massachusetts? Answering these questions work were critical to expanding staari's reach and ensuring it worked well wherever it was implemented. But because the STAARI curriculum was available for free, it didn't generate revenue to support the staff needed to refine and update it. As a nonprofit dependent on grant funding, SERP needed new resources to keep improving STAARI and responding to the needs of districts, schools, teachers, and most importantly, the students. That's when we realized that Staari's Tier 1 evidence rating opened the door to a major funding opportunity from the US

Speaker C: Department of Education that positioned us to apply for an EIR grant to further develop STAARI to meet the needs of non Massachusetts districts.

Speaker A: EIR stands for Education, Innovation and Research. It's a federal program designed to help promising education projects grow beyond their local roots by allocating federal funding for research and development. In late 2017, SERP secured an EIR grant, enabling the team to further refine STAARI based on educator feedback and to recruit additional schools for a larger follow up study. With this new funding, the SERP team, led by Margaret, who had just completed her doctoral program and was hired by SERP to lead this work, was able to focus on five key challenges. One, helping administrators understand and adopt the curriculum two supporting teachers with implementation three making STAARI training accessible four demonstrating STAARI's effectiveness beyond Massachusetts and five building a model that could be sustained over time. Over the course of this episode, we'll dig into each one. Each challenge demanded close collaboration with teachers, schools and students, and persistence when things didn't go as planned.

Speaker B: The 2018-19 school year really gave us the opportunity to get feedback on our curriculum from the teachers that were working with it. And at that point, you know, we thought the curriculum was done. We already had evidence of effectiveness. Now we're just trying to scale into other regions.

Speaker A: But there was still a lot to do. The team's first challenge became clear. How do we ensure STARR speaks to the needs of the decision makers, school and district administrators?

Speaker B: Uh,

Speaker A: Challenge one Making it make sense for administrators Even with strong results from the first study and a dedication to user centered design during the development process, when CERT began recruiting schools to participate in the second study of staari, there was a major point of confusion for administrators. Suzannah Margaret quickly discovered that scaling a program also means aligning with how schools are structured.

Speaker C: Middle schools told us, we really expect a, uh, middle school program to be a three year program.

Speaker B: The researchers who originally designed STAARI had made two series, and each series had four units. Some kids after one year of Starry might be reading on grade level or close enough to grade level that they don't need an intervention anymore. Other kids, depending on how far behind they were at the beginning, they may need a second year of Starry in order to get on or close to grade level. So two series made sense. But when we went into schools with Those two series, the question we were getting is like, but middle school is three years. Which series do the seventh graders use? And the other thing we learned was that the vast majority of teachers were not getting to the fourth unit. So those fourth units, which contain some really great books and content, were basically sitting on the shelf gathering dust untouched. So we decided to take our two series of four units each and make it into three series of three units each. One one series for sixth, one series for seventh, one series for eighth.

Speaker A: This was harder than it sounds, since the fourth unit works on different skills than the first three units. So there was a lot of rewriting, but it now made sense to administrators. Since then, administrators have continued to influence the Starry's development. Suzanne, what do they say?

Speaker C: We heard from high schools that, wait, we really need this high school. So we developed a high school version.

Speaker A: And it was an honor to be a part of the development of the new series four for high school. To see for myself how challenging book selection is for this population of students, how teacher input regarding background knowledge needs led to fluency passage development, and how the high school scheduling considerations led to other changes. Now we have a program that can support struggling students beyond middle school. We'll revisit the development of series four in another episode.

Speaker B: Uh,

Speaker A: Challenge two, Making it work for the teachers. Okay, let's rewind to our list of challenges. Once administrators were on board and Starry was in classrooms, Zepp was getting feedback from teachers. These teachers had a different perspective from the teachers in the first study who were part of the development work. These teachers were given a complete program and tasked with implementing. They faced practical challenges. Pacing lessons, managing materials, and figuring out how to monitor student progress. And these challenges echoed some of the questions we had gotten from early adopters using Starry. Once the CEHRT materials were made available for free online, each grant provided additional funds to be able to work directly with teachers who were implementing STAARI and learn from their experiences. Margaret, tell us about these changes.

Speaker B: The vast majority of teachers were not getting through the number of units that we hoped they were getting through. Um, they were moving too slowly through the lessons. So we added timestamps we with an estimated length of time they should be spending on each activity. So this activity should take 10 to 15 minutes. This one should take 15 to 20 minutes. And you know what, if your kids are having a really robust discussion and it goes 25 minutes, fine. But if it's going 45 minutes, then there's an issue with pacing that needs to be addressed. And that helps the coaches too, um, to be able to call that out.

Speaker A: So Starry wasn't just being studied. It was being stress tested in new classrooms.

Speaker B: And then another piece of feedback that we got was that there's just a lot going on with Starry. There's a lot of materials, there's the teacher lesson plans, there's the student workbooks, there's the fluency passages, there's the slides. So one thing that we did was we added thumbnails of the workbook pages to the teacher lesson plans so that as the teacher is looking at their lesson plans, they can see a little picture of what the students should be looking at in their workbook. So that was another small change just to improve usability.

Speaker A: And once again, teacher feedback drove innovation. This time a, uh, paper and pencil progress monitoring guide.

Speaker B: Margaret Another question that we kept getting over and over again was, what about progress monitoring? How do I know whether my students are making progress in staari? And that was kind of a tough question because a lot of reading interventions out there have assessments that are closely aligned or even embedded within the computer program. And we did not want to add another assessment to staare students schedules. We really felt like these kids are already taking a lot of tests. We don't want to take any more instructional time away to focus on assessment. So what we did was we created an assessment guide. It's a paper and pencil guide, um, that does have a pre and post assessment for each unit. Um, that's a brief passage with some questions that are really tightly aligned to the skin skills taught in that unit. And then we flagged specific assignments, specific workbook pages within that unit for the teacher to collect and grade in order to monitor students progress on, um, the decoding skills and the comprehension skills that we're teaching within that unit. We also included some guidance about how to know when your students are making adequate progress, what to do if this progress monitoring shows you that students are not really making the progress that you hope they were making. Um, and so that's something that we hope can support teachers and support administrators in really knowing whether students are making the progress that they should be making. In addition to their scores on whatever assessments they're already taking, um, during the school year, they should show progress on those assessments too. As a result of being in staari,

Speaker A: each change, pacing, visuals, progress monitoring came directly from the classroom.

Speaker B: No changes that really affected the heart of the curriculum or the theory of change, what we believe about how kids learn to, but just changes that made the curriculum more usable for teachers.

Speaker A: And that constant back and forth between research and classroom practice is what gives Starry its staying power. But updating the materials wasn't enough. Teachers also needed training, which leads us to challenge three training teachers at scale with a small team and a limited travel budget. Once Starry was out in the world for anyone to use, we were immediately asked, how do I get trained in staari? Even though the comprehensive teacher lesson plans allowed some teachers to use STAARI without training, administrators and other teachers were all looking for training. In response, SERP hosted a STAARI institute each summer with registration fees where anyone could attend and be trained in staari. It was all expensive as none of it could be subsidized by material sales or grant funds. And it prevented a lot of motivated individual teachers or teachers in small districts from getting the training they wanted because it just wasn't accessible. With the EIR UH grant, the SERP team began experimenting with training modules containing expert led instructional videos for various Starry components with lots of classroom video and tips for implementation. Just as the team was building out a blended model of training with in person and video modules, everything changed.

Speaker B: Prior to 2020, the only way that we had to deliver STAARE professional learning was at these multi day in person summer institutes. Then Covid hit. We could no longer do in person professional learning. So we had to pivot in record time.

Speaker A: The team developed online training modules for a comprehensive Starry professional learning series.

Speaker B: We started putting videos online. Some videos of us talking, some classroom videos. We did it fast because of COVID but then we took the next year or so to really build it out and to a comprehensive and if I do say so myself, very high quality professional learning series.

Speaker A: The pivot born out of necessity became a cornerstone of how Starry would scale from that point on.

Speaker B: We now think is better than what we had before for a number of reasons. One, it's scalable. We're a small staff. We can't travel to every school district in the country to lead in person professional learning. Two, it's affordable even for large districts, it's expensive to fly two SERP staff members out there and pay for two days of our time to lead that professional learning. And for small districts that may only have one or two starry teachers, since it's a tier, two, it's completely impossible for them to afford this in person professional learning. Three, it's flexible. When we did in person professional learning, a teacher who couldn't be there on that day at that time had had no way to access the material. Having it all online and asynchronous means that teachers can do it on their schedule. And we know that best practice in professional learning is for it to be job embedded and completed over time. So having it online and asynchronous means that it can be spread out over the course of the school year, um, using sort of a just in time scheduling model so that teachers don't have to get it all in one big dump and then just try to hold onto it and retain it. So we had that professional learning online. We felt like this is great. This is so much better than what we've done before.

Speaker A: Better and scalable. One of our sorry coaches in Jackson, Mississippi, Patrice Williams Razor, was asked about the benefit of this approach as well. There are videos there that are helpful where they can see another teacher's approach to teaching components of the lesson that they may not quite understand from their teaching the lesson or from my modeling the lesson. I think that's important for them to be able to see multiple perspectives. So I think having access to the online component can definitely help to deepen their knowledge and not only that, to understand the why and the how behind some of the things that are done within the program. But once again, it didn't stop there.

Speaker B: One of the goals of the ER midphase study was to build capacity for districts to be able to support SARA implementation on their own. So we developed this coach Professional learning series series. It's an add on to the Starry teacher professional learning series. So the idea is that coaches will learn the program along with their teachers by viewing and participating in the modules about the components of Starry. And then for each component of Starry, for each session of the professional learning series, there's an add on for coaches saying, what are some of the things that teachers tend to struggle with when implementing this particular component? And how can you support teachers in working through those challenges with this particular component?

Speaker A: And it still didn't stop there. In 2024, New York City Public schools made a new request. Live full day virtual training sessions. Syrup hesitated. Six hours on zoom sounded unrealistic.

Speaker B: And at first we said, no, this is a horrible idea. Nobody wants to sit at their computer for six hours attending a virtual training. And New York said, this is what we're doing. Can you do this? So we reluctantly agreed. And it went so much better than we thought it would. By January, New York was reaching out to us, saying, all the Starry trainings for the rest of the year are full with a wait list. Can you add some more?

Speaker A: And we've heard from many teachers that it works with feedback from our latest training, including this gem from a New York City teacher who said, and I quote, this is the best PD I've been to in a long time. This model, online, asynchronous for teachers and for coaches and live virtual options, is now central to how STAARI scales. Uh, next up, challenge four Proving STAARI Works Beyond Massachusetts The EIR funding provided a huge opportunity to make changes to the curriculum and training. But more than that, this second study was pivotal because to scale nationally, SERP needed to show STAARI could succeed in places that looked nothing like Massachusetts. Having proof that STAHRI worked in Massachusetts didn't automatically convince educators that STAARI would work in new contexts. Suzanne, can you tell us more about the second study?

Speaker C: We had four districts involved. Jackson, Mississippi, um, Washington, D.C. new York City, and Baltimore, Maryland.

Speaker A: Four very different districts. Jackson, in the rural South, Washington, a political capital with its own complex school system. New York, a sprawling urban metropolis and Baltimore, a city with deep history and challenges of its own. Each community brought a unique context the team needed to understand, and they were eager to dig into those differences. And those four districts gave the team a chance to see how Starry performed in different contexts. But an unexpected challenge arose. A global pandemic. And as schools responded to the unprecedented challenge, there were ripples of impact on every facet of life, including the second STAARI research study.

Speaker C: The pandemic got in the way and so we were on hold for over a year and were unable to get back into schools.

Speaker A: Data from two years of the study became unusable. And when we finally were able to get back into schools, there wasn't enough funding to do the study as intended for an unplanned, um, third year.

Speaker B: When we were able to get back into schools in 2122, Staari was needed more than ever before because at that stage we knew that students had lost quite a bit of ground during over a year when school was closed. Everybody was talking about learning loss. Some scholars were saying that there was no way that students were going to make up for the lost time, or at least they weren't going to make up for it without high dosage one on one tutoring, which under resourced school districts did not have the capacity to provide. And this was still a really challenging school year. Um, there were a lot of student absences, teacher absences, time lost to social emotional needs from students returning from a year and a half of not being in school.

Speaker A: When the data came in, the results were powerful. Margaret, share the good news.

Speaker B: We were able to get a statistically significant effect on the Mississippi State ELA test, um, for our students in Jackson. And that's actually a really big deal because most intervention research, particularly focusing on adolescents, does not show an impact on state standardized tests.

Speaker A: Yes, in this study, staarry students grew at ah, twice the expected rate for middle school students. And the growth they made exceeded the estimated learning loss caused by Covid related school closures. Even with all of the twists and turns, the outcomes were noteworthy. STAR's positive impact was affirmed once again with two rigorous studies, first in Massachusetts and now in Mississippi. The evidence base was growing. But even a proven program isn't necessarily sustainable. Which brings us to challenge five building something sustainable. Uh, let's hear how Suzanne thought about this challenge.

Speaker C: Initially our thinking was if we wanted to have partnerships with school districts and researchers, then we shouldn't be making money on the products of the work. Uh, and then there's also the fact that the federal government was paying for the research with taxpayer dollars. So we thought we should make the product available to anyone free of charge. But there were two problems with this. First, without a revenue stream, uh, we couldn't maintain the program. Even answering questions and building and maintaining a download setter come with costs. The second thing is that large districts that were accustomed to purchasing programs and having everything packaged and delivered were not really interested in downloading and printing the lesson plans and workbooks and then purchasing all of the literature and supplies on their own. We really needed to make this easy for them.

Speaker A: And here's Margaret's take.

Speaker B: As we were talking to superintendents and principals from other districts, one of the first questions was how to order the materials. And when we said, oh, it's free, you can download it for free. They said, but who is going to print hundreds of copies of these materials? Who's going to make sure that the photocopiers and all the buildings are working to be able to do that and buy all the paper and ink? I've had one assistant principal tell me that it is actually more expensive to print starry materials in house than to order them because paper and ink are so expensive. So as we scaled, we've learned that districts are often more interested in ordering convenience than in keeping it free.

Speaker C: We needed to develop a business strategy, but we weren't quite ready to give up on those initial commitments. So we decided on a, uh, both and strategy. We kept the SERP materials as free Online downloads, so any teacher who wanted to teach the program could. And we also developed partnerships with printers and book distributors to provide convenience for districts with a small markup and licensing the professional learning series, we're now able to maintain the program.

Speaker A: Many people think the solution to the problem cert faced is to turn the program over to a for profit publisher. Why didn't you take that approach?

Speaker C: I'd have to say we were a bit skeptical about the profit motive. I've seen publishers, once they own a program, make changes in order to feed their bottom line, which is expected. They are for profit companies, but in education, sometimes those changes are at the expense of what will work best for teachers and students. So we decided to maintain control over the product.

Speaker A: But Suzanne, you have a research background. What made you think you could pull this off?

Speaker C: That's a really good question. You could ask that about SERP more generally. We really have no blueprint for anything we've done, but we have a problem solving culture in the organization and a, um, really terrific team. Um, Ali Weige and Beverly Hofmaster, both long time SERP employees, just began putting the systems in place for initial sales. And then we continuously improved the process to make it ever easier for schools and districts to order. And the system's still becoming more efficient all the time.

Speaker B: We developed this bundle order system they can just put in, you know, this is how many sixth grade teachers and students I have, this is how many seventh grade teachers and students I have. And it'll populate a quote telling them exactly what they need and they can get it all shipped to them from serp.

Speaker A: So practical solutions to end user feedback, a SERP storefront and bundled curriculum and booksets. And even though this was a response to districts requesting ordering printed materials for convenience, it helped SERPs and star sustainability too to build and grow a source of funding that can be cycled back into improvements to curriculum materials and professional development in the future. But scaling came with new responsibilities too, like supply chain challenges and logistical considerations. For example, when one of the Starry's core books, Soldier, went out of print, the team had to negotiate with publishers and bulk order 10,000 copies to keep classrooms running. Since then, this has happened with a few titles, and over half of those copies continue to live in SERP's tiny 13th floor office space in Washington D.C. with orders shipped out as they come in. Through this entire experience, we've learned that scaling isn't just about pedagogy. It's about logistics and learning new capabilities while keeping students and teachers at the center of decision making.

Speaker B: A, uh, publishing company would have a lot more capacity than we have to scale up the program to market it. We are going to continue following the contours of a problem and finding ways to improve the program based on the feedback that we're collecting. And a publishing company is probably not going to do that. They want it done, wrapped up, with a bow on it and out the door not to reopen the curriculum again because they have a new idea about a way to make it better. And that's what we do at serp. We keep on having new ideas about ways to make it better.

Speaker A: Today, STAR is available nationwide in urban, rural and suburban schools. Teachers can still download materials for free, or districts can order curriculum and literature and supplies so they have everything they need. And the work continues. We are currently studying STAR in even more places. In fact, we've just wrapped up recruiting new classrooms in California, Illinois, Maryland and Wisconsin. I'll save the Starry recruitment story for another episode, but suffice it to say, it is clear to me there is a need in our nation's schools that STAARI can solve from coast to coast. And in districts near and far from staari's origin in Boston. The core STAARI model has held fast evidence based instruction, engaging novels, and meaningful classroom conversation. And the scale is significant. STAARI is now used in nearly every US state plus DC and in a couple other countries too. In the last five years, there has been over a quarter million downloads of Starry curriculum. And our vision for Starry is clear. To become self sustaining, nationally known, and continuously responsive to the evolving needs of educators and students. But not for size, for impact. Suzanne, what's your hope?

Speaker C: Well, I guess my hope for a Starry's future is a hope for kids who are struggling with reading. I hope that they all get the opportunity to be exposed to a program that can move them to becoming, uh, capable and confident readers. We hear from many Starry students that for the first time, they are actually enjoying reading. So my hope is that we are able to support many more schools to give students that opportunity to find the joy in reading and the success that comes with being a, uh, confident and competent reader.

Speaker A: Um, after a decade of scaling, what have we learned? Scaling isn't linear. It's not about growing for growth's sake. It's about responding to data, to teachers, to the unexpected.

Speaker C: So with each phase of development, we learn more about what districts need to be successful in implementing the program and to allow the program to spread and, and we took those problems on one after another, and that's really what we set out to do as an organization to stick with the problem of practice and to solve all of the components of that problem.

Speaker A: In the end, that's what scaling starry really means. Staying with the problem until it's solved and then staying a little longer to listen to what still needs. Work and sustainability requires more than just good intentions. It requires infrastructure, flexibility, and a willingness to keep solving problems one after another.

Speaker C: All of the phases of STAARI reflect layers of problems that emerged as we did the work that we could solve one after another. What Richard L. Moore referred to as following the contours of the problem over time so that we work towards something that is a genuine solution to a problem that many districts are struggling with.

Speaker A: Because behind every research study and curriculum guide are teachers trying to reach readers and students like Jasmine, finding their voices through engaging literature and lively classroom discussions. In the end, scaling starring was never just about getting bigger. It has always been about getting better. And that meant staying curious, adaptable, and truly listening to practitioners. Before we sign off, I always ask every guest this season one question. How did you learn to read?

Speaker C: Here's how Suzanne answered My reading journey started as, uh, the fifth child in a family of 11. My parents read to the first three and then as the fourth and fifth came along, no more reading. By the time I was a little kid, there were no more books that made it, uh, through a life in the house. So reading was entirely what I got in school. And I began as a slow reader, got a lot of negative feedback, um, because I was in a Catholic school and the nuns told me that I did IQ testing, that I had an A high iq. So why wasn't I performing better on all of the other tests? And the answer was I wasn't finishing. I was reading too slowly to finish the test. And I always felt like, well, if they just gave me time to finish. But I was a slow reader. And I was a slow reader all the way until high school when there was a library in our school. And then I had access to books and then I began reading on my own. And that was the beginning of, um, really loving to read. But all through K to 8, reading was sort of an Achilles heel for me.

Speaker A: Um, thanks for listening to this episode of Serb Stories. To learn more about Staari, visit serpinstitute.org Starry and if you believe like we do, that middle school readers deserve better, share this episode and follow along as we bring more research with purpose to the mic. I'm Dr. Kayla Jones. See you next episode.