Finding the Students Schools Miss

Speaker A: Foreign. Welcome, uh, to the latest episode of Tech and Learning Conversations. I'm Kevin Hogan and I'm glad you found us. This is the podcast where we talk about the educators, leaders and innovators shaping the future of K12. Today's guest has spent his career asking a deceptively simple question. What do we actually know about kids? And are we using that knowledge to help them? AJ Gutierrez is the new CEO of Equal Opportunity Schools, a ah, national nonprofit working with more than 900 schools across 35 states to identify students who are ready for advanced coursework but are too often passed over. Before this role, he co founded Saga Education and helped build the high impact tutoring movement into a national priority, securing more than $7 billion in public investment along the way. So he's here today to talk about data AI, chronic absenteeism, and what it actually takes to find the students that schools are missing. Have a Listen. Okay. A.J. thank you so much for joining me today. I really appreciate it. I'm looking forward to, uh, hearing your insights.

Speaker B: Oh, thank you so much for having me and looking forward to the discussion.

Speaker A: And we might as well just jump right into the weeds here. Um, the work that you're doing, the work that you have done and the work that you've just begun to do, uh, with your new role at eos is really, um, I wouldn't, I won't say controversial. I just said it's, there's a lot happening in the space, right? I mean between how we use data in, in schools, um, the importance of personalized learning, um, the importance of what higher education means and the importance of whether or not kids who are quote, advanced are going into college at all. There's a whole different range of ways we can go. But this being tech and learning conversations, I kind of want to focus a little bit on the technology side of stuff. But first let's talk a little bit about how you got into this space and what inspires you and what inspires you about your new position.

Speaker B: Well, I mean, if I had a draw through line for my entire career, it's really being thoughtful about research and evidence base and how that's shaping decisions we're making. Uh, in K12, uh, you know, this innovation apparatus is R and D apparatus that ah, you see in other industries in healthcare and technology, that doesn't exist the same way in education, especially when we're talking about things like rigorous randomized control research studies. Those are the gold standard for scientific research. When I co founded Saga Education and helped cultivate the high impact tutoring movement to what I'm currently doing with Equal Opportunity Schools. That's the lens in which I look at things in my new capacity. I started as a CEO, uh, at, uh, Equal Opportunity Schools in July. So I'm still going through the hazing process as if you will. You know, they basically ask, hey, A.J. we would love for you to come in, help us think about our new strategy. Given all the changes in the landscape.

Speaker A: Right.

Speaker B: What you described, Kevin, and as a leader, how I go about doing that is thinking about what is it that an organization does really well, like what's their core competencies, what are their superpowers, and how can that be a core solution to a problem and the environment. And, and along the way, what I learned that Equal Opportunity Schools did really, really well is generating meaningful insights about students. Initially, when Equal Opportunity School started, it was about, hey, we have a leaky talent pipeline in both ends. There's a huge subset of students who are ready for advanced coursework who were, for many reasons, were overlooking. And so if you look at things like the PSAT merely, uh, there was a large study that EOS conducted with, um, Ed Trust, where they were missing on 80% of who would have qualified. So Eco Opportunity Schools did really great work doing that for many years. And I'm coming in saying, hey, there's so much more in which we, where we can apply these insights to help support meaningful decisions we're making in K12. And this is really important as we think about the future schooling, as we think about AI or whether we want micro schools, really important discussions about that I think should be really centered on what we know about kids. And going back to other industries, people are obsessed with their customers, the thought processes, their motivations, their aspirations. And I don't think that same level of obsession, uh, exists the same way in K12 education. So if you want to be serious about developing a new AI or chatbot, what do you really know about kids that's going to help support that? And this is not just from an equity play. At the end of the day, this is what good businesses do. And as I think about the future of Equal Opportunity Schools, um, it's going to be really centered, uh, around generating this type of evidence and information about kids to support what practitioners are doing, um, during the day, supporting kids. And so that's what we're in the process of doing now.

Speaker A: I think going to your point about comparing education in other industries, one of the things that's interesting I found over the years, talking to folks like yourselves and to district leaders is like that end customer. It's a two tiered, if not even a three tiered customer level. Right. I mean we always talk about the kids, it's all about the kids. But I know a lot of the work that you do that gathering that data and that insight is important to the faculty member to then help the kid and then by extension explain to the parents what the situation is via the faculty.

Speaker B: Right.

Speaker A: So I mean that ends some, add some elements of complexity to it. Can you talk a little bit about um, the cards and how you do it, how EOS does it and what makes it distinct from just say your, your plain Jane state tests or PSA ts or sats?

Speaker B: Yeah, so we, what we typically do at ah, equal opportunity schools is collect that traditional data you're talking about, you know, grades, uh, PSAT scores, etc. That coupled with robust surveys on students like who do they feel they feel connected to a trusted adult in school, what are their future aspirations, what do they like to do? That type of insight around, ah, a particular kid coupled with this other information tends to be really powerful in decisions and choices we're making for them. And so, um, EOS has had a great example of that with uh, advanced coursework. Um, there's been a, ah, study with Mathematica, which I published a story about this last summer in Fast company where uh, we were able to increase the average number of students placed in advanced coursework by 52 students per school. Uh, and this was a study around uh, about 400,000 students. And one of the things that's interesting about this is almost two full classrooms of kids who would have otherwise been overlooked. Now when it comes to the technological piece, and this is where AI can play a really important role, is when we talk about things like trusted adult or mindset, it tends to be this squishy, provocative thing in the environment. I think it becomes, you, uh, get this, um, debates about being woke, or is this being too left? You see the sentiment. But what I think is the next frontier of learning is if you use AI and predictive analytics on this type of information and see how it correlates and relates to outcomes you really care about, then these types of insights about kids could be really important proxies for us to be much more proactive, um, in how we engage them during the year to prevent things like chronic absenteeism, to support kids in advanced coursework. And so that's what we're doing currently in partnership with the Learning Collider. So the Learning Collider is based out of the University of Texas. Um, that entire initiative is focused on, uh, developing ethical AI tools. And not only that, making sure when those tools are created, we have rigorous analysis on, um, things like bias and things like efficacy. Because even when you look at some of the most preeminent AI companies, those companies still aren't doing a good job evaluating the efficacy of their own work. And so that's going to be really important part of our engagement with the learning collider moving forward. So most recently they had analysis on 800,000 students. We had survey data on 800,000 students. And right out of the gate of the first couple of weeks, they started to see these relationships between the predictive nature and whether a kid feels connected to a trusting adult in school, uh, and whether there will be chronically absent their grades. And so this is interesting because this could open the door in new ways. We're looking and tracking the effectiveness of schools like now more and M More. In today's age with AI, People are wondering, well, what do we teach, what do we measure and why? And so I think there's an opportunity to kind of broaden, uh, how we think about how we track the progress of schools as well.

Speaker A: Yeah. Now, as I mentioned earlier, there seems to be a general sentiment. I mean, maybe it's post Covid, maybe it's because of this, this phenomenon of AI and how it's affected everyday, uh, school work, um, that there's a skepticism. There are districts who are maybe a little burned out from some, some ed tech promises made during COVID made during remote learning, um, that underperformed over promised and underperformed and just, it even extends more to an environment of there's too much screen time happening. Um, there's other kind of general anti edtech, uh, sentiment. How do you position EOS in that climate?

Speaker B: Well, I think we've seen a lot of examples of really exciting technological solutions not meeting up to expectations. Um, and I think that even the most rigorously evaluated meta studies like at J. PAL North America, at mit, uh, I've shown little to no significant, um, academic differences. And you gotta remember the audience you're working with, you're talking about teachers and principals who are working like 8 in the morning to 8 at night. They just don't have time and capacity. So at the end of the day, you can't just share a dashboard or a widget and expect a big change. Because at the end of the day, you're talking about shifting mindsets and shifting human behavior. So what does this mean? As we think about technology in education, uh, I think education is always going to be human led, uh, but technology augmented. And so you have to be really thoughtful about not only building these tools, but how you're thoughtfully engaging, training and supporting educators, um, who already have a high cognitive load. And you have to keep that in mind so that whatever we provide them is actually useful in decisions you're making during the school year. Um, because you can't just overwhelm a teacher with lots of information and expect them to now analyze that. Right. And think about our good friend, uh, Carvalho, who recently resigned, uh, as a superintendent in la. Now there's important lessons from that experience. First of all, I don't think we should be in the business of discouraging superintendents to, to be innovative. I think to try to be bold is really, really important. What is really important about that example though is what is the process that you want to take to bring the community along and really being crystal clear about the core problem you're trying to solve and working backwards from there. So he kind of jumped in, I think, uh, right into the swimming pool, cannonball style and pan out. Um, but I think, uh, what does this mean moving forward? I think states should start thinking about what does innovation look like? What does that flywheel look like for us at the state when we see these shifts happening at the federal level with reductions in funding like, uh, EIR or ies, I know lots of wild acronyms in education, but these are long standing research entities. And if you're going to quote, unquote, kick it back to the states, well, what supports and guidance we're providing state leaders to do that things are really wonderful examples of great innovation happening. Take Maryland for example, with um, initiative for proven practices. So Governor Moore has a $40 million initiative in Maryland where he has public private partnerships to create this fund that invests in the most rigorously evaluated initiatives. And that initiative itself is also part of an evaluation. So if you take a page from that blueprint, what a state leader could essentially do is say, hey, let's provide this kind of framework in our state. But you know what? I think we should have a ladder system of funding. So not only reserving funding for the most rigorously evaluated, but there's a set of things that we're curious about that we want to learn more about. So creating a space for that, uh, a ladder system similar to what you see in EIR could be really helpful in creating this flywheel. And a, uh, state can ultimately decide, well, what are Our priorities in terms of learning at the state level. And that's essentially what they can use to guide how they approach that. And if you can tie in public private partnerships on that, that's just icing on the cake, which could be super valuable, especially at a time when budgets are getting tighter. This is a concrete example of the type of innovation flywheels that I think would be really important at the state level, which can support things like, um, what do we know about chronic absences? What do we know about AI use in education? So I think this AI question, um, will be solved if we all work together to create space for rigorous, thoughtful research moving forward.

Speaker A: Yeah, it's that whole linear graph right between, you have the data, you turn the data into information and the information becomes knowledge. Right. It was that human application that makes the information knowledge. Right? Yeah. Uh, you, you mentioned la, and that's in a perfect segue into my next question, which is about scale. And when we talk about, and another example, the complexity of education maybe versus other industries, um, you have a school district with 750,000 students in it.

Speaker B: Right?

Speaker A: Versus most likely are. Most likely. Our readers are and listeners right now are at school districts that are small or medium sized. Are there different ways, um, that they need to think about this than, say, than the big boys?

Speaker B: You know, this is such, so this is such an important question to ask and I wish I knew, uh, the answer because I think that'll solve a lot of challenges. But here's a couple of things to think about. So I think the first is even for the most rigorously evaluated initiatives or programs, um, these are the ones that we know exist with gold standard RCTs. The big question from a scientific and research perspective, the jargon that they use is like, is it generalizable? That is, can you take this at work in one context and apply in a different context? And what must be true, what hallmarks of success are really important? So being thoughtful about that is a whole area of work in of itself. So something that worked really well, like high impact tutoring in a place like Boston or Chicago, um, that might be a little more difficult to roll out in a concept. If you're a rural community in New Mexico and you have the same kind of access to that kind of town pipeline or colleges or universities to leverage. Right. So you have to try to think outside the Bakken, like how do you steer human capital to be tutored? So this isn't a great example of the generalizability question and challenge that people, um, have to Wrestle with now as it relates to your question about scale. This is my personal take, especially as a nonprofit leader. I think ultimately the end game for social impact organizations should be demonstrating what's possible, generating evidence that ultimately shifts policy and practice. And that's what I did at SAGA Education with the high impact tutoring movement. Everyone knows tutoring is a great way to help kids, but how do you do that in a cost effective and scalable way? So when I co founded saga, working closely with the University of Chicago Education Lab, every time we had something, we tried something new. Use a rigorous evaluation to understand the impact on student outcomes. And so we were part of 12 different randomized trials locally and internationally. And we can't just let that evidence sit there and collect dust. You have to be really intentional about that storytelling and dissemination of that information with, uh, folks at the U.S. department of Ed, um, other think tanks, et cetera. And then from there you can start that movement building. And eventually I work with Saga. When 2020 hit, a lot of folks like, what do we do to help kids right now? And high impact tutoring was refreshing the mind for lots of folks, became part of the set of things the US Department of Education integrated in their guidelines for Covid recovery. And from 2020 to where we are, uh, now, over $7 billion have been spent on high impact tutoring. And so that's a really wonderful example of evidence using evidence strategically and thoughtfully to create movements to ultimately shift policy, practice and what we our dollars. And so I think that's the kind of approach nonprofits will need to take for scale. Because ultimately an individual organization can't possibly serve all the students who the. The total addressable market. That just will never be possible.

Speaker A: Yeah. Now, I know you just got started, so I'm taking it easy on you, but I'll put you on the hot seat.

Speaker B: Yeah.

Speaker A: For say, three years from now or five years from now. What. What are your. What are your hopes? We'll put it that way. Where do you hope to see both individually, your work, which is kind of that overall topic that we've been talking about advance over the coming months. Give me your glass half full.

Speaker B: What I'm really excited about with Equal opportunity Schools, um, is addressing what is a really important challenge in K12 currently. And there's a number of them, but it's really thinking about chronic absenteeism. M. Uh, this is important for a lot of states and districts where if you see reduction in enrollment that impacts your bottom line, makes it much more difficult for you to support and Serve kids. And historically folks have approached that from a top down, um, strategy. But because of the heterogeneity, the variability, why kids aren't showing up, a top down approach is not going to work. And you have to think differently. What do you know about AJ and circumstances and starting from the bottom up. And I think that's when there's a situation of heterogeneity. What's the solution? Personalization. And so I think if we focus on individual students, then I think we can increase the odds of improving average daily attendance. And so over the next three years, I want to make a giant leap forward on our understanding of ways in which we can support in improving average daily attendance, uh, for students. And really starting off with having, um, a series of quasi experiments and studies on what are types of research and insights about kids that are more predictive in determining proactively whether they're on track to staying in school or dropping out. So that's what I'm really excited about. And the scientific literature on chronic absenteeism, um, it's still not as strong as it could be. There aren't a lot of rigorous randomized trials on this issue. Is a model among the reasons why it's really tricky and challenging for, you know, state commissioners and superintendents. There's a few study out there that are really interesting, like text messaging families, reminding them of the importance of attending school, etc. But yeah, this is, this is, uh, an area where we just need much more insight.

Speaker A: Yeah, well, I guess that's the first step. You have to show up, right?

Speaker B: That's right.

Speaker A: Well, A.J. thank you so much again for your time and your insights. A lot, uh, to chew through here, but congratulations, uh, on your work. Congratulations on your new position and uh, we'll be, we'll be following along hopefully with, uh, with your successes.

Speaker B: Thank you so much and thanks for having me.

Speaker A: So thanks to AJ for that, uh, great conversation. If today's conversation got you thinking about which students in your district might be ready for more and who's responsible for finding them, well, that's exactly the point. You can learn more about EOS and their work with advanced coursework@eoschools.org and if you're a district leader interested in the kind of survey based AI assisted approach AJ described, they're a good place to start. For more conversations like this one, subscribe to the Tech Learning podcast wherever you listen. And visit techlearning.com for news, resources and practical guidance for schools and district leaders. We'll see you next week.