Capstone Spotlight: SKYGUARD and AI for Agriculture

Speaker A: Foreign. Welcome to Technical Leadership Talks, the podcast that brings you advice, insights and knowledge from industry leaders and experts on how to navigate managing in a technical field. Technical Leadership Talks is brought to you by Texas A University's Master of Engineering Technical Management, an online graduate program that trains industry professionals for leadership positions. I'm Katie, METM's digital marketer and your host for this episode. In this episode, we sit down with alumnus John Dawdry to explore his METM Capstone project. Inspired by a real world challenge facing small scale poultry owners, John created Skyguard, an open source AI power raptor alert system designed to help protect flocks. John shares how he transformed an idea into a working technology to start. Could you share who you are, your role, and the company you work for?

Speaker B: Hi Katie. I'm John Daughtridge. I'm the Director of Global Launch and Business Solutions at General Motors. My teams and I are responsible for managing the IT portfolio for manufacturing products as well as implementing change in our over 100 manufacturing plants around the world to ensure that we can build the next greatest vehicles and mobility products for our customers.

Speaker A: Today we're going to talk about your Master of Engineering Technical Management Capstone project. So can you give me a little bit of an overview on your capstone that you completed?

Speaker B: Absolutely. The capstone was really the culmination of two years of work at the METM program. Um, we had studied pretty in depth AI and I was excited as it was a upcoming technology to really leverage it. And so I built a system which monitors the skies above small farms, homeowners and others, and looks for birds of prey. It keeps an eye out for raptors, vultures, really anything that's going to come down and cause challenge to either small livestock or just pets. And so the system monitors the sky and alerts homeowners, um, or farm owners when there's something dangerous above a.

Speaker A: That's so unique. How did you come up with that and why was it important to you? Did you grow up on a farm? Like, I was so curious when I heard what your capstone project was about.

Speaker B: I did not. Uh, in fact, I was looking for something that would leverage technology like I said. And I think we all remember the global pandemic maybe six years ago. Well, there was a whole lot of, um, new sourdough makers. There was a whole lot of, um, you know, oh, I really got into inline skating or, you know, remember some other thing that was cool for my youth and dove into it. Yeah, for some reason I ended up deciding that, uh, chickens would be a lot of fun. Backyard chicken farming became, uh, what my family did. And we had a few chickens, started with a couple, and very quickly found out living in Austin, Texas, that chicken hawks are a thing. And they were after those birds. And you know, when you're at home all day, there's, there's trauma when you hear the chickens squawking in the backyard and hawks flying over. And so I, I thought, gosh, this is a good chance to do something interesting. I built a small prototype at the time and I realized that I needed a lot more investment in time and code to make it what I wanted it to be in the future. And I got that opportunity and the skills in the MBTM program.

Speaker A: How does the AI work in simple terms?

Speaker B: In simple terms, I use what's known as a segmentation model first. So built on the Ultralytics YOLO framework, I established a segmenter which first just looks and says, is there a bird in the whole sky? And I'm using really just commercial off the shelf hardware like consumer grade webcam, consumer grade Raspberry PI is the back end and it watches that sky. It cuts out the little bird or what it thinks might be a bird, and then it sends it to a custom classifier that I built that classifies the birds into one of over 570 different species.

Speaker A: What technologies or tools did you use to develop it?

Speaker B: I was lucky. That AI was just coming up on the crest in, uh, development capacity. So tools like Cursor, tools like Claude were just getting developed and I was able to learn and leverage those as I was developing. I built most of it in Cursor, but was able to bounce back and forth between OpenAI's Codex as well as Claude to conduct kind of audits of what Cursor suggested as I prompted it and then wrote code to develop for it. So I really, the timing of this program for me was perfect. It was transitioning, um, an AI model based development from my historical background, which has been just writing code myself. And it was a complete multiplier in the amount of work I could get done in a particular weekend because this was not a full time gig for me, for the masters. And that helped a lot. But it also, um, opened my eyes to the capabilities that are out there with these new models and new development capabilities. The Ultralytics platform was really the backbone. Um, I use transfer learning to put my images and images from a model called Innate Birds into the model and classify them. The technology is really an industry standard workflow for convolutional neural networks. It's a simple segmentation model, as I mentioned, that goes to a classifier. But that doesn't mean that this was easy. In fact, this is a problem which has been, and you look in the open source communities, a very difficult one to solve. There are bird models that identify birds by audio, but doing so in birds in flight is a particular challenge.

Speaker A: Can you explain to me exactly what happens if there, if something like a raptor is, is detected? Does a sound go off? Like, how does it, how does that work?

Speaker B: There's a few different alerting mechanisms. I've uh, enabled sound, but more importantly Discord. I hooked into Discord with some webhooks and can publish to Discord channels. And that gives you the ability to message multiple people, uh, in a household in case not everybody's home at that time. And that that was the easiest, lowest friction way for me to provide alerting. I did put some hooks in there so people can develop their own. This is an open source program. I've got it published on GitHub. Um, and people are able to download and suggest improvements to the code. So if they issue a pull request, I can take a look at it and add improvements. That is somewhat limited because the first thing people want is automated deterrence. In fact, as people hear about this and I've gotten messages over the Internet, hey, can we add a sound cannon? Can we add something that would drive the predator away? The challenge with that is bald eagles and other birds are protected by, uh, the U.S. and we have to be careful of that and not run afoul with those laws.

Speaker A: Right, Right. So was there any surprising discoveries that you had during your development?

Speaker B: Truly, Katie, I thought this was going to be easier than it was. I dove into it knowing I had the NA Birds library from Cornell, who does a really good job and is really the backbone behind much of the audio libraries. And from their ebirds website, they've got a ton. And I'm a wildlife photographer. I shoot birds for fun, um, with cameras, not guns. And in shooting those images, you're looking for this portrait, glamour shots, looking image. You get the bird's eye in perfect focus. Maybe the bird's in flight, maybe it's on a branch, Whatever the challenge is, when you're using a webcam and you're looking at the sky for a predator, they don't stop for that glamour shot. Like they don't get in the lighting booth and get all perfect. Instead you end up with, uh, often backlit image, the sun being behind the bird. You can only see it in a real quick video, uh, shot that you've got. And as I break that video down into individual frames, I'm trying to identify by the shape of the bird alone what type of bird is it. So when you're using the commercial libraries that are available or the open source ones, all of them are photographers like me who are trying to get that perfect shot and training a model. You need the shot of what is actually going to exist in the world.

Speaker A: Hm.

Speaker B: That you're trying to test. And so what I've had to do is go shoot a whole bunch of what would I really be pretty bad photos if you're trying to show people pictures of birds, but are incredibly useful when you're trying to target and understand. But that made me build a brand new workflow into my photography, which is I'm going to shoot some birds, I can't even tell what they are and then I'm going to uh, figure out what it was, label it, put it into my label training model, retrain. And so I ran I don't even know how many iterations of training this AI model so that I'd be able to identify birds when they weren't at their best, when they weren't at their best, lip when they weren't at their best angle and trying to figure out what they were.

Speaker A: But it's neat that you had the photography skill that you then had to deconstruct and reconstruct for this specifically.

Speaker B: It was fun to be able to leverage that and have a good reason to wake up and get out in the early morning to shoot birds.

Speaker A: Yeah, you'll see some birds. Yeah. Would you say that was your biggest challenge?

Speaker B: Definitely. The biggest challenge was getting enough images that were labeled that identified birds for the model to use to train. I had other challenges as you work through these systems. Uh, for instance, I originally wanted to put deterrence into it as again, as a small farm chicken owner, I wanted to deter birds from coming in and attacking those chickens. But unfortunately, um, like I found out I couldn't. So there was a lot of learning that had to happen in there as well. I also had technical challenges to overcome just in my knowledge and learning. I used uh, these type of networks in the past, but I was lucky to take our AI class as part of the engineering school. Um, that did a whole lot for helping me tune the models once I developed them. Um, it's not just a matter of, oh, good, throw all this data at a YOLO model, tell it to build a pipeline and you're Done. There's a whole lot of tuning that has to happen. And I was lucky to have that background from the program.

Speaker A: What kind of impact could this technology have on farmers or other agricultural operations?

Speaker B: I was really shocked. Uh, I found out through this program, doing some research. There are 11 million small farms and home chicken owners in the US and the impact that this could have would be enormous for those 11 million farms to get some peace of mind that their, their poultry can roam free in their yards, but will also be protected. Uh, in a lot of cases, chickens lived cooped up their entire day and it's because of these predators. And so this gives those farm owners an opportunity to let their chickens roam free and really show what is some amazing personalities in these chickens. They're, they're pretty cool pets.

Speaker A: Yeah, they get to live a, uh, nicer life. So have you continued developing the project since you graduated and wrapped it up?

Speaker B: I actually did. Uh, I had a huge security release this weekend. I got working on it. You may have heard of some of the more recent AI models. Uh, Anthropic famously released Mythos and then now they're fable model. But these are getting extremely good at finding vulnerabilities, security vulnerabilities in software. And so anyone who has open source code or any proprietary code that they're working on needs to stay on top of it more than ever. And so this weekend I made sure to put a push in and developed, uh, quite a bit of new code to protect this system from external sources. Yeah, absolutely. Staying on top of it.

Speaker A: And you talked a little bit about how other people were asking you about it. So have you opened it up to other people to use at this point?

Speaker B: It is available for people to clone into their own world and write code. And should they find something that's useful that they think should go into the core product, they can push it back to. And so I've allowed them to do that. I haven't gotten any submissions yet, though. Many people have cloned the repository, meaning that they found use for it somewhere. And I'm excited about that. I really wanted to develop this to help other people and had the opportunity through the program, so I did.

Speaker A: What was the biggest takeaway you gained from your capstone experience?

Speaker B: As much as we as developers and engineers want to solve a problem ourselves, the community is even better. And so I was lucky to have a great sponsor in the program, Dr. Rustam Modi, who was my faculty advisor. I also had, um, other members of the faculty. Ahmed MacMood M was a professor who I probably burned his ear off, um, over different technical strategies to try and solve some of the challenges that we ran into. And I was really grateful for those connections to be able to develop the product in a way that was better than I would have been able to do it myself.

Speaker A: Mhm.

Speaker B: And that was really a neat thing to do when as a developer I really just want to sit down and write code. And the planning, the socializing, the thinking ahead of time has really saved a lot of the work that I would have had to go through.

Speaker A: Thank you for listening in to today's episode. Tech Leadership Talks is brought to you by Texas A&M University's Master of Engineering Technical Management. To learn more, Visit MetM. TMU. EDU.