Episode 66: From Paper to Digital Without Downtime: Lessons from a Global Brewery

Welcome to Digitalization Tech Talks, the podcast focused on addressing common issues in the process industries and how they can be, and actually in many cases are already being addressed with the use of new and emerging technologies. Our goal is to provide you with ideas that may be helpful to you as part of your own digital transformation. Today we're bringing two exciting firsts to the series. Number one is that we're going to talk about beer.

More on that soon. And the second is that this is Cole Lewis' first time in the co-host seat. Sounds like the perfect combination for a toast. This episode's topic pertains to the transition from paper to digital at a global brewery, which is a very desirable move across the process industries.

Before we bring on our guest, I'd like to welcome Cole to the show. Hello, Cole. It's great to have you on board. What are your thoughts on beer and today's topic?

Thanks, Don. Yeah, no, it's great to be here and really not a bad way to start as a co-host. Getting to talk about digitalization and beer. These are two exciting topics on their own, so I think it should be a really fun conversation today.

But yeah, on the surface, moving from paper to digital, I feel like it might sound simple, but it's actually a pretty big shift. In April, I was at the Craft Brewers Conference and talked to a lot of breweries who were trying to work through this exact same challenge. So it's not really as easy as it seems for brewers or really anyone in the process industries. But yeah, I'm looking forward to hearing how our guest approaches this, what our listeners can take away from it.

Well, thanks, Colin. Welcome again to the show. Great to have you on board. And this is a challenge for sure, what our guest has accomplished, and we'll learn more about that.

And I think we're going to have a lot of questions for him as well. So why don't we go ahead and bring him on? Fred Hussman is a senior program manager at Sky.io.

Fred has more than 35 years of experience working on automation projects in the process industries, and we're going to talk about one of those today. Hello, Fred. Welcome to Digitalization Tech Talks. Can you tell us about your background in manufacturing digitalization and briefly introduce the project we're going to discuss today?

Yes, Don. Thanks for having me. Like you said, I have been working in the automation of process plants in the OT space for 35 years now, but recently have begun supporting the ITOT integration necessary to go digital. Now, to go digital, companies in the process industries need a unified namespace that contains all the data of their entire enterprise.

They call it the one source of truth. So the first step is harvesting manufacturing data from the automation system. In the end, you want a database that contains all your historical and real-time information so that you can unleash AI on it. Most of our customers in the processing industries are at step one.

Sky.io is working with Constellation Brands right now to harvest tens of thousands of tags from their batch DCS system where the beer is made as part of a brewing MES project. And for those that don't know Constellation Brands, they operate some of the largest breweries in the world and their flagship beer brands are Modelo and Corona. Thanks, Fred.

That's great to hear about your experience at Constellation Brands and the complex project that you're working on there. What first drew you to that challenge of digitizing operations? And how do you approach customers who might worry about disruptions to production? Many of our customers are concerned about harvesting data from their operations because they are worried about shutting down their production.

They're also hesitant to start the digitalization effort because they think that new points of failure will be introduced. And I would say they're right to be concerned about that. So we spent a significant time demonstrating to Constellation Brands that we could implement the MES system at the breweries without shutting down the plant and without putting beer making at risk because in the end, it's about the beer. Well, cheers to that, Fred.

And hopefully we can lower the concern level for someone in our audience as well that may have those same concerns. And I have a couple of questions I'd like to pose to you. At a high level, what problem was the brewery trying to solve and why was no disruption to live processes a hard requirement from day one And the second question is and what were the limitations of the existing paper approach that ultimately forced change The Consolation Brands they have a corporate initiative to go digital across their entire enterprise.

They have dozens of digitalization projects that are executing in parallel. The Brewing MES project is just one of those. It's the first and arguably the most important. The project delivers full order tracking from their ERP software, SAP.

The project also delivers complete batch traceability from the grain silos through the brew house, fermentation, and filtration until finished beer is sent to packaging. And then dozens of production reports are now electronic. As far as the limitations of the paper-based approach, paper is not a very good way to do things. There was incomplete records, inaccurate records or reports, delayed reports, even missing reports.

And many of their operators, they spent their entire shift just filling out paper forms or manually extracting data from the automation system just to type it into an Excel spreadsheet. That's what they did all day. It's mind-numbing work. And now they can focus instead on making better beer.

Those limitations make sense. And I think we all want to get out of those Excel spreadsheets and start doing more value-added work and things that we were meant to do in our jobs. So maybe follow up to Don's previous question, what were some of the biggest perceived risks, whether that was product quality, uptime, or operator trust? And how did the team address those concerns?

Also related to those risks, how did you ensure stakeholders were confident that production control would not be compromised? Well, I guess the biggest perceived risk, and to be honest, the biggest real risk, was now needing not only the automation system to be working correctly to make the beer, but now you needed the MES system to also be working correctly, because we were introducing another point of failure. We implemented an OPC UA interface between the automation system and the MES system, but now that's a critical interface.

It has to work. So we had to come up with a way to still make beer if that interface stopped working for a time. We implemented data buffering directly in the new MES data harvesting function blocks in the DCS so that we could tolerate that MES to DCS interface being down. For some period of time, we just set an hour.

For brewing, that's okay. It's a slow process. We tested it. It works.

We don't lose any data and we can still make beer. Regarding the stakeholders, after we started the project work, the constellation brands, we call them CBI, their top management was concerned that we could pull this all off. So about halfway through the implementation, we had an executive update meeting at the CBI Beer Division Headquarters in San Antonio, Texas. After that meeting, we gained their confidence.

And now their question is, how fast can they go digital at all the breweries? Yeah, that's a pretty good point you make there about getting the executives on board. That's a topic that's come up from time to time in our other episodes is making sure you have that executive support. So like you said earlier, there's a lot of extra things you were adding to it that might add other levels of risk and concern.

And I'm sure that raised their level of concern as well, but it sounds like you guys did a great job in getting their confidence. So that's great. So Fred, let's talk about separating data collection from core production logic. What does that mean in practical terms on the plant floor?

And why is that separation so critical when you're digitizing live operations at scale? So the CBI breweries, they are large and flexible. Breweries are just that way. But flexibility means complicated at the automation layer.

There are sometimes hundreds of source and destination combinations for one transfer. This means there is a lot of complicated automation logic down in the DCS. And then you have batch management in the mix. So the DCS code was already hard enough to decipher before adding a bunch of data harvesting blocks.

The way we wind up modifying the DCS code, it is obvious. Which code is there to make beer and which code is there for MES data harvesting purposes? That separation is obvious in the controllers and in the HMI application. Yeah hearing how complex that DCS logic is I imagine you can just go and make these changes in trial or production and play around with that I know your approach included some full virtual simulation of your entire brewery So how did simulation play in and what did simulation allow you to test or validate before touching the real systems Did it change the speed or safety of the decision compared to maybe a more traditional rollout approach?

For this kind of project, there was no way to do it without a lot of testing. We're harvesting tens of thousands of tags from the DCS as beer is made. So we need to collect data to track every single automatic raw material and product transfer and every manual edition. There's a lot of those.

We also need to collect thousands of data elements to populate all of the dozens of production reports. The only way to test all this data collection is to run recipes all the way through the batch system. And that's not easy. So to do that, you need a simulation system or a digital twin to do it.

And at first, we plan to use the hardware-based simulation architecture, but for this large CBI brewery project, the cost of the hardware-based solution was approaching $1 million, and it just wasn't going to happen. Then we investigated a software-based virtual controller simulation solution, and in the end, it worked for our purposes with the batch manager and the DCS. And the cost of that virtual controller solution was about 10% of the hardware-based solution, so for about $100,000.

We used that virtual controller digital twin for iterative testing for an entire year. We could not have properly tested that system without it. So the team would have been terrified to go to go live without that testing under their belt. Well, it's great to hear how much you were able to leverage the simulation technology and particularly that virtual controller and how that worked so well and saved so much cost.

That's great. Let's talk in terms of how this project can be leveraged beyond brewing. Which lessons from this brewery example do you think would translate well to other manufacturing environments or industries? Yeah, I think that our solution works for all industries, at least all the processing industries.

The automation layer could be different, the MES layer could be different, but I think our solution still works. I think the key is to use a standard and supported interface like OPC UA. Also, design a way to tolerate that interface being down for a time, and build a digital twin to test all the data collection. If you do those three things, you can be successful.

And I guess one more important part of the solution is finding a way that the automation team can work independently from the MES team. Like in our case, a different vendor worked on the DCS beer making code, while the SkyIO team worked separately on the DCS MES code. So we worked apart for many months and tested on separate digital twins. That's an important point.

So before GoLive, we merged our tested projects together using the DCS textual interconnection feature. That was a big deal, and it enabled us to go live much faster. We didn't have to wait for the automation code to be done before we could start making progress on the MES, and we had a good way to bring them together just in time for GoLive. Thanks, Fred.

Yeah, those are great lessons. And that's cool to hear about how both teams are able to work separately, but then come together at the end. and put everything into production, go live much faster. What's one of the mindset shifts that organizations need to make before technology can really help them at scale?

So I think AI is a big topic now. And most organizations do realize that they must leverage AI to stay competitive, but they won't be able to leverage AI in their operations unless they have a unified namespace for AI to evaluate. And companies are way behind on harvesting contextualized data that they need from the OT layer. That's what this project was about.

And from my experience, AI is not going to help much with that contextualization part. Companies need to partner with a vendor that is good at IT-OT integration and can get to the OT data and contextualize it at the point of capture, not later. Yeah, that's definitely an important point, Fred. And I know you kicked off at the beginning of our conversation talking a little bit about AI.

Not surprised to hear it come in here at the end. It has become a really key topic. So interesting to hear that. That's definitely a good point.

As we wrap up our conversation we always like to include some key takeaways We talked about a lot of different topics in our brief time together here If you could pull out what you think are the key takeaways that you can leave with our audience what would they be Yeah, I think I have four. First, automated data capture. Right now, for most companies, especially in the processing industries, operators are collecting all the data, and it's a huge manual process, and it's really all they do all day long.

They fill out forms. They actually pull data manually from the HMI and the control system and then type it into a spreadsheet. All that's done automated now. All that data capture and all those report populations are all done automated and electronically.

And that's really the foundation. It makes everything else in the digital transformation stack work. So that's step one. And you can do it, as we've shown, without impacting production and while the plan is running.

So this is the first step. And companies, if they want to leverage AI, they got to get started on this step. Second, I would say, is, you know, there's a little bit of fear about going digital, especially by certain groups like the operations level. A lot of the operators think it's going to eliminate their job.

But in reality, I think digitalization empowers the operator. What it does eliminate is the operator's time to manually log the production counts and the downtime events and the quality checks and the material consumption. and instead they can focus on the higher value human-centric tasks such as critical decision making, exception handling, and process optimization. Basically, operators can now focus on making better products, or beer in this case.

A third takeaway, everyone's talking about AI and they right away want to say, hey, how can we leverage AI? But they can't really do it until they have the data lake. You can't just start using AI. You basically ask AI a question, well, it can't give you an answer unless it has this data lake of one source of truth to evaluate and then get back to you.

So step one is you need to start working on the one source of truth right now. Once you have that data lake created, then AI can unlock immediate value for the operators through predictive maintenance, machine learning, advanced process control, smart alarm management, all that stuff. And we really think where it's going is that a genetic AI will evolve to the point pretty quickly where it taps the operator on the shoulder and says that the current batch, the batch running right now, is about to get away from you.

And here is what you need to do about it. A final takeaway. The key, I think, for the companies that are starting down this journey, especially on the MES implementation, is to partner with a vendor that is good at orchestrating. They call it orchestration between IT and OT, but that's the key.

So you have to find a partner that can bridge that OT-IT convergence gap that's happening. So you need a partner that understands both the plant floor and the IT architecture. And we have had many customers. They start down this path and they partner with an IT, I would say, centric integrator.

And then they come back to us later because they learned that that IT-only vendor didn't, what they say, have the OT chops. Basically, they struggled on that OT data harvesting contextualization part, and that's step one. Then that's what our customers say, but Sky.io can do both sides.

Fred, I think those were four great takeaways and can only come from 35 years of experience. So I really appreciate you spending some time with us today and sharing those with our audience. So thanks for joining us today, Fred. Thanks for having me.

I was happy to participate and I had fun. And that does bring us to the end of this episode. If you have questions about any of the topics we discussed today, please feel free to connect with Fred directly. His contact information is included in the show notes.

And Cole and I are always open to your comments, your thoughts, and feedback on the show or the topics. You can also find our contact information in the show notes. We'd love to hear from you. If you liked what you heard today, please help to spread the word by rating the show.

And if you're not already a subscriber and would like to be notified when new episodes are released, please subscribe to the series. Thank you for listening to this episode of Digitalization Tech Books.