The Data Flowcast · 2026-07-30 · 31 min
Migrating two decades of legacy job orchestration is the kind of project most teams quietly avoid. In this episode, Marc Lamberti is joined by [Sanket Patel]( linkedin.com/in/sanket-patel-a2a57b3a ), Director of Engineering at Trading Technologies, to talk through how his team is consolidating 20 years of C#/.NET-based scheduling and a recently acquired company's stack onto Airflow, Astronomer, and Snowflake. The conversation covers orchestrator selection, observability, the operational economics of managed Airflow, the human side of migrations, and where AI fits into the new data platform. Key Takeaways: 00:00 Introduction. 02:11 Sanket's background and role leading data platform initiatives at Trading Technologies. 04:32 The data landscape at TT: silos, OLTP-driven reporting, and an acquired company running orchestration on a 17-18 year old ASP.NET stack. 10:53 Choosing Airflow and Snowflake as the foundation of the new data platform. 11:36 Why Airflow won over legacy schedulers like Autosys plus Informatica, especially past the 2,000-3,000 job mark. 16:34 Collapsing thousands of legacy jobs into 15-20 Airflow DAGs.
Using Airflow for diverse client projects at Accion Labs79 / 100
What's New in Apache Airflow® 3.377 / 100
Running Airflow 3 in a regulated environment at OTPP57 / 100
Managing a Customer Analytics Platform with Airflow at Skimlinks66 / 100
Building a custom Tableau provider for Airflow at JLR69 / 100