Industrial DataOps #9 with Databricks - David Rogers on scaling AI
Welcome to Episode 9 in our Special DataOps series. We’re getting closer to Hannover Messe, and thus also the end of this series. We still have some great episodes ahead of us, with AVEVA, HiveMQ and Celebal Technologies joining us in the days to come (and don’t worry, this is not the end of our podcasts, many other great stories are already recorded and will be aired in April!)In this episode, we’re joined by David Rogers, Senior Solutions Architect at Databricks, to explore how AI, data governance, and cloud-scale analytics are reshaping manufacturing.David has spent years at the intersection of manufacturing, AI, and enterprise data strategy, working at companies like Boeing and SightMachine before joining Databricks. Now, he’s leading the charge in helping manufacturers unlock value from their data—not just by dumping it into the cloud, but by structuring, governing, and applying AI effectively.Databricks is one of the biggest names in the data and AI space, known for lakehouse architecture, AI workloads, and large-scale data processing. But how does that apply to the shop floor, supply chain, and industrial operations?That’s exactly what we’re unpacking today.Join Our Community Today! Subscribe for free to receive all new postWhat is Databricks and How Does It Fit into Manufacturing?Databricks is a cloud-native data platform that runs on AWS, Azure, and Google Cloud, providing an integrated set of tools for ETL, AI, and analytics.David breaks it down:"We provide a platform for any data and AI workload—whether it’s real-time streaming, predictive maintenance, or large-scale AI models."In the manufacturing context, this means:* Bringing factory data into the cloud to enable AI-driven decision-making.* Unifying different data types—SCADA, MES, ERP, and even video data—to create a complete operational view.* Applying AI models to optimize production, reduce downtime, and improve quality."Manufacturers deal with physical assets, which means their data comes from machines, sensors, and real-world processes. The challenge is structuring and governing that data so it’s usable at scale."Why Data Governance Matters More Than EverGovernance is becoming a critical challenge in AI-driven manufacturing.David explains why:"AI is only as good as the data feeding it. If you don’t have structured, high-quality data, your AI models won’t deliver real value."Some key challenges manufacturers face:* Data silos → OT data (SCADA, historians) and IT data (ERP, MES) often remain disconnected.* Lack of lineage → Companies struggle to track how data is transformed, making AI deployments unreliable.* Access control issues → Manufacturers work with multiple vendors, suppliers, and partners, making data security and sharing complex.Databricks addresses this through Unity Catalog, an open-source data governance framework that helps manufacturers:* Control access → Manage who can see what data across the organization.* Track data lineage → Ensure transparency in how data is processed and used.* Enforce compliance → Automate data retention policies and regional data sovereignty rules."Data governance isn’t just about security—it’s about making sure the right people have access to the right data at the right time."A Real-World Use Case: AI-Driven Quality Control in AutomotiveOne of the best examples of how Databricks is applied in manufacturing is in the automotive industry, where manufacturers are using AI and multimodal data to improve yield of battery packs for EV’s.The Challenge:* Traditional quality control relies heavily on human inspection, which is time-consuming and inconsistent.* Sensor data alone isn’t enough—video, images, and even operator notes play a role in defect detection.* AI models need massive, well-governed datasets to detect patterns and predict failures.The Solution:* The company ingested data from SCADA, MES, and video inspection cameras into Databricks.* Using machine learning, they automatically detected defects in real time.* AI models were trained on historical quality failures, allowing the system to predict when a defect might occur.* All of this was done at cloud scale, using governed data pipelines to ensure traceability."Manufacturers need AI that works across multiple data types—time-series, video, sensor logs, and operator notes. That’s the future of AI in manufacturing."Scaling AI in Manufacturing: What Works?A big challenge for manufacturers is moving beyond proof-of-concepts and actually scaling AI deployments.David highlights some key lessons from successful projects:* Start with the right use case → AI should be solving a high-value problem, not just running as an experiment.* Ensure data quality from the beginning → Poor data leads to poor AI models. Structure and govern your data first.* Make AI models explainable → Black-box AI models won’t gain operator trust. Make sure users can understand how predictions are made.* Balance cloud and edge → Some AI workloads belong in the cloud, while others need to run at the edge for real-time decision-making."It’s not about collecting ALL the data—it’s about collecting the RIGHT data and applying AI where it actually makes a difference."Unified Namespace (UNS) and Industrial DataOpsDavid also touches on the role of Unified Namespace (UNS) in structuring manufacturing data."If you don’t have UNS, your data will be an unstructured mess. You need context around what product was running, on what line, in what factory."In Databricks, governance and UNS go hand in hand:* UNS provides real-time context at the factory level.* Databricks ensures governance and scalability at the enterprise level."You can’t build scalable AI without structured, contextualized data. That’s why UNS and governance matter."Final Thoughts: Where is Industrial AI Heading?* More real-time AI at the edge → AI models will increasingly run on local devices, reducing cloud dependencies.* Multimodal AI will become standard → Combining sensor data, images, and operator inputs will drive more accurate predictions.* AI-powered data governance → Automating data lineage, compliance, and access control will be a major focus.* AI copilots for manufacturing teams → Expect more AI-driven assistants that help operators troubleshoot issues in real time."AI isn’t just about automating decisions—it’s about giving human operators better insights and recommendations."Final ThoughtsAI in manufacturing is moving beyond hype and into real-world deployments—but the key to success is structured data, proper governance, and scalable architectures.Databricks is tackling these challenges by bringing AI and data governance together in a platform designed to handle industrial-scale workloads.If you’re interested in learning more, check out www.databricks.com.Stay Tuned for More!Subscribe to our podcast and blog to stay updated on the latest trends in Industrial Data, AI, and IT/OT convergence.🚀 See you in the next episode!Youtube: https://www.youtube.com/@TheITOTInsider Apple Podcasts: Spotify Podcasts: Disclaimer: The views and opinions expressed in this interview are those of the interviewee and do not necessarily reflect the official policy or position of The IT/OT Insider. This content is provided for informational purposes only and should not be seen as an endorsement by The IT/OT Insider of any products, services, or strategies discussed. We encourage our readers and listeners to consider the information presented and make their own informed decisions. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit itotinsider.substack.com
South Africa