Case Study #4: Data Quality Beyond Null Checks — Building a Proactive Monitor for Production Pipelines
How a team stopped discovering data problems from business complaints and started catching them before anyone woke up.
Read more →This is my personal space where I share my work experience, lessons from real projects, and reflections on data engineering. If you want to get to know me and how I work, you're in the right place.
I'm Allan, a Senior Data Engineer with 6+ years of experience designing and building data platforms that companies can rely on for real business decisions. My work goes beyond pipelines — I define architecture, set governance standards, lead technical teams, and take ownership of platforms end to end.
Over the years I've worked across Microsoft Fabric, Databricks and Azure on financial data, sales analytics and operational integrations in complex, high-volume environments. I've had the opportunity to work at companies like Stanley 1913, Microsoft and Ambev, leading greenfield implementations and business-critical data projects at a regional and global scale.
I work best with companies that treat data as a strategic asset — not just an operational necessity.
How a team stopped discovering data problems from business complaints and started catching them before anyone woke up.
Read more →How a single configurable framework replaced what would have been dozens of individual pipelines.
Read more →How reading the Spark UI before touching any code led to a 57% reduction in pipeline runtime.
Read more →