Lead Data Engineer

About the role

We are seeking an experienced Senior Data Engineer to drive the performance, governance, and AI-native maturity of our enterprise Data Platform in Databricks. This is a Databricks-focused Data Engineering role with a working understanding of DevOps practices — designing scalable pipelines, tuning workloads for performance and cost, and operationalizing modern data and AI capabilities on Lakehouse.

What you'll do

• Design and develop scalable data pipelines and Lakehouse solutions on Databricks
• Tune Databricks workloads for performance and cost, including cluster sizing, query optimization, and Delta Lake table design
• Establish and enforce best practices for partitioning, clustering, and workload isolation
• Track performance trends, identify high-cost queries, and partner with source teams and end users to resolve long-running loads
• Design and operationalize Unity Catalog for data governance — access control, lineage, and security

What you need

• 6–11 yrs of experience
• SQL
• Python
• Databricks
• Spark
• CI/CD
• Unity
• M.Sc or B.Tech or B.E. or B.Sc or BCA or M.Tech or MCA

Good to have

• Terraform
• ETL