Lead Data Engineer

About the role

This is a senior individual-contributor and technical leadership role that drives design standards and guides a team's engineering practices. It will appeal to you if you combine deep technical mastery with the ability to lead the technical direction of a team and elevate its overall engineering quality.

What you'll do

• Set and enforce design standards, patterns, and best practices for pipelines built on Databricks, Spark, Python, PySpark, and dbt
• Provide technical leadership and direction to a team of engineers, owning architecture and design reviews
• Lead the design of CI/CD (GitLab, Jenkins), DataOps, and data observability strategy for the team
• Own Databricks platform administration standards and deployment governance using Databricks Asset Bundles
• Drive resolution of complex technical challenges across pipelines and the platform

What you need

• 8–14 years in data science & ai
• Python
• Databricks
• Spark
• dbt
• CI/CD
• Git
• M.Sc or B.Tech or B.E. or B.Sc or BCA or M.Tech or MCA

Good to have

• Jenkins