Lead II - Data Engineering_AWS Databricks

About the role

We are looking for strong Lead Databricks Developer with streaming expertise.

What you'll do

• Design, develop, and lead enterprise-scale data platforms and pipelines using Databricks, PySpark, Delta Lake, and Spark SQL
• Build and optimize scalable batch and real-time data ingestion solutions using Kafka, Event Hubs, Auto Loader, Structured Streaming, and Delta Live Tables (DLT)
• Develop and manage Lakehouse architectures leveraging Medallion (Bronze, Silver, Gold) design patterns and data engineering best practices
• Implement and govern enterprise data assets using Unity Catalog, including data lineage, metadata management, security policies, and access controls
• Collaborate with architects and cloud teams to build and deploy data solutions on AWS and Azure, leveraging services such as S3, ADLS, Event Hubs, IAM

What you need

• 7–9 years in data science & ai
• AWS
• Databricks
• Spark
• LLMs
• MLOps
• Kafka
• Any graduate

Good to have

• Terraform
• IAM
• CI/CD
• Data privacy
• Azure