Lead Consultant(Product/Domain)

About the role

We are seeking a Senior Databricks Data Engineer with 6+ years of data engineering experience to design and scale our next-generation data platform. In this role, you will be the technical lead for our Databricks on AWS ecosystem. You will architect robust, cost-effective pipelines that ingest, process, and serve massive datasets to power our analytical and AI initiatives.

What you'll do

• Databricks & AWS Data Pipeline Engineering
• Architect end-to-end data products using the Medallion Architecture (Bronze, Silver, Gold layers) natively built on Amazon S3 and Delta Lake
• Design and scale production pipelines using PySpark, SQL, and Delta Live Tables (DLT) for both high-throughput batch and real-time streaming data
• Ingest diverse data streams by integrating Databricks with AWS messaging systems like Amazon Kinesis or Managed Streaming for Apache Kafka (MSK)
• Optimize Lakehouse storage layouts by leveraging Delta features such as Liquid Clustering, Z-Ordering, and data compaction

What you need

• 6–11 yrs of experience
• SQL
• Git
• Python
• AWS
• Unit testing
• Kafka
• Any graduate

Good to have

• ETL
• CI/CD
• IAM
• Databricks
• Spark
• Unity