DataBricks Data Engineering

About the role

Experience building ETL/ELT pipelines, implementing incremental loads, and applying data warehousing and dimensional modeling concepts. Proficiency with Git, code reviews, automated testing, and deployment practices.

What you'll do

• Design, build, and maintain batch and streaming data pipelines using Databricks, PySpark, Python and SQL
• Ingest and integrate data from databases, files, APIs, and cloud storage into a lakehouse architecture
• Develop bronze, silver, and gold data layers using Delta Lake and implement incremental processing, deduplication, and change data capture where required
• Optimize Spark workloads, SQL queries, compute configurations, and storage layouts for performance and cost efficiency
• Schedule and orchestrate data workflows; implement monitoring, alerting, error handling, and recovery mechanisms

What you need

• 5–8 years in data science & ai
• Databricks
• SQL
• ETL
• Spark
• Data modelling
• AWS
• Any graduate

Good to have

• Terraform
• Python
• GCP
• CI/CD
• Azure
• Git