Databricks Engineer II (Pyspark+

About the role

Databricks - Unity catalog, Auto loader, Spark, pySpark , Workflows and Jobs Advanced SQL - Mastery on Window function, CTE, Store procedure, joins, CTEs, aggregations, query optimization. Delta Lake — ACID transactions, time travel, schema evolution. Pyspark , Incremental pipelines, Batch data pipeline and streaming.

What you'll do

• Design and implement ELT pipelines using PySpark and Databricks Workflows
• Write efficient, reusable, and scalable Python code for data processing and automation tasks
• Orchestrate complex workflows using Azure Data Factory and integrate with other Azure services
• Collaborate with analysts and data scientists to model and deliver trusted datasets
• Monitor and troubleshoot data pipelines, ensuring high availability and reliability

What you need

• 5–8 yrs of experience
• Databricks
• ETL
• Python
• Spark
• Azure Data Factory
• SQL
• Any graduate

Good to have

• Machine learning
• Azure
• Git
• AWS
• CI/CD
• GCP