Senior Databricks Engineer
PuneFull-time20–30 yrsPosted over a month ago
Leadership data science role at EXL in Pune. Needs 20+ years.
About the role
Design, build, and maintain Databricks workspaces, clusters, and compute pools across dev/test/prod environments. Configure and manage Databricks Unity Catalog for data governance, access control, fine-grained permissions, and data lineage. Optimize cluster configurations — instance types, auto-scaling policies, spot/preemptible nodes — for cost and performance.
What you'll do
- Design, build, and maintain Databricks workspaces, clusters, and compute pools across dev/test/prod environments
- Configure and manage Databricks Unity Catalog for data governance, access control, fine-grained permissions, and data lineage
- Optimize cluster configurations — instance types, auto-scaling policies, spot/preemptible nodes — for cost and performance
- Implement workspace-level best practices: folder structures, access controls, secret management (Databricks Secrets / Azure Key Vault / AWS Secrets Manager)
- Manage Databricks jobs, workflows, and multi-task job orchestration with dependency management. Delta Lake & Lakehouse Architecture
What you need
- Databricks
- Machine learning
- SQL
- AWS
- ETL
- MLOps
Good to have
- LLMs
- Kafka
- Spark
- dbt
- CI/CD
- Azure