Databricks Tech Lead
BengaluruFull-time20–30 yrsPosted over a month ago
Leadership data science role at EXL in Bengaluru. Needs 20+ years.
About the role
Design, build, and maintain Databricks workspaces, clusters, and compute pools across dev/test/prod environments. Configure and manage Databricks Unity Catalog for data governance, access control, fine-grained permissions, and data lineage. Optimize cluster configurations — instance types, auto-scaling policies, spot/preemptible nodes — for cost and performance.
What you'll do
- Design, build, and maintain Databricks workspaces, clusters, and compute pools across dev/test/prod environments
- Configure and manage Databricks Unity Catalog for data governance, access control, fine-grained permissions, and data lineage
- Optimize cluster configurations — instance types, auto-scaling policies, spot/preemptible nodes — for cost and performance
- Implement workspace-level best practices: folder structures, access controls, secret management (Databricks Secrets / Azure Key Vault / AWS Secrets Manager)
- Manage Databricks jobs, workflows, and multi-task job orchestration with dependency management. Delta Lake & Lakehouse Architecture
What you need
- Databricks
- Machine learning
- ETL
- MLOps
- LLMs
- Spark
Good to have
- AWS
- Kafka
- dbt
- CI/CD
- Azure
- Terraform