Senior Data Engineer
GurugramFull-time8–14 yrsPosted 4 months ago
Leadership data science role at EXL in Gurugram. Needs 8+ years and a B.Tech, B.E. or B.Sc.
About the role
The Databricks Data Engineer will be responsible for designing, building, and optimizing scalable data pipelines and lakehouse solutions using Databricks. The role requires strong hands‑on experience in data engineering, distributed data processing.
What you'll do
- Design, build, and maintain ETL/ELT pipelines on Databricks using PySpark, Spark SQL, and Delta Lake
- Develop and optimize data ingestion frameworks, data transformations, and end-to-end workflows for batch and streaming use cases
- Implement Delta Lake-based architectures, including versioning, schema evolution, and ACID-compliant pipelines
- Work with stakeholders to understand data requirements and translate them into scalable data engineering solutions
- Manage and optimize Databricks clusters, jobs, and notebooks for performance and cost efficiency
What you need
- SQL
- Databricks
- Spark
- Machine learning
- ETL
- AWS
Good to have
- Data modelling
- Kafka
- Redshift
- CI/CD
- MIS reporting
- Azure