Data Engineer Sr (Databricks)
BengaluruFull-time8–14 yrsPosted 6 days ago
Leadership data science role at NTT DATA Services in Bengaluru. Needs 8+ years.
About the role
Data Engineer Sr (Databricks) at NTT DATA Services. Develop Databricks notebooks, jobs, and workflows to replicate and enhance DB2/Guidewire-based pipelines and transformations.
What you'll do
- Develop Databricks notebooks, jobs, and workflows to replicate and enhance DB2/Guidewire-based pipelines and transformations
- Implement Delta Lake tables and patterns (bronze/silver/gold, ACID, time travel, schema evolution) for migrated data
- Integrate Databricks with AWS/S3 or Azure ADLS, ADF/Synapse, Key Vault, and Snowflake as required
- Optimize Databricks clusters, jobs, and queries for performance and cost
- Implement incremental loads, CDC patterns, and batch schedules for large datasets
What you need
- Databricks
- SQL
- ETL
- Spark
- Mainframe
- Scala
Good to have
- AWS
- Snowflake
- dbt
- Azure
- CI/CD
- Reconciliation