Data Platform Modernization Engineer - Junior (Databricks)
BengaluruFull-time3–6 yrsPosted over a month ago
Mid-level data science role at NTT Data in Bengaluru. Needs 3+ years and a B.Sc, B.Tech or B.E..
About the role
We are seeking a hands-on Data Engineer / Developer with 3–6 years of experience to support an enterprise data platform modernization initiative. The role will focus on migrating legacy Informatica PowerCenter and AWS Glue workloads to Databricks and developing scalable, reliable data pipelines using established migration patterns and common frameworks.
What you'll do
- Analyze existing Informatica workflows, mappings and AWS Glue jobs to understand source-to-target mappings and transformation logic
- Develop Databricks notebooks and data pipelines to migrate existing ETL workloads to the Databricks Lakehouse
- Develop data transformations using Python, PySpark, Spark SQL and Databricks SQL
- Implement Bronze, Silver and Gold layer processing following established project architecture and coding patterns
- Develop batch and incremental processing pipelines using Delta Lake and established watermark/control mechanisms
What you need
- Databricks
- SQL
- Python
- AWS
- ETL
- Informatica
Good to have
- Redshift
- CI/CD
- Spark
- Unit testing
- SSIS
- Regression testing