Data Engineer Sr. Consultant
BengaluruFull-time3–6 yrsPosted over a month ago
Mid-level data science role at NTT Data in Bengaluru. Needs 3+ years and a B.Sc, B.Tech or B.E..
About the role
Data Engineer Sr. Consultant at NTT Data. Analyze existing Informatica workflows, mappings and AWS Glue jobs to understand source-to-target mappings and transformation logic.
What you'll do
- Analyze existing Informatica workflows, mappings and AWS Glue jobs to understand source-to-target mappings and transformation logic
- Develop Databricks notebooks and data pipelines to migrate existing ETL workloads to the Databricks Lakehouse
- Develop data transformations using Python, PySpark, Spark SQL and Databricks SQL
- Implement Bronze, Silver and Gold layer processing following established project architecture and coding patterns
- Develop batch and incremental processing pipelines using Delta Lake and established watermark/control mechanisms
What you need
- SQL
- Python
- AWS
- ETL
- Informatica
- Databricks
Good to have
- Redshift
- CI/CD
- Spark
- Unit testing
- SSIS
- Regression testing