Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg

About the role

Data Engineer - Python AND Kafka AND (Hadoop OR HDFS OR Hive) AND Snowflake AND apache AND (iceberg at NTT DATA Services. Engineer will be part of the datastore-migration Factory team that will be responsible to perform for the end-to-end datastore migration from on-prem DataLake to AWS….

What you'll do

• Engineer will be part of the datastore-migration Factory team that will be responsible to perform for the end-to-end datastore migration from on-prem DataLake to AWS…
• This is a high visibility and crucial project for Goldman Sachs
• Data Reconciliation & Quality
• A rigorous approach to data validation is required. Candidates must work with reconciliation frameworks to build confidence that migrated data is functionally equivalent…

What you need

• 3–5 yrs of experience
• Python
• Hadoop
• Snowflake
• Kafka
• Hive
• SQL
• B.Tech or B.E. or B.Sc or BCA or M.Tech or MCA or M.Sc

Good to have

• Spark
• Data modelling
• Reconciliation
• CI/CD
• Java
• Kubernetes