Data Engineer Pyspark
GurugramFull-time5–8 yrsPosted today
Senior data science role at Citi in Gurugram. Needs 5–8 years.
About the role
This is a hands-on development role responsible for building, testing, optimizing, and supporting batch and real-time data-processing solutions using technologies such as Apache Spark, Databricks, Kafka, Scala, Python, and SQL.
What you'll do
- Develop, test, enhance, and support batch and real-time data pipelines using Apache Spark, Databricks, Kafka, Scala, Python, and SQL
- Write clean, reusable code and participate in code reviews, unit testing, integration testing, data validation, and deployment activities
- Analyze Spark execution plans, DAGs, metrics, and logs to troubleshoot failures and improve processing performance
- Build and maintain Iceberg or Delta Lake solutions using established Lakehouse and medallion architecture patterns
- Collaborate with technical leads, developers, analysts, and support teams to deliver assigned work and resolve production issues while following established controls and…
What you need
- Spark
- SQL
- Python
- Databricks
- Kafka
- Scala
Good to have
- ETL
- Unit testing