Data Engineer
PuneFull-timeJunior levelPosted 1 week ago
Data science role at Citi in Pune.
About the role
In this role, you will design, develop, and maintain scalable data pipelines and analytical solutions, supporting large-scale data operations across enterprise systems. You will work within a full software development lifecycle (SDLC) environment spanning Development (NAM), UAT, and Production.
What you'll do
- Design, develop, test, and maintain PySpark and Apache Spark data pipelines for batch and streaming workloads
- Manage and monitor Spark Batch Jobs (running hourly or every 2 hours) that store audit, inbound, and outbound data into Hive tables (e.g., gfolyreg_managed.SWXPTT_audit)
- Respond to ITRS Alerts and system monitoring events (e.g., job failures, kills, or jobs not triggered within 30 minutes of scheduled time)
- Work with large-scale data sources across Oracle, Hadoop, Impala, and SAS Server
What you need
- Spark
- ETL
- SAS
- Hadoop
- Hive