Data Engineer
PuneFull-time2–5 yrsPosted 1 week ago
Mid-level data science role at Match Me Resume Match | Citi in Pune. Needs 2+ years and a B.Tech, B.E. or B.Sc.
About the role
Citi is seeking a Applications Development Programmer Analyst 2 (C10) to join its Technology – Data Engineering function, focused on the PySpark / Spark-based Data Platform. In this role, you will design, develop, and maintain scalable data pipelines and analytical solutions, supporting large-scale data operations across enterprise systems.
What you'll do
- Design, develop, test, and maintain PySpark and Apache Spark data pipelines for batch and streaming workloads
- Manage and monitor Spark Streaming Jobs (running Sunday evening through Saturday afternoon) performing Eligibility and Data Quality (DQ) checks
- Manage and monitor Spark Batch Jobs (running hourly or every 2 hours) that store audit, inbound, and outbound data into Hive tables (e.g., gfolyreg_managed.SWXPTT_audit)
- Respond to ITRS Alerts and system monitoring events (e.g., job failures, kills, or jobs not triggered within 30 minutes of scheduled time)
- Work with large-scale data sources across Oracle, Hadoop, Impala, and SAS Server
What you need
- Python
- SQL
- ETL
- Spark
- Kafka
- Hive
Good to have
- Git
- SAS
- Hadoop
- R