Big Data Engineer
PuneFull-time7–10 yrsPosted 3 months ago
Senior data science role at EXL in Pune. Needs 7–10 years.
About the role
We are looking for an experienced Lead Data Engineer with 7–10 years of hands-on experience to design, build, and maintain scalable data pipelines and processing systems in an on-premises Big Data environment . This is a technical leadership role: beyond strong individual contribution, the ideal candidate will own architecture and design decisions, set technical direction, and mentor and support other developers on the team.
What you'll do
- Lead the design, development, and maintenance of robust, scalable data pipelines for ingestion, transformation
- Own architectural and design decisions for data solutions, evaluating trade-offs and defining technical standards for the team
- Mentor, guide, and support other data engineers through code reviews, design reviews, technical coaching, and hands-on problem-solving
- Build and optimize data workflows using Python, Spark, and the Hadoop ecosystem
- Work extensively with Hadoop ecosystem components (Hive, HDFS, Impala) to manage and query large-scale data
What you need
- Machine learning
- Python
- SQL
- ETL
- Hadoop
- Spark
Good to have
- Hive
- Data structures