Snr Data Engineer
Gurugram, NoidaFull-timeJunior levelPosted 2 months ago
Data science role at Alight Solutions in Gurugram and Noida.
About the role
Gurgaon, Haryāna, India, 122018; Noida, Uttar Pradesh, India, 201304
What you'll do
- Build and maintain high volume ETL/ELT pipelines across Hadoop (HDFS, Hive, Spark, Kafka) and AWS (Glue, EMR, Lambda, Step Functions, Redshift)
- Develop distributed data processing solutions using PySpark, Spark SQL, and scalable cloud serverless patterns
- Implement reusable data ingestion frameworks for batch, ability to design & implement Orchestration process and Leverage AI
- Optimize data workflows using partitioning, bucketing, compression, file formats (Parquet/ORC)
- Understanding hybrid data lake architectures using S3 + HDFS, ensuring data governance and best practices are adheres
What you need
- SQL
- AWS
- Python
- Hadoop
- Spark
- ETL
Good to have
- Data modelling
- Kafka
- Redshift
- Airflow
- Data warehousing
- Snowflake