Data Engineer Pyspark and Mongo DB
PuneFull-time15–18 yrsPosted 3 months ago
Leadership data science role at Aligned Automation Services in Pune. Needs 15–18 years.
About the role
We are looking for an experienced Data Engineer with strong expertise in MongoDB, PySpark, and Python to design, develop, and optimize scalable data pipelines. The ideal candidate should have experience working with large datasets, NoSQL databases, distributed data processing, and cloud-based data platforms.
What you'll do
- Design, build, and maintain scalable ETL/ELT data pipelines using PySpark and Python
- Develop data ingestion frameworks to process structured, semi-structured, and unstructured data
- Work extensively with MongoDB for data modeling, querying, indexing, aggregation, and performance optimization
- Optimize Spark jobs for high-performance processing of large datasets
- Build reusable data transformation and validation frameworks
What you need
- Spark
- MongoDB
- Python
- SQL
- ETL
- Data structures
Good to have
- AWS
- Databricks
- Kafka
- Airflow
- GCP
- Azure