Data Engineer
BengaluruFull-time2–5 yrsPosted 1 week ago
Mid-level data science role at Quest Global in Bengaluru. Needs 2+ years.
About the role
Data Engineer at Quest Global. Develop, maintain, and optimize Spark-based data processing pipelines using Scala.
What you'll do
- Develop, maintain, and optimize Spark-based data processing pipelines using Scala
- Work with distributed computing frameworks and resource management systems such as YARN
- Write complex SQL queries for data extraction, transformation, validation, and performance optimization
- Perform data validation, quality checks, and troubleshooting across datasets and jobs
- Monitor and improve data pipeline performance, ensuring high availability and reliability
What you need
- SQL
- Hadoop
- ETL
- Spark
- Scala
- Hive
Good to have
- Python
- Java
- CI/CD
- Git