Data Engineer Pyspark

About the role

This is a hands-on development role responsible for building, testing, optimizing, and supporting batch and real-time data-processing solutions using technologies such as Apache Spark, Databricks, Kafka, Scala, Python, and SQL.

What you'll do

• Develop, test, enhance, and support batch and real-time data pipelines using Apache Spark, Databricks, Kafka, Scala, Python, and SQL
• Write clean, reusable code and participate in code reviews, unit testing, integration testing, data validation, and deployment activities
• Analyze Spark execution plans, DAGs, metrics, and logs to troubleshoot failures and improve processing performance
• Build and maintain Iceberg or Delta Lake solutions using established Lakehouse and medallion architecture patterns
• Collaborate with technical leads, developers, analysts, and support teams to deliver assigned work and resolve production issues while following established controls and…

What you need

• 5–8 yrs of experience
• Spark
• SQL
• Python
• Databricks
• Kafka
• Scala
• Any graduate

Good to have

• ETL
• Unit testing