Lead Senior Associate | Data Engineering | Chennai | Engineering as a Service/ Operate

About the role

We are seeking a Sr Data Engineer I to support the reliability, scalability, and performance of Disney’s Ad Technology big data platforms and pipelines. This role focuses on hands-on development and operations for Databricks, Spark, Flink, and Airflow (MWAA) environments, ensuring that production data workflows are resilient, cost-efficient, and performant at scale.

What you'll do

• Design, maintain, and optimize Spark and Flink pipelines running in Databricks and Airflow
• Ensure reliability and availability of data workflows in production
• Debug and resolve issues in notebooks, jobs, and orchestration workflows
• Tune Spark jobs, clusters, and data pipelines for performance and cost efficiency
• Analyze logs, queries, and workloads to identify bottlenecks and implement improvements

What you need

• 4–7 yrs of experience
• SQL
• ETL
• Python
• Databricks
• Spark
• AWS
• B.Tech or B.E. or B.Sc or BCA

Good to have

• CI/CD
• Git
• Jenkins
• Airflow
• Datadog
• Prometheus