Vice President, Data Engineering

About the role

The platform currently includes core components such as Apache NiFi, Apache Spark, and MinIO, and supports multiple internal applications for data ingestion, processing, and storage. We are seeking an experienced and visionary Vice President to build and lead a Multi-Agent ETL Platform team.

What you'll do

• Drive modernization from legacy and on-prem systems to modern, cloud-native, and hybrid data platforms
• Architect and lead the development of a Multi-Agent ETL Platform for batch and event streaming
• Define and implement data ingestion, transformation, and delivery pipelines using scalable frameworks (e.g., Apache Airflow, Nifi, dbt, Spark, Kafka, or Dagster)
• Leverage LLMs, and agent frameworks (e.g., LangChain, CrewAI, AutoGen) to automate pipeline management and monitoring
• Ensure robust data governance, cataloging, versioning, and lineage tracking across the ETL platform

What you need

• SQL
• ETL
• Python
• Spark
• Kafka
• Airflow
• B.Tech or B.E. or B.Sc or BCA

Good to have

• dbt
• LLMs
• AWS
• Azure Data Factory
• Data modelling
• Hadoop