Lead Data Engineer

About the role

You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.

What you'll do

• Build services that will integrate directly with Client's products and with external vendors
• Work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc.
• Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc
• Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage
• Support operations to manage the production environment and help in resolving production issues with RCA

What you need

• 8–14 years in data science & ai
• Python
• SQL
• AWS
• Spark
• Databricks
• ETL
• B.Tech or B.E. or B.Sc or BCA

Good to have

• Hadoop
• dbt
• Airflow
• Snowflake
• JIRA
• Shell scripting