Lead Data Engineer
PuneFull-time8–14 yrsPosted 4 months ago
Leadership data science role at EXL in Pune. Needs 8+ years and a B.Tech, B.E. or B.Sc.
About the role
You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
What you'll do
- Build services that will integrate directly with Client's products and with external vendors
- Work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc.
- Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc
- Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage
- Support operations to manage the production environment and help in resolving production issues with RCA
What you need
- Python
- SQL
- AWS
- Spark
- Databricks
- ETL
Good to have
- Hadoop
- dbt
- Airflow
- Snowflake
- JIRA
- Shell scripting