Senior Developer, Data Engineer
HyderabadFull-time3–6 yrsPosted over a month ago
Mid-level data science role at Intercontinental Exchange (ICE) in Hyderabad. Needs 3+ years.
About the role
Job Purpose Intercontinental Exchange, Inc. (ICE) is seeking an experienced Senior Data Engineer to join the AI Centre of Excellence (AI CoE) team. The AI CoE drives the adoption of artificial intelligence and advanced analytics across ICE's global exchange and financial data business, bringing together data engineering, machine learning, and applied research to deliver intelligent solutions at scale.
What you'll do
- Own Apache Airflow end-to-end
- Write DAGs that handle complex multi-step workflows across ingestion, transformation, validation, and delivery
- Also review other engineers' DAGs and raise the standard across the team
- Keep Airflow running well on Kubernetes
- Manage the deployment using KubernetesExecutor or CeleryKubernetesExecutor, handle Helm upgrades, tune pod templates and resource limits
What you need
- Machine learning
- Python
- ETL
- Pandas
- Airflow
- Databricks
Good to have
- Kafka
- MLOps
- dbt
- Spark
- AWS
- Kubernetes