Airflow Engineer
ChennaiFull-time4–7 yrsPosted 2 months ago
Mid-level engineering role at EXL in Chennai. Needs 4+ years and a B.Tech, B.E. or B.Sc.
About the role
Review existing shell scripts and API/cURL invocation logic to determine the appropriate Airflow DAG design, task structure, error handling, and retry configuration. Design, develop, and maintain Apache Airflow DAGs to orchestrate shell scripts, API calls, and dependent jobs with appropriate scheduling and operational controls.
What you'll do
- Analyze existing Cybermation job schedules, dependency patterns, runtime parameters, execution frequency, owners, success/failure behavior
- Review existing shell scripts and API/cURL invocation logic to determine the appropriate Airflow DAG design, task structure, error handling, and retry configuration
- Design, develop, and maintain Apache Airflow DAGs to orchestrate shell scripts, API calls, and dependent jobs with appropriate scheduling and operational controls
- Configure Airflow schedules, task dependencies, retries, logging, alerting hooks, variables, connections, parameters
- Perform dry runs, unit testing, schedule validation, API response validation, failure/retry validation, and production-readiness checks for migrated jobs
What you need
- Airflow
- Python
- Unit testing
- Linux