Lead II - Data Engineering
Thiruvananthapuram, HyderabadFull-time7–9 yrsPosted 1 week ago
Senior data science role at UST in Thiruvananthapuram and Hyderabad. Needs 7–9 years.
About the role
Design and develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications. Build scalable backend services and data engineering solutions that support data processing and operational workflows. Develop and maintain data transformation processes using PySpark, SQL, Hive, Python, Scala, and related technologies.
What you'll do
- Design and develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications
- Build scalable backend services and data engineering solutions that support data processing and operational workflows
- Develop and maintain data transformation processes using PySpark, SQL, Hive, Python, Scala, and related technologies
- Work with AWS cloud services such as S3, EC2, EMR, Lambda, DynamoDB, and API Gateway to build cloud-native data and backend solutions
- Build and enhance workflow orchestration using Apache Airflow, including advanced DAG design, dependency management, scheduling, monitoring, and failure handling
What you need
- Python
- AWS
- Spark
- Airflow
- ETL
- REST APIs
Good to have
- SQL
- DynamoDB
- Business analysis
- Investment banking
- Hive
- Scala