Lead II - Data Engineering

UST·Thiruvananthapuram

Pay not stated7–9 yrsFull-time

Apply on UST's site Track your applications and see your fit: start free

Read from the company's own careers page; you apply on their site.

How well do you fit this job?

Upload your resume to see each requirement checked against your profile, in under a minute.

Check my fit

Hard to read?

About the role

Design and develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications. Build scalable backend services and data engineering solutions that support data processing and operational workflows. Develop and maintain data transformation processes using PySpark, SQL, Hive, Python, Scala, and related technologies.

What you'll do

Design and develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications
Build scalable backend services and data engineering solutions that support data processing and operational workflows
Develop and maintain data transformation processes using PySpark, SQL, Hive, Python, Scala, and related technologies
Work with AWS cloud services such as S3, EC2, EMR, Lambda, DynamoDB, and API Gateway to build cloud-native data and backend solutions
Build and enhance workflow orchestration using Apache Airflow, including advanced DAG design, dependency management, scheduling, monitoring, and failure handling