Lead II - Data Engineering
ThiruvananthapuramFull-time7–9 yrsPosted 1 week ago
Senior data science role at UST in Thiruvananthapuram. Needs 7–9 years.
About the role
Lead II - Data Engineering at UST. Design, develop, and maintain ETL/ELT pipelines using AWS Glue (PySpark and Python shell jobs), Glue Data Catalog, crawlers, and workflows.
What you'll do
- Design, develop, and maintain ETL/ELT pipelines using AWS Glue (PySpark and Python shell jobs), Glue Data Catalog, crawlers, and workflows
- Write complex, optimized SQL for data transformation, validation, and analysis across platforms such as Amazon Redshift, Athena, and relational databases
- Build reusable Python libraries, frameworks, and utilities for data ingestion, transformation, and quality checks
- Model data for analytical and operational use cases (star/snowflake schemas, data lakes/lakehouse patterns)
- Implement data quality, lineage, monitoring, and ing for production pipelines
What you need
- Python
- SQL
- AWS
- CI/CD
- ETL
- Spark
Good to have
- Redshift
- Data warehousing
- Data modelling
- Snowflake
- Root cause analysis
- Unit testing