Lead I - Data Engineering
BengaluruFull-time5–7 yrsPosted 1 week ago
Senior data science role at UST in Bengaluru. Needs 5–7 years and a B.Tech, B.E. or B.Sc.
About the role
Lead the design and development of scalable data pipelines and data-processing solutions using Databricks and Apache Spark. Develop and optimize complex data transformations using PySpark/Python and SQL. Design efficient ETL/ELT workflows for high-volume datasets. Perform performance tuning and optimization of Spark jobs, SQL queries, and data pipelines.
What you'll do
- Lead the design and development of scalable data pipelines and data-processing solutions using Databricks and Apache Spark
- Develop and optimize complex data transformations using PySpark/Python and SQL
- Design efficient ETL/ELT workflows for high-volume datasets
- Perform performance tuning and optimization of Spark jobs, SQL queries, and data pipelines
- Provide technical leadership, design guidance, and hands-on support to the engineering team
What you need
- SQL
- Python
- Spark
- Databricks
- Data analysis
- ETL
Good to have
- AWS
- Data modelling
- Data warehousing
- CI/CD
- Azure
- Git