Lead II -Databrick Engineer
ThiruvananthapuramFull-time8–14 yrsPosted 1 week ago
Leadership engineering role at UST in Thiruvananthapuram. Needs 8+ years.
About the role
Lead Databricks Engineer-Lead Databricks Developers with streaming expertise.
What you'll do
- Design and develop end-to-end data pipelines using Databricks (PySpark, Delta Lake)
- Build real-time streaming ingestion pipelines from Kafka and other event-driven sources
- Implement layered data architecture (Bronze, Silver, Gold) to support transformation and reporting. Develop data processing logic for transactional data ingestion
- Support cloud migration from on-premise systems to Databricks + Snowflake ecosystem. Optimize pipeline performance to meet SLA-driven processing timelines. Implement and…
What you need
- Kafka
- Databricks
- Spark
- Git
- AWS
- ETL
Good to have
- Data privacy
- CI/CD
- Data modelling
- Airflow
- Snowflake