Lead II - Databricks Engineer
ThiruvananthapuramFull-time8–14 yrsPosted 1 week ago
Leadership data science role at UST in Thiruvananthapuram. Needs 8+ years.
About the role
Lead Databricks Developer (Offshore) Looking for 2 strong Lead Databricks Developers with streaming expertise.
What you'll do
- Design and develop end-to-end data pipelines using Databricks (PySpark, Delta Lake)
- Build real-time streaming ingestion pipelines from Kafka and other event-driven sources
- Implement layered data architecture (Bronze, Silver, Gold) to support transformation and reporting. Develop data processing logic for transactional data ingestion
- Support cloud migration from on-premise systems to Databricks + Snowflake ecosystem. Optimize pipeline performance to meet SLA-driven processing timelines. Implement and…
What you need
- Databricks
- Spark
- Kafka
- ETL
- AWS
- Data modelling
Good to have
- Data privacy
- Airflow
- CI/CD
- Git
- Snowflake