Data Engineer- GCP/Databricks
BengaluruFull-timeHybrid5–9 yrs30 L a yearPosted over a month ago
Senior data science role at AuxoAI Engineering in Bengaluru. Needs 5+ years. Hybrid; pays ₹30 LPA.
About the role
This role is ideal for someone with deep experience building scalable batch and streaming data workflows across cloud and lakehouse environments, strong hands-on engineering skills, and a drive to mentor junior engineers.
What you'll do
- Design and build scalable batch and streaming data pipelines across bronze, silver, and gold medallion layers
- Build historical and incremental ingestion using Auto Loader/Spark Structured Streaming/Kafka feeds, with GCS/Azure/AWS storage and Databricks Jobs orchestration
- Develop and maintain Databricks-based pipelines using Spark and Delta Lake for lakehouse architecture, including migration of legacy or on-premises data sources
- Design and maintain analytical data layers in BigQuery or Databricks SQL, applying best practices in partitioning, clustering, and performance tuning
- Implement SQL/PySpark transformations for wide and semi-structured data, including wide-to-long processing and typed or hybrid models suited to consumer requirements
What you need
- Databricks
- GCP
- SQL
- Python
- Spark
- ETL
Good to have
- BigQuery
- Data modelling
- Data warehousing
- Machine learning
- CI/CD
- Terraform