Data Engineer
PuneFull-time3–8 yrsPosted 2 months ago
Mid-level data science role at Xoriant in Pune. Needs 3+ years and an M.Sc, B.Tech or B.E..
About the role
In this role, you will design, build, and operate scalable data pipelines and curated datasets on Microsoft Azure, with a strong focus on hands-on development using Databricks, Apache Spark, and Delta Lake. You will collaborate with QA and DevOps engineers and other stakeholders to develop reliable, high-quality data solutions that support business intelligence, reporting, and downstream analytics.
What you'll do
- Build and maintain scalable data pipelines on Microsoft Azure using Databricks and Azure services for data ingestion, transformation, validation, and publishing
- Develop curated, analytics-ready datasets for business intelligence, reporting, and downstream data consumers
- Apply fact table design and dimensional data modeling concepts to build efficient and reliable analytical datasets
- Implement incremental data loading patterns, backfills, and late-arriving data handling to minimize unnecessary full data reloads
- Establish data quality checks, reconciliation processes, pipeline monitoring, and alerting mechanisms to ensure data accuracy and reliability
What you need
- Machine learning
- SQL
- Python
- ETL
- LLMs
- Databricks
Good to have
- Spark
- Data modelling
- Azure Data Factory
- Azure
- Reconciliation
- Cloud security