Data Engineer

About the role

We are looking for a Data Engineer with strong expertise in Azure Databricks , PySpark , and distributed computing to develop and optimize scalable ETL pipelines for manufacturing analytics. The role involves working with high-frequency industrial data to enable real-time and batch data processing .

What you'll do

• Build scalable real-time and batch processing workflows using Azure Databricks , PySpark , and Apache Spark
• Perform data pre-processing, including cleaning, transformation, deduplication, normalization, encoding, and scaling to ensure high-quality input for downstream analytics
• Design and maintain cloud-based data architectures , including data lakes , lakehouses , and warehouses , following Medallion Architecture
• Deploy and optimize data solutions on Azure (preferred) , AWS , or GCP , with a focus on performance, security, and scalability
• Develop and optimize ETL/ELT pipelines for structured and unstructured data from IoT, MES, SCADA, LIMS, and ERP systems

What you need

• 2–4 years in data science & ai
• Python
• Spark
• Databricks
• Azure
• Machine learning
• ETL
• B.Tech or B.E. or B.Sc or BCA or M.Tech or MCA or M.Sc

Good to have

• AWS
• MLOps
• GCP
• Docker
• Kubernetes
• SQL Server

What you get

• ₹12 L–₹16 L a year