Data Engineer

About the role

Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. A hands-on, build-and-run role: writing and operating the ETL/ELT pipelines, lakehouse tables, and graph data models that bring data from many enterprise source systems into a governed, analytics- and AI-ready platform.

What you'll do

• Build, schedule, and maintain ETL/ELT pipelines — extracting from source systems, transforming with PySpark/Spark SQL
• Write and maintain transformation logic for incremental loads, change data capture (CDC), deduplication, slowly changing dimensions (SCD), and schema evolution
• Hands-on troubleshooting of failed or delayed pipeline runs — diagnosing root cause, fixing transformation bugs, and rebuilding/backfilling data as needed
• Model and query graph data (vertices, edges, properties) directly in Azure Cosmos DB for Apache Gremlin
• Write SPARQL queries and author Turtle (.ttl) files to populate and query RDF-based knowledge graphs, working with a triplestore such as Graphwise GraphDB

What you need

• 5–8 years in data science & ai
• Machine learning
• SQL
• ETL
• Python
• Pandas
• Spark
• M.Sc or B.Tech or B.E. or B.Sc or BCA or M.Tech or MCA

Good to have

• Airflow
• AWS
• GCP
• Databricks
• Azure
• NoSQL