Lead Software Engineer, Data Engineering

About the role

Lead Software Engineer, Data Engineering at ValGenesis. Architect, Design, develop, and maintain data ingestion, transformation, and orchestration pipelines (batch and real-time).

What you'll do

• Architect, Design, develop, and maintain data ingestion, transformation, and orchestration pipelines (batch and real-time)
• Build and optimize data Lakehouse architectures using Azure Synapse, Delta Lake, or similar frameworks
• Integrate and manage structured and unstructured data sources (SQL/NoSQL, files, documents, IoT streams)
• Develop and operationalize ETL/ELT pipelines using Azure Data Factory, Databricks, or Apache Spark
• Collaborate with Data Scientists to prepare and serve ML-ready datasets for model training and inference

What you need

• 6–12 years in data science & ai
• Machine learning
• SQL
• ETL
• Python
• Azure Data Factory
• MLOps
• Any graduate

Good to have

• LLMs
• Unity
• Databricks
• Data modelling
• Airflow
• dbt