Data Engineer

About the role

Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. As a Data Engineer, a typical day involves designing, developing, and maintaining comprehensive data solutions that support the generation, collection, and processing of data.

What you'll do

• Design, develop, and maintain data pipelines and ETL processes using Microsoft Azure services (e.g., Azure Data Factory, Azure Synapse, Azure Databricks, Azure Fabric)
• Utilize Azure data storage accounts for organizing and maintaining data pipeline outputs. (e.g., Azure Data Lake Storage Gen 2 & Azure Blob storage)
• Collaborate with data scientists, data analysts, data architects and other stakeholders to understand data requirements and deliver high-quality data solutions
• Optimize data pipelines in the Azure environment for performance, scalability, and reliability
• Ensure data quality and integrity through data validation techniques and frameworks

What you need

• 5–9 years in data science & ai
• Databricks
• Azure
• Data analysis
• ETL
• Spark
• Azure Data Factory
• Any graduate

Good to have

• Python
• C#
• Scala
• Java
• CI/CD