Data Engineer 2

About the role

This job is integral to the development and optimization of data structures and pipelines, ensuring quality and efficient data flow. It supports advanced analytics and machine learning initiatives, while also managing data storage solutions. The role requires collaboration with partners for process optimization and adherence to data governance standards.

What you'll do

• Developing data structures and pipelines to collect, standardize, and transform data for insight generation and reporting
• Ensuring data quality throughout ingestion, processing, and loading to target tables, with a focus on standardization and transformation rules
• Creating ingestion frameworks for various data types and establishing methods for data consumption through views, extracts, and APIs
• Maintaining data warehouses and lakes, including API development for data access, while managing solutions on Kubernetes, Teradata, and Databricks platforms
• Determining suitable storage solutions on-prem (minIO, Teradata) and cloud (AWS S3, Redshift) based on data sensitivity and access requirements

What you need

• 2–5 yrs of experience
• AWS
• ETL
• Machine learning
• Databricks
• Redshift
• Data structures
• Any graduate

Good to have

• REST APIs
• Kubernetes