Data Engineer II

About the role

This role requires a well-rounded data engineer with hands-on experience in data processing technologies, a good understanding of data modeling concepts, and the ability to collaborate effectively with various stakeholders.

What you'll do

• Utilize PySpark and Python to construct efficient and scalable data pipelines
• Integrate data from multiple source systems into a unified target system
• Use Apache Airflow to orchestrate and schedule data pipelines, ensuring timely and reliable execution
• Understand existing data pipelines and make enhancements based on evolving business requirements
• Implement improvements to optimize performance and maintainability

What you need

• 3–6 yrs of experience
• Python
• SQL
• Data modelling
• Airflow
• Spark
• Data warehousing
• Any graduate

Good to have

• AWS
• dbt
• ETL
• Data structures
• Docker
• Root cause analysis