DE&A -Data Engineering Apache

About the role

Design, develop, and implement scalable, resilient, and high-performance data pipelines and data architecturesfor enterprise data platforms. Build and optimize ETL/ELT pipelines supporting batch, near-real-time, and streaming data processing requirements.

What you'll do

• Design, develop, and implement scalable, resilient, and high-performance data pipelines and data architecturesfor enterprise data platforms
• Build and optimize ETL/ELT pipelines supporting batch, near-real-time, and streaming data processing requirements
• Design and implement enterprise data ingestion and integration workflows using Apache NiFi, including data routing, transformation, orchestration, monitoring
• Develop distributed data processing solutions using Apache Spark with Scala and/or Python (PySpark)
• Design and implement data solutions across relational database management systems (RDBMS) and modern data lakehouse technologies such as Apache Iceberg

What you need

• 7–13 years in data science & ai
• SQL
• Python
• Machine learning
• ETL
• Spark
• Kafka
• Any graduate

Good to have

• Data modelling
• CI/CD
• Scala
• Root cause analysis
• Test automation