Senior Data Engineer

About the role

SOTI is committed to providing its employees with endless possibilities; learning new things, working with the latest technologies and making a difference in the world.

What you'll do

• Design, build, and maintain production grade ETL/ELT pipelines using SQL, Python, PySpark, and Spark SQL
• Own end to end data pipelines from ingestion through curated, analytics ready datasets
• Maintain and enhance existing integrations to ensure stability, performance, and scalability
• Troubleshoot complex pipeline failures and act as a key escalation point for production support
• Collaborate with Lead Engineers and Data Architects on on-prem to cloud migration initiatives

What you need

• 7–13 years in data science & ai
• SQL
• Python
• ETL
• Spark
• Azure Data Factory
• Azure
• Any graduate

Good to have

• CI/CD
• Test automation
• Git
• SQL Server
• Reconciliation