Senior Data Engineer-1
HyderabadFull-time5–9 yrsPosted 3 weeks ago
Senior data science role at Mastercard in Hyderabad. Needs 5+ years and a B.Sc, B.Tech or B.E..
About the role
This is an experienced individual-contributor role that owns end-to-end pipeline delivery and mentors peers. It will appeal to you if you combine deep technical expertise with the ability to independently deliver complex data solutions and raise the bar for the engineers around you. You will bring cutting edge software and full stack development skills with advanced knowledge of cloud and data lake experience while working with massive data volumes.
What you'll do
- Own the end-to-end design, development, and delivery of complex batch and real-time data pipelines using Databricks, Spark, Python, and PySpark
- Drive dbt-based transformation architecture with rigorous Test-Driven Development (TDD) and modular design
- Design and own CI/CD pipelines using GitLab and Jenkins, and champion DataOps practices across the team
- Administer and optimize Databricks objects and the platform using Databricks Asset Bundles, managing performance, access, and cost
- Establish data observability frameworks covering quality, freshness, lineage, and anomaly detection
What you need
- Python
- SQL
- Databricks
- Spark
- dbt
- AWS
Good to have
- CI/CD
- Git
- Jenkins
- GCP
- Azure
- ETL