Principal Data Engineer

About the role

This is a senior individual contributor role on the team that builds and runs HG's data platform: everything between a raw vendor file landing in our lake and a finished dataset arriving in a customer's hands.

What you'll do

• Own the architecture of the data platform end to end, and make the calls on build vs. buy, batch vs. incremental, and where each workload belongs
• Build the quality layer that catches silent failure: contracts at every handoff, freshness and completeness monitoring
• Improve entity matching and attribution — and make match decisions explainable to the customers who depend on them
• Make our release cycle boring. Recurring deliveries should not depend on people watching them
• Treat compute cost as a first-class engineering metric across Spark, orchestration, and the serving tier

What you need

• SQL
• Machine learning
• Spark
• AWS
• Databricks
• Data modelling
• Any graduate

Good to have

• Python
• Terraform
• CI/CD
• Scala
• Java
• Docker