Lead Data Engineer

About the role

We've completed a multi-year modernisation - migrating from Redshift to StarRocks + Apache Hudi - and cut export latencies from 25 seconds to under 5 seconds. The infrastructure groundwork is done. Now we're building the AI-native data layer that will power revenue-generating features like natural language querying for HackerRank for Work customers.

What you'll do

• Own and evolve the data platform - StarRocks (OLAP), Apache Hudi (Data Lake), Trino, Spark, and Apache Ranger - ensuring performance, reliability, and security at scale
• Own in-product data features - exports, insights dashboards, interview analytics, and the self-serve Custom Reports interface
• Enable self-service pipelines for internal teams (AI platform, analytics, go-to-market), reducing ad-hoc data requests and scaling data access across the org
• Enforce robust data security - access controls, Apache Ranger policies, and confidence-scoring guardrails for AI-generated outputs
• Lead technical design reviews and define engineering standards for the data team

What you need

• 6–11 yrs of experience
• Spark
• AWS
• Go-to-market
• Redshift
• Any graduate

Good to have

• LLMs
• Lean