Senior Data Engineer
BengaluruFull-time5–10 yrsPosted 4 months ago
Senior data science role at HealthEdge in Bengaluru. Needs 5+ years.
About the role
Description Sr Data Engineer Position Summary We are looking for a self-sufficient BI Analyst / Data Engineer to join a small, high-impact analytics team. In this role you will own the full lifecycle of business intelligence delivery—from building and maintaining ETL pipelines in AWS to designing interactive dashboards in Amazon QuickSight.
What you'll do
- ETL / Data Pipeline Development Design, build, and maintain AWS Glue (PySpark) ETL jobs that ingest data from multiple source systems such as Salesforce, JIRA
- Develop and optimize AWS Step Functions workflows for pipeline orchestration, including parallel execution, error handling, retry logic, and concurrency management
- Implement incremental loading strategies, schema evolution handling, and data quality checks within Glue scripts
- Work with Apache Iceberg table format on S3 for ACID-compliant analytics tables, including MERGE operations, compaction, snapshot expiration, and orphan file cleanup
- Integrate supporting AWS services such as EventBridge, SQS, SNS, Athena, and CloudWatch into pipeline architectures
What you need
- SQL
- ETL
- AWS
- Python
- Spark
- Data warehousing
Good to have
- Data analysis
- Terraform
- Data modelling
- CI/CD
- JIRA
- Salesforce