Data Engineer- DataBricks
RemoteFull-time5–8 yrsPosted 2 months ago
Senior data science role at Paloaltoinnovationlabs. Needs 5–8 years and a B.Tech, B.E. or B.Sc. Remote.
About the role
Data Engineer- DataBricks at Paloaltoinnovationlabs. Design and implement CI/CD pipelines for data.
What you'll do
- Design and implement CI/CD pipelines for data
- Pipelines (ingestion jobs) and transformation projects (e.g., dbt on
- Orchestrate Databricks jobs and workflows
- End-to-end (ingestion, transformation, quality checks)
- Integrate automated testing into CI/CD
What you need
- Databricks
- Python
- CI/CD
- SQL
- Terraform
- Azure
Good to have
- Git
- Bash
- PowerShell
- Airflow
- dbt
- Test automation