Senior Data Lead - Azure Data Factory (ADF), Databricks

About the role

Senior Data Lead - Azure Data Factory (ADF), Databricks at HCLTech. Design, develop, and maintain scalable data pipelines using Databricks.

What you'll do

• Design, develop, and maintain scalable data pipelines using Databricks
• Build and optimize ETL/ELT processes for structured and unstructured data
• Develop solutions using PySpark, Spark SQL, Delta Lake, and Databricks Workflows
• Implement data ingestion frameworks from various source systems including APIs, databases, cloud storage, and streaming platforms
• Design and maintain Bronze, Silver, and Gold data layers following Medallion Architecture

What you need

• 7–13 yrs of experience
• Databricks
• Azure Data Factory
• SQL
• Python
• Machine learning
• ETL
• Any graduate

Good to have

• Scala
• AWS
• Spark
• Data warehousing
• Snowflake
• Data modelling