Module Lead - AWS Databricks GenAI Job

About the role

We use the following session cookies, which are all required to enable the website to function: "careerSiteCompanyId" is used to send the request to the correct data center "JSESSIONID" is placed on the visitor's device during the session so the server can identify the visitor "Load balancer cookie" (actual cookie name may vary) prevents a visitor from bouncing from one instance to another

What you'll do

• Design, develop, and optimize scalable data pipelines using AWS and Databricks
• Build and manage modern Lakehouse architectures leveraging Delta Lake, Iceberg and Unity Catalog
• Implement data governance capabilities including data quality, data lineage, metadata management, and data security
• Develop ETL/ELT solutions using Python, PySpark, and SQL
• Design and deploy GenAI solutions, including RAG-based chatbots and AI agents

What you need

• AWS
• LLMs
• Databricks
• SQL
• Python
• ETL

Good to have

• Spark
• MLOps
• Data modelling
• Data warehousing
• Redshift
• DynamoDB