Senior Technical Manager

About the role

To design, build, and maintain scalable, reliable, and secure data pipelines that enable seamless integration of diverse global data sources for Generative AI solutions. The Data Engineer ensures high-quality, timely, and accessible data for AI models by managing RAG indexes, monitoring pipeline health, and continuously onboarding new data sources.

What you'll do

• Design, implement, and maintain robust data pipelines and ETL processes to support business and product requirements
• Monitor the performance and reliability of data pipelines, proactively addressing issues and optimizing workflows
• Connect and integrate diverse data sources (structured and unstructured) across global systems
• Identify and integrate new potential data sources to enhance the breadth and quality of available data
• Build and maintain Retrieval-Augmented Generation (RAG ) indexes for efficient data retrieval and model performance

What you need

• 12–22 yrs of experience
• SQL
• LLMs
• Python
• Spark
• Databricks
• Machine learning
• B.Tech or B.E. or B.Sc or BCA or M.Tech or MCA or M.Sc

Good to have

• ETL
• Azure
• REST APIs