Software Engineering Lead (GO & Python, AWS)

About the role

We are seeking a Senior Software Engineer to join the Enterprise Logging team and lead development of a next-generation log processing pipeline. This role is responsible for designing, building, and operating a high-throughput Lambda-based consumer that transforms and delivers ~500 TB/day of cloud log data from AWS, GCP, and Azure into Splunk Cloud.

What you'll do

• Design and develop a serverless event processing pipeline (AWS Lambda) that reads from Kinesis Data Streams, transforms events
• Implement event parsing and transformation logic across multiple cloud log formats (CloudTrail, VPC Flow Logs, GCP Audit Logs, Azure Activity Logs, and others)
• Build and maintain source/source type mapping configurations supporting 59,000+ unique log type combinations
• Develop tiered data routing logic (Keep / Reduce / Skip) to optimize Splunk licensing costs
• Implement timestamp extraction and normalization across heterogeneous event formats

What you need

• 5–9 yrs of experience
• Python
• AWS
• Go
• Git
• Kafka
• GCP
• Any graduate

Good to have

• Azure
• Splunk
• Terraform
• ETL
• CI/CD
• IAM