DevOps Engineer - AI Runtime Services

About the role

AI Runtime Services owns the shared runtime that AI at Demandbase runs on — two ways. First, it's the layer that enables and supports the AI features in the Demandbase platform: product teams ship LLM features on top of it without reinventing model access, spend control, safety, and observability. Second, the team builds and supports internal AI solutions — the tooling and services the company uses to work with LLMs day to day.

What you'll do

• Reliability, for real. SLOs, on-call, incident response, and the postmortems for the runtime. This is a genuine SRE ownership role
• Cost control. Per-team and per-model attribution, budgets, quotas. LLM spend is the kind of line item that quietly triples
• LLM observability. Traces, spans, prompt/response capture, ingest and billing visibility for every AI feature in the company
• Evals and experimentation infra. The shared harnesses, datasets, and scoring plumbing product teams use to know whether a prompt or model change actually made things…
• Caching and guardrails. Response and semantic caching to cut latency and redundant spend; input/output safety, PII handling, and policy enforcement at the gateway

What you need

• AWS
• Terraform
• Kubernetes
• Python
• LLMs
• Incident response
• Any graduate

Good to have

• CI/CD
• GCP
• Datadog
• Prometheus
• Machine learning
• Plumbing