Experience:

• 5+ years of strong development experience
• Experience deploying and operating LLM inference services in production
• 3+ years hands-on experience in performance optimization and systems programming for AI/ML workloads
• Strong production coding skills in Python plus Go or Rust
• Demonstrated ability to deliver measurable production improvements
• Proven skill in root-cause analysis across model, runtime, networking, and infrastructure

Responsibilities:

• Own production inference from handoff to production-grade serving
• Handle release engineering, capacity planning, cost optimization, and incident response
• Reduce end-to-end latency and increase throughput on real production traffic
• Scale inference across heterogeneous GPU fleets
• Build benchmarking suites, metrics, and tooling for latency, throughput, GPU utilization, memory, and cost
• Improve monitoring, tracing, and alerting
• Participate in incident response and postmortems
• Evaluate research and implement pragmatic inference optimizations
• Work with data science and product teams to translate business requirements into SLOs

Additional responsibilities:

• Optimize inference stacks and related components such as schedulers, KV cache, batching, and memory
• Harden systems through operational learning and postmortem follow-up

Nice to have:

• CUDA/kernel programming
• Speculative decoding

More skills:

GPU systems, continuous batching, KV cache management, speculative decoding, profiling, memory bandwidth, latency tradeoffs, monitoring, tracing, alerting, benchmarking, systems programming

Other:

• Engineering team
• Equal opportunity employer
• All qualified applicants receive consideration without regard to protected characteristics
• Accommodation support is available on request
• Talent Privacy Notice, Privacy Center, and AI Hiring Guidelines are referenced
• AI tools may be used for administrative tasks in the hiring process
• Accessibility statement is available