We are seeking highly skilled and motivated software engineers to join us and build AI inference systems that serve large-scale models with extreme efficiency.

Company: NVIDIA

Role: Senior Software Engineer, AI Inference Systems

Location: Remote

Experience:

Bachelor's degree (or equivalent experience) in Computer Science (CS), Computer Engineering (CE) or Software Engineering (SE) with 7+ years of experience
Master's degree in CS/CE/SE with 5+ years of experience
PhD degree with thesis and top-tier publications in ML Systems, GPU architecture, or high-performance computing

Qualification:

Bachelor's degree in Computer Science (CS), Computer Engineering (CE) or Software Engineering (SE)
Master's degree in CS/CE/SE
PhD degree in ML Systems, GPU architecture, or high-performance computing

Role Focus: • Contribute features to vLLM • Profile and optimize the inference framework (vLLM) • Develop, optimize, and benchmark GPU kernels • Build and extend high-level DSLs and compiler infrastructure • Define and build inference benchmarking methodologies and tools • Architect the scheduling and orchestration of containerized large-scale inference deployments on GPU clusters across clouds • Conduct and publish original research

Nice to have:

Experience building and optimizing LLM inference engines (e.g., vLLM, SGLang)
Hands-on work with ML compilers and DSLs (e.g., Triton, TorchDynamo/Inductor, MLIR/LLVM, XLA), GPU libraries (e.g., CUTLASS) and features (e.g., CUDA Graph, Tensor Cores)
Experience contributing to containerization/virtualization technologies
Experience with cloud platforms (AWS/GCP/Azure), infrastructure as code, CI/CD, and production observability
Contributions to open-source projects and/or publications

More skills:

Go, Rust, Linux namespaces, cgroups

Other:

Base salary determined based on location, experience, and pay of employees in similar positions
For Poland: The base salary range is 292,500 PLN - 507,000 PLN for Level 4, and 375,000 PLN - 650,000 PLN for Level 5

Senior Software Engineer, AI Inference Systems

Skills Required

Description

Experience:

Qualification:

Nice to have:

More skills:

Other:

Prepare for this role

More vLLM jobs