Senior Software Engineer (AI Inference & Runtime Platform)

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer (AI Inference & Runtime Platform) based in the United States.

Join a mission-driven engineering team building advanced AI infrastructure to address complex challenges across critical industries, including energy, sustainability, logistics, and climate technology. In this role, you will own the runtime foundations that power AI workloads, from high-performance model inference to secure execution environments for AI-generated code. You will design and operate scalable systems across Kubernetes clusters, GPU infrastructure, and cloud platforms. Working across Rust, Python, distributed systems, and infrastructure automation, you will help deliver reliable, cost-efficient, and secure AI services. This opportunity is ideal for an engineer who has built and operated production-grade inference or serverless GPU platforms and enjoys solving challenging systems problems. You will also contribute to open-source software and help establish engineering standards as the platform evolves.
\You will own the core infrastructure that enables AI models and agent-generated workloads to run securely, efficiently, and reliably in production. Manage the model-serving infrastructure for open-weight large language models, including deployment, configuration, cold-start optimization, model-specific service-level objectives (SLOs), upgrades, and canary releases.
Own the stateful data plane end to end, ensuring reliable operation, documented recovery procedures, and regularly tested restoration processes.
Build and maintain FastAPI control-plane services, infrastructure-as-code configurations using Terraform/OpenTofu and Bicep, and dashboards that support effective operations.
Provide the infrastructure foundation used by backend engineering teams, collaborating across services and investigating issues that span the platform and application layers.
Establish and maintain observability, performance, reliability, and operational practices that help teams diagnose problems and make informed decisions.
At least 5 years of experience delivering production systems using a systems programming language. Strong Rust expertise is valuable, while deep experience in Go, C/C++, or Zig combined with a genuine willingness to work in Rust is also relevant.
Practical knowledge of secure AI-agent execution, including least-privilege access, preventing credentials from entering sandbox environments, maintaining audit trails, and requiring human approval for actions that modify systems of record.
Experience deploying or operating open-weight LLM inference infrastructure, such as vLLM, SGLang, or comparable serving systems, and packaging models into reliable, metered production endpoints.
Strong distributed-systems performance skills, including measurement-driven optimization, latency analysis, and the ability to demonstrate improvements with concrete metrics.
Practical experience with technologies such as Rust and Tokio, Python and FastAPI, Kubernetes operators, controller-runtime, Kubebuilder, CRDs, KEDA, Karpenter, GPU device plugins, or Dynamic Resource Allocation (DRA), along with a willingness to learn unfamiliar components.
Comfort operating across AWS, Azure, GCP, and managed GPU cloud environments, using infrastructure-as-code tools such as Terraform/OpenTofu and Bicep, along with observability tooling such as OpenTelemetry.
A bachelors degree is required; Ability to travel twice a year for company summits.
Current authorization to work full-time in the United States. Willingness to complete a written take-home assignment followed by a live, two-hour technical session with the engineering team.
Equity:

Equity participation in an early-stage company.
Bonus:

Eligibility for performance-related bonus compensation.
Healthcare:

Health insurance with meaningful coverage for dependents.
Flexible time off:

Paid time off designed to support flexibility and work-life balance.
Remote work:

Fully remote working culture, with a cluster of teammates based in Seattle.
The opportunity to contribute to AI transformation across critical industries through impactful applications in energy, sustainability, climate, logistics, and related fields.
The opportunity to shape foundational AI infrastructure, work across the stack, and contribute to open-source projects.
We use an

AI-powered matching process

to ensure your application is reviewed quickly, objectively, and fairly against the roles core requirements. Data Privacy Notice:

By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR).