AI Engineer (Remote)

Insight Softmax is a leading organization in the fields of machine learning, data science, and high-performance computing (HPC). We are dedicated to providing innovative solutions in computing technology to solve complex problems in various sectors including research, science, manufacturing, finance, and data analytics.
We are hiring an AI Engineer for our engineering team. You will implement the AI platforms and services our architects design and our customers run: inference stacks, AI gateways, RAG and agent systems, and the automation around them.
You will work directly inside customer environments, from neoclouds with thousands of GPUs to enterprises deploying their first production LLM service, and on our own platform product work for vendor partners.
You will spend much of your time embedded with a customer team, shipping working systems and leaving behind documentation and automation they can run.
Deploy and operate LLM inference services on Kubernetes and bare metal GPU clusters, including model serving, autoscaling, GPU scheduling and performance tuning.
Build and operate AI gateway and routing layers: tenant keys, model pools, quotas, rate limits, caching, failover and usage metrics.
Implement RAG pipelines, agentic workflows, evaluation harnesses and the APIs that expose them to customer applications.
Write production-quality services and automation in Python, with infrastructure as code for the surrounding platform.
Instrument AI services for observability: latency, throughput, token and cost accounting, quality signals.
Integrate AI services with customer identity, security and data systems.
Document deployments and hand over runbooks to customer and internal operations teams.
Reserve time each week for pre-sales support, internal platform work and knowledge sharing.
At least 4 years of software or platform engineering experience, with at least 2 years building and running LLM or ML services in production.
Strong Python. Comfortable building services, CLIs and automation, with tests.
Strong linux chops.
Deep knowledge of at least one of AWS, GCP, or Azure. Preferably AWS.
Experience with LLM application patterns: prompt and context management, retrieval, tool use, structured output, evaluation.
Able to work independently inside a customer environment and communicate progress clearly.
AI gateway tooling (for example LiteLLM, Envoy or Traefik based gateways, Kubernetes

Gateway API inference extensions).
Vector databases and embedding pipelines.
Multi-tenant GPU platforms, GPU partitioning and Dynamic Resource Allocation.
Observability stacks (OpenTelemetry, Prometheus, Grafana, Datadog) applied to AI services.
Spectro Cloud Palette and PaletteAI, Run:ai, or similar GPU platform products.
Bonus Experience and Skills:

Air-gapped or regulated environment deployments.
Using linux as your primary desktop workstation environment.
Full time position
~ Remote work environment
~Some customer meetings will occur each week at non-standard / nighttime hours, as we

currently support a global team across 3-4 continents.
~ Some customer engagements include on-site work at US customer sites, typically for

workshops, platform bring-up and go-live. Expect occasional US travel.

AI systems may be used with the recruitment and screening processes for this job requisition.