Principal Architect - Reinforcement Learning (Physical AI)

Career Area:

Technology, Digital and Data

Job Description:
Your Work Shapes the World at Caterpillar Inc.
When you join Caterpillar, youre joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We dont just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.
Help Build the Future of Caterpillar –
At Caterpillar, technology always has a purpose, which is to solve our customers’ toughest challenges. Through Cat Technology, we are solving problems by building the intelligence layer that connects machines, data, and people to make jobsites safer, more productive, and more sustainable. By combining deep domain expertise in physical systems with software, connectivity, autonomy, and AI, we deliver solutions that work in the real world—on real jobsites, at global scale.
You’ll build and deploy against one of the most unique data foundations—over 1.6 million connected assets generating real-world data daily. These data and platform capabilities are enabling the development of AI models, edge computing architectures, and software systems that scale across fleets, products, and industries. The result will be a new generation of machines that continuously learn, improve, and deliver performance at scale.
Be Part of What’s Next in Autonomous Construction Sites
Construction autonomy is one of the most complex challenges in applied AI, and at Caterpillar, advancements in physical AI, simulation, sensing, and edge computing are turning things that once felt impossible—intelligent machines operating in dynamic jobsites—into reality.
Our connected ecosystem brings together massive volumes of high-quality data to create a foundation where engineers like you can build and deploy against.
In these roles, you’ll work at the intersection of the physical and digital worlds. You’ll help design and deliver intelligent systems that enable machines to perceive their environment, make informed decisions, and support safer, more productive operations.
Apply today to build the new era of construction autonomy at Caterpillar.
The Learning Systems Architect owns how the machine learns — the end-to-end system that turns simulation, digital twins, and fleet data into high-performance, field-proven policies and models.
The

Principal Architect – Reinforcement Learning (RL) Systems

is responsible for the architecture of the learning system that produces Caterpillar’s Physical AI models: the reinforcement learning, imitation learning, and foundation-model post-training pipelines that train perception, prediction, planning, and control policies for excavators, dozers, loaders, and haul trucks. Where the Simulation Engine architect owns the runtime the policy trains in, the Digital Twin architect owns the fidelity of what it trains against, and the AI Factory architect owns the compute and data platform it trains on, this role owns the learning problem itself: how tasks are formulated as observations, actions, rewards, and curricula; which policy and model architectures are used; how large-scale closed-loop training in simulation is structured; and how learned policies are evaluated, made safe, and deployed onto machines.
The architect defines the reference architecture for learning-based autonomy at Caterpillar: Gym-style task and environment specifications on top of Isaac Lab / Isaac Sim, Omniverse, and hybrid neural simulation (NVIDIA Cosmos-class world models); massively parallel GPU-based RL (PPO, SAC, GRPO and successors) and imitation learning (behavior cloning, distillation, offline RL) at fleet-data scale; vision-language-action and foundation-model post-training with RL; reward and metric design tied to productivity, safety, machine health, and operator-quality outcomes; and the policy evaluation, regression, and safety-case methodology that gates deployment. The role sets standards for reproducibility, experiment rigor, and benchmark hygiene, and partners with the Experimentation & Operations architect to run training and evaluation at scale.
This role that turns research into repeatable engineering, and is measured by field task performance of deployed policies — not by simulation benchmarks alone.
Developing detailed architecture deliverables to solve business problems.

~ Designing an applications technical infrastructure, such as specific databases, programming languages, utilities, and testing approaches.

~ Leading the evaluation and deployment of new technologies to add or enhance existing digital technical capabilities.

~ Participating in addressing business requirements for applications and collaborating with cross-functional teams to deliver digital solutions that meets business results.

Requirements Analysis: Knowledge of tools, methods, and techniques of requirement analysis; ability to elicit, analyze and record required business functionality and non-functionality requirements to ensure the success of a system or software development project.

~ Knowledge of techniques and tools that promote effective analysis; Platform Architecture: Knowledge of technologies and methods to design processing mechanisms and roadmaps to execute business application systems; Target Architecture: Knowledge of target architecture; ability to develop the IT blueprint and roadmap while aligning the architecture and processes with business strategies and objectives.

~ ability to effectively transmit, receive, and accurately interpret ideas, information, and needs through the application of appropriate communication behaviors.

Typically 8+ years (or PhD plus 5+) in robot learning with RL taken from algorithm to deployment on real hardware: PPO, SAC, GRPO, model-based / world-model RL, reward shaping, curriculum, and compounding-error handling in closed-loop training.

~ Imitation Learning and Foundation-Model Post-Training – Behavior cloning, offline RL (IQL, GAIL), distillation, diffusion policies; vision-language-action or robot foundation models with RL-based post-training (RLHF / DPO / GRPO).

~ Large-Scale Closed-Loop Training in Simulation – Architected RL frameworks on GPU-parallel simulators (NVIDIA Isaac Lab / Isaac Sim, MuJoCo, Brax) with Ray, Kubernetes, or Slurm; used neural / hybrid simulation (NVIDIA Cosmos, neural reconstruction) for training or augmentation.

~ Sim-to-Real Transfer – Closed sim-to-real gaps via domain randomization, system identification, actuator and latency modeling, and real-to-sim reconstruction, measured by field task success; hydraulic or heavy-machinery dynamics highly valuable.

~ Policy Evaluation and Safety – Metrics for learned policies (safety, progress, productivity, comfort), statistically rigorous policy comparison, continuous evaluation and regression, adversarial / long-tail scenario testing, runtime safeguards; Controls and Robotics Foundations – MPC, impedance control, trajectory optimization; perception-to-action pipelines under real-time constraints.

~ Data Flywheel Literacy – Shapes demonstration and fleet data collection, auto-labeling, and synthetic data strategy for imitation and offline RL.

~ Software Engineering – Strong Python and PyTorch (JAX a plus), working C++ for real-time integration, CUDA familiarity, experiment tracking, Bazel or monorepo tooling, CI/CD.

~ Architecture Leadership – Turns research into repeatable engineering; mentors ML engineers and researchers; led a shipped learning-based capability.

~ Domain Exposure – Off-highway, construction, or mining autonomy (Bedrock Robotics, Built Robotics, Applied Intuition), or robot learning at NVIDIA (Isaac / GEAR), Waabi, Wayve, Zoox, Figure, 1X, Agility, Boston Dynamics, or Tesla Optimus.

~ Master’s or doctoral degree in Computer Science, Robotics, Machine Learning, Controls, or related field preferred; This position requires the candidate to work full-time at the Irving, TX office (Dallas)
Domestic relocation assistance is available for this position.
Visa sponsorship is available with this position.
Youll work on some of the most challenging applications of AI in the physical world, helping build systems that transform vast amounts of machine and sensor data into the intelligence that powers autonomous and AI-enabled equipment. If youre excited about applying AI to real-world machines operating in complex environments, wed like to hear from you.
147,760.00 - $240,110.00 Compensation and benefits offered may vary depending on multiple individualized factors, job level, market location, job-related knowledge, skills, individual performance and experience. Medical, dental, and vision benefits*
~ Paid time off plan (Vacation, Holidays, Volunteer, etc.)*
~Health Savings Account (HSA)*
~ Flexible Spending Accounts (FSAs)*
~ Employee Assistance Program*
~ Incentive bonus*
~ Disability benefits
~ Life Insurance
~ Tuition Reimbursement

* These benefits also apply to part-time employees
Relocation is available for this position.Visa sponsorship is available for eligible applicants.

Caterpillar is an Equal Opportunity Employer, Including Veterans and Individuals with Disabilities. Join our Talent Community.