AI Engineer (Remote)

AI Engineer
Technology
Remote
Travel:
Accuity partners with hospitals and health systems through a technology-enabled, physician-led model that improves clinical documentation integrity, coding accuracy, reimbursement optimization, and quality outcomes.
The AI Engineer develops and operates the AI systems that support automation across Accuity’s clinical documentation and revenue cycle workflows. Reporting to the Vice President, Data Science and AI, this role builds production services using large language models, intelligent agents, retrieval systems, knowledge graphs, and traditional machine learning.
The AI Engineer owns the technical systems that allow applications to use AI reliably and securely. This includes model integration and adaptation, agent orchestration, retrieval-augmented generation, vector and graph-based knowledge systems, Model Context Protocol services, evaluation pipelines, deployment automation, observability, and production support.
Depending on the use case, this may involve prompt and context engineering, retrieval, fine-tuning, tool integration, classical machine learning, or a combination of these methods.
The AI Engineer partners with Data Science to move validated approaches into production, Platform Management to use approved infrastructure and delivery pipelines, Data Architecture to access governed data, and application engineering teams to deliver AI capabilities through stable and reusable services.
Success in this role is measured by the reliability, security, maintainability, and measurable workflow impact of the AI systems delivered across the Accuity ecosystem.
AI and Agent Engineering
Build and maintain production systems using large language models, machine learning models, and intelligent agents.
Develop agent workflows that use tools, retrieval, structured outputs, state, and human approval controls.
Integrate AI systems with internal applications, data sources, APIs, and approved external services.
Build reusable AI services with clear, stable interfaces for application teams.
Develop retrieval-augmented generation systems using approved enterprise content and data.
Build and maintain document ingestion, embedding, indexing, search, reranking, and citation pipelines.
Implement vector databases, hybrid search, knowledge graphs, and graph-based retrieval where appropriate.
Maintain data provenance, access controls, and processes for keeping indexed information current.
Measure retrieval quality and correct missing, outdated, duplicated, or irrelevant results.
Connect agents to approved applications, databases, APIs, and business systems.
Develop reusable integrations that can support multiple AI applications.
AI and Machine Learning Operations
Deploy and operate LLMs, agents, retrieval services, and machine learning models in production.
Maintain automated pipelines for testing, versioning, deployment, promotion, retraining, and rollback.
Track versions and lineage across models, prompts, tools, datasets, indexes, and configurations.
Implement monitoring for quality, reliability, latency, usage, and cost.
Evaluation, Quality and Security
Build automated evaluation and regression testing for models, prompts, agents, retrieval systems, and tools.
Work with Data Science to convert evaluation methods and production thresholds into release controls.
Test systems against representative, unusual, adversarial, and high-risk scenarios.
Implement safeguards against prompt injection, data leakage, unauthorized tool use, and unsafe outputs.
Ensure AI systems comply with HIPAA, HITRUST, PHI protection requirements, and Accuity policy.
Preserve human review, decision traceability, and audit logging where clinical or coding judgment is involved.
Partner with Data Science to move validated AI approaches into production.
Partner with Platform Management and Data Architecture on infrastructure, deployment, data access, and governance.
Partner with application teams to deliver AI capabilities through reliable and documented services.
Communicate system behavior, limitations, risks, and tradeoffs clearly to technical, clinical, and business audiences.
Perform additional responsibilities as needed to support data science, technology, and organizational goals.
Master’s degree in computer science, artificial intelligence, machine learning, or a related technical field, or equivalent practical experience, required.
PhD degree in computer science, artificial intelligence, machine learning, or a related field preferred.
5+ years of experience in software, AI/ML, or platform engineering, including ownership of production systems.
Hands-on experience developing and operating LLM applications, AI agents, and retrieval-augmented generation (RAG) solutions.
Strong Python proficiency, including experience with AI/ML libraries, vector search, tool integration, and human-in-the-loop workflows.
Experience deploying and maintaining AI/ML systems using automated testing, evaluation, CI/CD, monitoring, and production controls.
Experience implementing AI observability, explainability, security, and governance practices.
Experience with Azure AI Foundry, Azure ML, Databricks, vector databases, or comparable AI platforms.
Healthcare data experience, particularly clinical coding, CDI, claims, or revenue cycle; AI Systems Engineering:
Understands how models, agents, tools, retrieval, knowledge systems, and application services work together in production.
Language Model Engineering:
Selects, adapts, evaluates, deploys, and maintains language models based on measurable requirements.
Treats deployment, monitoring, support, documentation, and recovery as part of engineering the system.
Uses repeatable evidence to determine whether changes improve or degrade system performance.
Treats data protection, access control, traceability, and approved routing as core architecture requirements.
Builds shared services and patterns that reduce duplicate work across AI initiatives.
Explains system behavior, limitations, risk, and tradeoffs clearly.
Works effectively across data science, platform, data architecture, application engineering, clinical, and external partner teams.
Evaluates new AI methods and tools without introducing them before their value, reliability, and risks are understood.
Remote Work Effectiveness:
Communicates proactively, documents decisions, and maintains visibility into work in a fully remote organization.
Employees must be able to communicate via phone, email, etc. Physical effort and exposure to physical risk are limited to that of an office role / environment.