ML Cloud Infrastructure Engineer

ABOUT US: Havoc is a leader in all-domain collaborative autonomy. Its software-defined hardware approach powers military and commercial-grade autonomous systems across sea, air, and land to sense, decide, and act together in complex and contested environments. Havoc connects assets, enabling them to share information, adapt in real time, and continue operating even when communications are disrupted or denied. Havoc optimizes mission performance and minimizes human risk. Havoc was founded in 2024 and headquartered in Providence, Rhode Island. Learn more at Havoc: All-Domain Collaborative Autonomy http://havocai.com/ . ABOUT THE ROLE As a Machine Learning Cloud Infrastructure Engineer, you will build and operate the infrastructure HavocAI teams use to train, evaluate, deploy, and monitor machine learning models safely and reliably. You will develop the pipelines, services, platforms, and integrations connecting data lakes, telemetry stores, simulation environments, training workloads, cloud compute, and deployed models. Your work will enable autonomy, data, and software engineers to move efficiently from raw field and simulation data to reproducible datasets, scalable training, rigorous evaluation, and production deployment. This role is ideal for a strong software, infrastructure, or data engineer with hands-on ML experience who enjoys building production systems at the intersection of data, models, compute, and autonomy. You should be comfortable operating in a fast-paced environment, solving ambiguous infrastructure problems, and building systems that must remain scalable, observable, secure, and reliable. WHAT YOU’LL DO ML INFRASTRUCTURE & PIPELINES - Build pipelines that transform raw multi-modal data—including telemetry, imagery, video, sensor, and simulation data—into curated, versioned training datasets. - Develop reproducible training and evaluation workflows that scale across cloud compute and GPU resources. - Build and maintain model deployment infrastructure for packaging, serving, inference, versioning, and rollback. - Implement experiment tracking, dataset lineage, model versioning, and other capabilities required for reproducible ML development. - Own data schema versioning and migration across pipelines, data lakes, and services as datasets and models evolve. CLOUD PLATFORM & INFRASTRUCTURE - Design, build, and operate scalable AWS infrastructure using Infrastructure as Code. - Build and maintain Kubernetes/EKS workloads and containerized environments for training, batch processing, evaluation, and model serving. - Develop self-service tooling and paved paths for compute scheduling, storage, data access, training, and deployment. - Improve utilization, scalability, and cost efficiency across cloud and accelerator infrastructure. - Build infrastructure that enables engineering teams to launch workloads safely without unnecessary operational overhead. RELIABILITY, EVALUATION & OBSERVABILITY - Build evaluation frameworks and regression testing for model quality, dataset integrity, and pipeline correctness. - Establish quality and reliability signals that help determine whether models are ready for production use. - Develop monitoring, logging, tracing, and observability across training jobs, data pipelines, and deployed models. - Diagnose and resolve performance, scaling, reliability, and infrastructure bottlenecks. - Maintain high standards for automation, testing, documentation, and operational readiness. CROSS-FUNCTIONAL ENGINEERING - Partner with Autonomy, Software, Data, Simulation, and Security teams to build ML infrastructure spanning edge data capture through cloud training and model deployment. - Contribute to CI/CD and release processes for models, datasets, and ML pipelines. - Translate engineering requirements into scalable platform capabilities that can support multiple teams and use cases. - Incorporate feedback from engineers and users to continuously improve ML development workflows. SECURITY & DATA MANAGEMENT - Implement secure infrastructure practices, including IAM least privilege, secrets management, access controls, and secure handling of sensitive and defense-related data. - Build data and ML workflows with reproducibility, traceability, and appropriate controls from the start. - Partner with security and infrastructure teams to ensure ML systems meet applicable operational and compliance requirements. WHAT WE’RE LOOKING FOR - 3+ years of experience in software engineering, infrastructure engineering, data engineering, ML infrastructure, or a related field. - Strong programming experience in Python, with experience in Go, C++, or another systems-oriented language preferred. - Experience building and operating production services, APIs, data pipelines, developer platforms, or infrastructure. - Hands-on experience with ML workflows such as dataset preparation, model training, evaluation, or deployment. - Experience with cloud infrastructure, preferably AWS, and Infrastructure as Code. - Hands-on experience with Kubernetes and containerized environments. - Strong understanding of production engineering fundamentals, including reliability, observability, testing, automation, and maintainability. - Ability to work effectively across engineering disciplines and solve ambiguous technical problems with a high degree of ownership. - U.S. Citizenship and ability to obtain and maintain a U.S. Government security clearance. NICE TO HAVE - Experience with MLOps and workflow platforms such as MLflow, Weights & Biases, Kubeflow, Ray, Airflow, or Dagster. - Experience with GPU/accelerator scheduling, distributed training, or large-scale ML workloads. - Experience working with multi-modal datasets including imagery, video, telemetry, sensor, or simulation data. - Experience supporting autonomy, robotics, simulation, or real-time systems. - Experience deploying ML models to edge or embedded environments. - Experience with AWS GovCloud, GCP Assured Workloads, or compliance-driven environments such as FedRAMP or IL4/IL5. WHAT SUCCESS LOOKS LIKE Within your first 12 months, you will have: - Built reliable and reproducible pipelines that move data from raw capture through curated, versioned training datasets. - Enabled training and evaluation workloads to run at scale with clear quality and reliability signals before deployment. - Delivered self-service ML infrastructure that reduces friction and accelerates Autonomy, Data, and Software teams. - Improved the observability, reliability, security, and cost efficiency of HavocAI’s ML infrastructure. - Established scalable foundations that allow ML development and deployment to grow alongside HavocAI’s autonomy capabilities. BENEFITS: - 100% Employer paid Health, Dental and Vision Insurance for you and your families - Life Insurance (Employer Paid) - Ability to participate in the companies 401k program (Matching) - Unlimited PTO policy with an enforced 2 week minimum - Equity Package - Work / Home Office Stipend - Global Entry - 16 Week Paid Parental Leave - Monthly Health and Wellness Stipend OUR VALUES: - Innovation: We are driven to break new ground. Every day presents an opportunity to challenge the status quo, think boldly, and deliver advanced solutions that transform the future of defense technology. - Integrity: We hold ourselves to the highest ethical standards, ensuring transparency, accountability, and trust in all our actions and partnerships. - Mission-Driven: We are focused on achieving impactful outcomes that align with our core mission—protecting lives through innovation. - Forward-Leaning: We continuously seek out new opportunities and remain at the forefront of technological advancements. We embrace change and anticipate the challenges of tomorrow with confidence and creativity. - Ownership of All Tasks: At HavocAI, no problem is too complex or too trivial. We believe that greatness comes from tackling the hardest challenges, but also in handling the smallest, sometimes thankless, tasks with the same level of commitment and care. - Servant Leadership: We lead by serving others, whether it’s supporting our employees, partners, or the broader community. Empowering those around us is key to achieving long-term success and making a lasting impact. HavocAI is an Equal Opportunity Employer and is committed to creating an inclusive and diverse workplace. We welcome applicants from all backgrounds and do not discriminate based on race, color, religion, gender, sexual orientation, age, national origin, disability, veteran status, or any other legally protected status.