[CAD] Evaluation Lead, Mechanical Design

Prometheus · CAD Working Group · Zurich THE ROLE We're hiring an Evaluation Scientist to measure how well AI systems perform mechanical design — modeling geometry with intent, making it manufacturable, and meeting its physical requirements. You'll work directly with our researchers and product teams to define what good mechanical design means for an AI system, and build the evaluations that hold models to that standard. To get there, you'll build and run a network of expert contractors who author and grade the evaluations: recruiting them, calibrating their judgment, and standing behind what they produce. WHAT YOU'LL DO Define the evaluation roadmap. Work with research and product teams to turn model capabilities and user workflows into a clear set of evaluations. Identify which dimensions of mechanical design matter, where models fail today, and what we should measure next. Own evaluation quality end to end. Build and run the pipeline that turns contractor-authored items into reliable measurements, including rubrics, gold sets, cross-review, and adjudication. Build and manage the evaluation network. Work with managed vendors to produce evaluations at scale, and recruit senior mechanical designers to set the quality bar through gold sets, calibration, and expert review. WHAT WE ARE LOOKING FOR Evaluation experience. You’ve built or operated rigorous evaluation systems before, ideally for AI or other complex technical systems. You understand how to design rubrics and gold sets, calibrate reviewers, measure agreement, diagnose noisy signals, and make sure an evaluation is actually measuring what it claims to measure. Mechanical design fluency. You understand the language, workflows, and core concepts of mechanical design well enough to work effectively with experienced practitioners, ask good questions, and judge whether an evaluation is testing something meaningful. You don’t need to be the deepest domain expert in the room. Operational ownership. You’ve run complex processes involving external vendors, contractors, or expert reviewers, and you know how to keep quality high as volume grows. You’re comfortable setting standards, monitoring performance, fixing weak points in the process, and taking responsibility for the reliability of the final output. Must have: - Very strong (maybe with industry exp) mechanical engineering background - Strong data science/analysis skills - Experience with data collection (automatic and/or from contractors) Nice to have: - Prior experience with ML training and evaluation WHY JOIN US - Collaborate with world-class researchers on groundbreaking projects. - Competitive salary, benefits, and flexible work arrangements. - A mission-driven culture where your impact is visible.