§procurement

For OEM & Tier-1 Procurement

Compare surrogate model vendors before procurement.

The evaluation problem

Physics-ML vendors make bold accuracy claims. Comparing them across different internal benchmarks and proprietary datasets is impossible. Comparotor gives procurement teams a single, neutral evaluation framework: the same dataset, the same SU2 reference, the same scoring formula for every vendor.

Why Comparotor for procurement

Vendor comparison

Evaluate multiple vendors on the same sealed test set. Private multi-vendor comparisons: no vendor sees another vendor's results or submission.

Procurement-ready reports

Signed PDF reports include dataset version, scoring breakdown, OOD performance, and a procurement interpretation section. Designed to survive engineering governance review.

Contamination resistance

Quarterly test-set rotation prevents vendors from overfitting to the benchmark. Evaluation results remain meaningful across procurement cycles.

Replay audit

Every evaluation can be replayed against the archived test set. Results are verifiable 24 months post-evaluation for contract disputes or post-procurement review.

Procurement workflow

Shortlist

Identify surrogate model vendors for your domain (turbomachinery, wind, eVTOL, propulsion)

Evaluate

Ask vendors to submit to Comparotor, or use our managed evaluation service

Compare

Receive private comparison reports showing all vendors on the same scorecard

Decide

Use signed, reproducible benchmark reports as procurement evidence

Benchmark coverage

RotorBench-Aero v0.1current

✓2D airfoil aerodynamics (Cl, Cd, Cm)

✓Low-speed to low-transonic (Ma 0.05–0.5)

✓Wide Reynolds number range (Re 500k–10M)

✓OOD generalisation to supercritical, wind-blade, and eVTOL sections

coming in future versions

–3D rotor / propeller evaluation

–Turbomachinery cascade flow

–Unsteady / time-domain predictions

Frequently asked questions

Can we keep vendor submissions confidential?Yes. In multi-vendor evaluations, no vendor sees another vendor's results. Our isolation is technical, not just contractual.

Do we need vendors to have Comparotor accounts?Vendors can submit directly, or we can run a managed evaluation on their behalf (Enterprise plan).

What formats do vendors submit in?ONNX or Docker. We accept most standard ML deployment formats.

Is there a Data Processing Addendum?Yes, available on Procurement Evaluation and Custom plans.

How long are results valid?Results can be replayed for 24 months. For procurement cycles longer than 24 months, contact us for extended retention.

Discuss your evaluation

Our team can walk you through multi-vendor evaluation design, managed submission options, and Enterprise plan terms.

Contact sales to discuss your evaluation View pricing Read the methodology