Security & Model IP

Submitted models are only ever accessed by automated evaluation runners. No human reviewer reads, inspects, or has access to model weights at any stage of the pipeline. The runner pulls your artefact from encrypted storage, executes it against the benchmark suite, and posts signed results back — then discards the in-memory copy.

Private submissions are strictly tenant-isolated. No other user or organisation can see your submission metadata, scores, or artefacts. Role-based access controls are enforced at the API layer; there is no administrative backdoor that exposes private data to other tenants.

Enterprise customers can request single-tenant execution environments, where their submissions run on dedicated infrastructure not shared with any other organisation. Contact [email protected] to arrange this.

All model artefacts and submission metadata are encrypted at rest using AES-256. Storage is provided by Cloudflare R2 with server-side encryption enabled by default.

All data in transit between your client, our API, and our storage layer is protected by TLS 1.3. Older protocol versions are not accepted.

Webhook payloads carrying evaluation results are signed using HMAC-SHA256. You can verify the signature against the secret key issued to your account to confirm that a result came from Comparotor and was not tampered with in transit.

Docker submissions run inside network-isolated containers with outbound internet access blocked for the duration of evaluation. This prevents a submitted model from reaching external endpoints during inference — protecting both the integrity of the benchmark and the confidentiality of the private test set.

ONNX submissions run inside a hermetic inference process with no filesystem access outside the sandboxed working directory. No environment variables, secrets, or adjacent-tenant data are visible to the model runtime.

The evaluation environment is ephemeral. Every resource — container, sandbox process, in-memory state — is destroyed after each run completes. Nothing persists between evaluations.

Submitted model weights are never used to train any Comparotor model or shared with any third party. The weights exist in our system solely to produce benchmark scores.

Private submission scores are never disclosed to any party other than the submitter without explicit written consent. Scores do not appear on the public leaderboard, in aggregate statistics, or in any Comparotor marketing material.

For public submissions, only the benchmark scores and associated metadata are published. The model artefacts themselves remain in private storage and are never exposed to other users or made available for download.

Enterprise customers can request deletion of their artefacts before the standard 24-month retention window closes. See the Post-contract termination section for details of what is deleted and when.

When a paid contract ends, all private submission artefacts associated with the account are deleted within 30 days of the contract end date. You will receive email confirmation when deletion is complete.

Public leaderboard entries — which contain only benchmark scores, no artefact data — may remain on the leaderboard after contract termination. If you want a public leaderboard entry removed, contact [email protected] and we will remove it.

If you discover a security vulnerability in Comparotor, please report it to [email protected]. We will acknowledge your report within 2 business days and keep you updated as we investigate and remediate.

We ask that you give us a reasonable opportunity to fix the issue before public disclosure. We do not pursue legal action against researchers who report vulnerabilities in good faith.

By default, submission artefacts are stored in Cloudflare R2 with data centres in the United States. Enterprise customers can request EU-only data residency, in which case artefacts are stored exclusively in Cloudflare R2 EU buckets.

Our data handling practices are designed to comply with GDPR requirements for personal data. See our Privacy Policy for details on how personal data is handled.