Security review rather than capability killed the shortlists, and it consumed four to eight weeks wherever the data questions were left to a vendor's standard answers
A contract repository holds the price of every deal and the cap on every liability. Handing it to an AI platform is a security decision before it is a product one.
Prepared by Redress Compliance · August 18, 2026 · AI procurement evaluations supported, 2024 to 2025.
Executive summary
Security review, not capability, killed shortlists. It consumed four to eight weeks wherever the data questions were left to a vendor's standard answers.
Buyers who put five questions in writing in week one roughly halved that and disqualified the weak options early rather than after a selection had been made.
The training use question separated vendors fastest. A contractual no cleared review. A policy statement that could change with a product update did not.
Local processing of sensitive usage exports satisfied the strictest security teams. Files that never leave the browser never enter the risk register.
Why is this a first order concern?
Because the data is uniquely sensitive and uniquely concentrated. A single repository holds your negotiated prices, your protections and your weak points, which is the dataset a competitor or an auditor would most like to read.
The value of applying a model to that data is real and so is the exposure. The two have to be weighed together rather than sequentially, which is what putting the questions in the demo actually achieves.
Security review is the long pole in procurement
Which means asking late costs weeks rather than costing a conversation. The vendors who answered crisply in the demo were the same ones who cleared procurement review quickly afterwards.
Which five questions decide it?
Five, and a vendor that hedges on any of them is telling you something worth hearing. Each has an answer that reads as safe and a warning sign that reads as movement.
| Question | What a safe answer sounds like | Warning sign |
|---|---|---|
| Where is our data stored? | Encrypted, in a named jurisdiction, under your control | Vague or shifting residency |
| Does our data train shared models? | A contractual no | A policy that can change |
| How is pooled benchmark data anonymized? | A stated anonymity floor, with no cohort traceable | No stated anonymity standard |
| Can sensitive files be processed locally? | In the browser, never uploaded | Everything must go to the server |
| What are the access and deletion terms? | Audit logs and a clean exit | No logs and unclear deletion |
Contributed benchmark data should carry an anonymity floor so that no cohort traces back to a company. That is a design property rather than a promise, and it can be described precisely by a vendor that has it.
The GenAI contracting red lines
The clauses that decide whether an AI agreement is safe to sign, and the language that fixes them.
Get the brief →What the security reviews showed
Across the AI procurement evaluations supported in 2024 and 2025, security review rather than capability killed shortlists. Three patterns recur.
- The training use question separated vendors fastest: a contractual no cleared review, and a policy statement that could change did not.
- Local browser processing of sensitive usage exports was the feature that satisfied the strictest security teams.
- The vendors who answered crisply in the demo were the same ones who cleared procurement review quickly later.
Ask whether your contracts train shared models, and require the answer in the contract rather than on the policy page. A policy changes with a product update.
- Your agreements decoded into plain English before the auditor interprets them for you
- Coverage grid: liability caps, intellectual property protections and service levels checked in one pass
- A defensible position paper generated in minutes rather than weeks
Why is the training answer the fastest filter?
Because it has only two forms and they are easy to tell apart. A contractual commitment survives a product update and a policy statement does not, and every vendor knows which one it is offering.
It also tests whether the vendor can move at all. A supplier that will put the commitment in the agreement has an approval path for contract change, which is the thing every later negotiation depends on.
The related trap is retention
A no training clause paired with indefinite retention leaves the data in the room regardless, which is worked through in the AI data governance guide.
Watch the briefing · 3:50Signing the Enterprise AgreementWhat the agreement has to cover: data and training terms, deprecation, and capacity.
What do the governance frameworks add?
Vocabulary. They let you press a vendor precisely rather than vaguely, which changes the answers you get because a specific question is harder to deflect than a general one.
Vocabulary, not a substitute
The published frameworks are where that vocabulary comes from: the AI risk management framework and the information security standard between them cover most of what a security team will ask.
They do not replace the five questions. They give each one a defensible form, so the security team is reviewing a stated position rather than an impression left by a demonstration.
The shadow adoption problem sits alongside this and is measured separately in the AI governance playbook, which covers the tools that never reached a review at all. The commercial half of the same agreement is worked through in the Anthropic negotiation guide.
What the evaluations measured, 2024 to 2025
Two cuts of the engagement file, both about timing rather than technology.
Wherever the data questions were left to a vendor's standard answers rather than asked directly in week one.
Buyers who put the five questions in writing at the start disqualified weak options early and shortened the review.
The saving is elapsed time rather than money, which is the scarcer resource in every evaluation that has a deadline attached to it.
Your first five moves
- Put the five questions in writing in week one, before the demonstration rather than after the selection, because that alone roughly halved the review.
- Require the no training answer in the contract rather than the policy page, since a policy changes with a product update and a clause does not.
- Ask for a stated anonymity floor on any pooled benchmark data, so no cohort traces back to a company, and treat the absence of a standard as the answer.
- Test whether sensitive usage files can be processed locally, because browser side processing was the feature that satisfied the strictest security teams.
- Fix access logging and deletion terms before signature. The GenAI practice runs the review alongside the commercial evaluation rather than after it.
Frequently asked questions
Why does this matter more than capability?
Because a contract repository holds the price of every deal, the cap on every liability and the terms of every relationship. Security review rather than capability killed the shortlists.
How long does review take?
Four to eight weeks wherever the data questions were left to a vendor's standard answers. Buyers who put the questions in writing in week one roughly halved that.
What are the five questions?
Where the data is stored, whether it trains shared models, how pooled benchmark data is anonymized, whether sensitive files can be processed locally, and what the access and deletion terms are.
What is an acceptable training answer?
A contractual no. A policy statement can change with a product update, which is why the commitment has to sit in the agreement rather than on a web page.
Why does the training question filter fastest?
Because it has only two forms and they are easy to tell apart. It also tests whether the vendor has an approval path for contract change at all.
What does an anonymity floor mean?
That contributed benchmark data is pooled so no cohort traces back to a company. A vendor that has designed for it can describe the standard precisely rather than generally.
Why does local processing matter?
Because a file that never leaves the browser never enters the risk register. It was the single feature that satisfied the strictest security teams in these evaluations.
Is a no training clause enough on its own?
No. Paired with indefinite retention it leaves the data in the room regardless, so retention and deletion terms have to be read alongside it.
What do governance frameworks contribute?
Vocabulary. They let you press a vendor precisely rather than vaguely, and a specific question is much harder to deflect than a general one.
When should the questions be asked?
In the demonstration, not after selection. The vendors who answered crisply there were the same ones who cleared procurement review quickly afterwards.