35 Questions to Ask an AI Email Vendor Before You Buy

These are the questions to ask an AI email vendor out loud, in a live conversation, while someone from their side is on the line and can be pinned to an answer.

A scorecard is what you test yourself in a trial. A security questionnaire is a document their compliance team fills in at its own pace. A sales call is the one setting where you hear what a vendor says before anyone edits it.

So every question below carries two things a bare list leaves out: why it matters, and what the weak answer sounds like. The weak answers are the useful column. We build AI Emaily, and eight of these are ones we answer badly.

The short answer#

Ask across seven groups: data handling and training, autonomy and safety, accuracy and failure modes, provider coverage, team and admin, commercial terms, and continuity and exit.

Then grade the shape of the reply rather than its content. Vendors rarely lie outright. They answer an adjacent, friendlier question, and the buyer nods because something was said.

One weak answer is a bad day. Two inside the same group is a pattern, and the pattern is what you are buying.

How to run this without sounding like an auditor#

You are not reading thirty-five questions down a phone line. Pick twelve and leave room to push on the ones that wobble.

1. 1

   Ask the default, not the capability

   Every product can be configured safely. Ask what ships switched on, and what happens if nobody opens the settings page.

2. 2

   Follow every claim with show me

   In the product, on this call, on a mailbox with something awkward in it. A live demonstration is worth ten feature-page bullets.

3. 3

   Send the written follow-up within a day

   Restate their answers in your own words and ask them to correct anything wrong. Privacy pages get edited; a saved reply does not.

4. 4

   Score the shape, not the warmth

   The friendliest call in a process is often the one that leaves you with the least in writing.

Group 1: Data handling and training#

This group decides whether the rest of the evaluation is allowed to happen. The recurring evasion is answering a narrower question than the one asked, so watch questions 2 and 3.

Group 2: Autonomy and safety#

Everything here is about what happens when you are not looking. Anchor each question to the shipped default: a configurable product and a safe product are different claims.

Group 3: Accuracy and failure modes#

Demos run on threads that work. This group is you asking, politely, to see the ones that do not.

Group 4: Provider coverage#

Coverage is the cheapest disqualifier in the process and the one buyers check last. Ask these with your least standard mailbox in mind.

Group 5: Team and admin#

Skip this group if you are buying one seat. Weight it above everything else if a security team has to approve the purchase. A control that is per user and optional is not a control.

Group 6: Commercial terms#

Four questions about the shape of the bill rather than its size. Prices move constantly here, so verify any number on the vendor's own pricing page at signature time.

Metering by how much mail arrives falls hardest on the busiest inboxes, which are the people buying triage. At least one vendor here does exactly that.

Group 7: Continuity and exit#

This group stopped being theoretical. Notion Mail announced its shutdown in June 2026, the inbox retires on 22 September 2026, and unexported drafts, snippets and auto-label rules are deleted the day before.

Nobody who bought it asked question 33 and got a bad answer. They never asked it.

The four shapes of a weak answer#

Nobody gives thirty-five clean answers, us included. What matters is telling the gap that is fine from the gap that ends it.

• Not yet. An honest roadmap gap. Ask which release, then check the last three things they shipped against the dates they gave.
• I do not know. Usually the wrong person in the room. Ask who does know and for a written answer by a date, then see whether it arrives.
• The reframe. You asked about training and got an answer about selling data. Repeat the original question, unchanged, and let the silence work.
• I will not say. The only fatal shape. A vendor who will not put retention, training, sub-processors or a notice period in writing has answered all four.

When several weak answers land at once#

Cluster them by group before reacting. One wobble anywhere is noise; two inside the same group is the group telling you something.

Weak on data handling plus weak on autonomy should end the evaluation. A system that acts on its own and will not describe what it does with what it read is a problem no later setting fixes.

Weak on commercial terms plus weak on continuity is usually an unfinished contract. Ask for the notice period, renewal cap and overage behaviour in the order form. Agreement repairs two rows in an afternoon; refusal answers question 33 properly.

Turn the list on us: where AI Emaily answers badly#

A question bank whose author passes it cleanly is a brochure. We build AI Emaily, so here are the eight of these thirty-five where our own answer is weak, checked July 2026.

The other twenty-seven we answer well. Autonomy is an explicit per-account setting, Manual, Copilot or Autopilot, with Copilot as the default. Sends stay approval-gated, every autonomous action lands in an audit log with its reasoning, and the destructive ones undo. We do not train on your mail and hold zero-retention arrangements with model providers.

On question 16, the drafting voice comes from a Personal Context you write plus client profiles you set, never inferred from your past messages, so you can point at the line that produced a sentence. On 18 and 19, Gmail, Outlook, Microsoft 365, iCloud, Fastmail, Proton and standard IMAP connect in one view with no migration.

Three questions belong to somebody else. If a security team gates on 6, 24 and 27, an assistant inside Google Workspace or Microsoft 365 beats us outright: your IT already runs the identity and audit controls, and both now bundle AI. On 34, any client that hands you MBOX or EML beats our JSON.

The follow-up email is the deliverable#

The call produces notes nobody trusts in six months. The email produces a record, and asking a vendor to confirm their own words is itself the last question in the bank.

Either it comes back corrected, which is a good vendor doing its job and leaves you with a document. Or it never comes back, which answers question 31 for you.