Audit Log Requirements for AI Email Tools: What to Demand

An AI email tool's audit log is the artefact you reach for on the day something goes wrong. Every other feature — drafting, triage, autonomous sends — is a bet that the tool behaves. The audit log is what lets you answer the question you will actually be asked when it doesn't: who did what, when, and on whose authority.

Most vendor audit logs were written for a person taking a manual action. The moment an agent enters the loop, a log that records "message sent" is worthless. You need to know which model wrote it, in what authority mode, against which trigger, on whose approval, and whether the send is still recoverable. This post is the written procurement requirement — the eleven fields the log must contain, how to test the log is complete during a trial, and where to draw the line before you sign.

We build AI Emaily. This is a decision-guide, not a roundup, so the placement is one section near the end that scopes where we fit and where we do not. Skip to the verdict if you already know what to demand and just want the honest picture.

The short answer#

An AI action log is useful after an incident when it captures the eleven fields below for every autonomous or assisted action. Together they let you reconstruct what happened, hold a named approver accountable, prove the action against a policy, and undo it if the tool supports undo. Missing any one field and you have a gap you will pay for the first time it matters.

The eleven are not novel — most are variants of what NIST and ISO/IEC 27001 already ask of a control log. What changes for AI is that the actor is a model, the decision has a version, and the approver may be a human or another automated policy. The log has to be honest about which.

Criteria that actually matter#

The eleven fields fall into five questions the log has to answer, and grouping them that way makes the checklist easier to defend to a security reviewer. Each question is one you will be asked verbatim after an incident.

• Who or what acted — Actor and Authority mode together. "Our AI" is not an actor; "gpt-4o-mini via the agent runner, in Copilot mode" is. If the log cannot tell a manual send from an autonomous one at the row level, the mode field is missing.
• Why it fired — Trigger and Input reference. A tool that logs a send without a pointer to the thread and message that provoked it cannot be replayed. If you can only see "agent replied," you cannot tell whether an incoming email was a legitimate customer or a crafted prompt-injection attempt.
• How it decided — Model, version and Decision. The output has to be recorded before it left, not just after. A log that only shows the delivered result cannot distinguish a good decision that was then mangled by a rendering bug from a bad decision that was faithfully executed.
• Who signed off — Approver and Timestamps. The three timestamps (proposed, approved, executed) are the ones that expose latency games — for example, a queue that approves in bulk hours after the human clicked away. Copilot mode requires a human approver id; Autopilot requires the named policy id, not the string "policy."
• What happened after — Result, Undo state and Retention. A log without an external message id cannot be correlated to a bounce or a spam complaint. An undo-state field that never says "expired" is a red flag: it means the vendor is not tracking the window, which usually means undo is not really implemented.

Scoring table: grade a vendor's log during the trial#

Score each field against three tiers. A field is present if the vendor's log shows it exists in the schema. It is complete if it is populated for every action, not only the ones the vendor demoed. It is exportable if you can pull it out in a machine-readable format on your own schedule, not via a support ticket.

You need complete on all eleven fields to run the tool in production. Exportable is table stakes on any regulated review; you cannot answer a legal-hold request via a screenshot of somebody else's admin console.

Worked example: replaying a wrong send from the log#

The test that matters is not whether the log looks tidy in a demo. It is whether you can reconstruct a specific incident from the log alone. Here is the scenario to run against a vendor during a trial.

The setup: turn on the agent in Copilot mode, connect a test mailbox, and send yourself a message that impersonates a customer asking for a quote. Approve the drafted reply. Then, in the log, answer these six questions with pointers to specific rows — no scrolling, no memory, no support call.

1. 1

   1. Which model wrote the draft, and which version was live at the time

   You should see the model identifier and version string on the proposed-decision row. If the log records only "AI drafted a reply," you cannot distinguish a regression in yesterday's release from a genuine error.

2. 2

   2. What did the model actually see

   Follow the input-reference field to the exact incoming thread. The log must reference the message id, not a subject line — subjects change and threads get archived.

3. 3

   3. What did it propose, before you touched it

   The decision field must contain the pre-approval body. If the only body in the log is the final sent version, the log has been overwritten by your edits and the model's actual proposal is lost.

4. 4

   4. Who approved it, and how long did that take

   The approver id must resolve to your user, not to "human" or an anonymous session. The gap between proposed_at and approved_at is the review latency — a Copilot log that shows sub-second approvals for a real human is not credible.

5. 5

   5. What is the external message id, and can you find it in the provider log

   The result field must contain the SMTP or Gmail API message id, not just "success." Cross-reference it against the provider's own log to prove the audit trail agrees with reality.

6. 6

   6. Is it still undoable, and if you undo it, does the log record that

   Undo must appear as its own log row — same actor, same input reference, new decision ("recall"), new timestamp. If the original row is edited in place or simply marked "undone," the log is mutable and no longer evidence.

Red flags#

These are the patterns that repeatedly disqualify a tool during a serious security review. Any one of them is a hard stop until the vendor either fixes the gap or writes it into the contract.

• One timestamp per row. If the log records only executed_at, the review latency and approval time are invisible. You cannot detect a queue that batches approvals or a policy that fired hours after the trigger.
• The approver is always "human" or "system." That is a schema tell — the field is a string constant rather than a foreign key to a user or a policy, and it will not stand up to "who signed off?"
• The log is mutable. If a support engineer can edit or delete rows on your behalf, the log is not evidence. Ask whether writes are append-only and whether the vendor's staff can amend rows post-hoc.
• The input reference is a subject line, not a message id. Subjects collide, threads merge and archives move. A log keyed to human-readable strings cannot be replayed against the provider six months later.
• Model version is a family, not a version string. "GPT-4 class" is not a version. You need the exact identifier the API returned, so a regression in a specific release can be isolated.
• Retention is described in the ToS and only there. ToS can change on notice. The retention period must appear in a signed DPA clause with a location, or you have no leverage the day the vendor changes it.
• Undo state is boolean. "Recoverable: yes" is not helpful without a window, an actor who can execute it, and a log row when the window expires. A vendor whose log never records expiry is usually not tracking the window at all.
• Export is by CSV from an admin page only. A real audit surface has an API. If you have to file a support ticket to get the log for a specific date range, the tool cannot serve a legal-hold request without vendor cooperation.

What we'd pick and why (honest)#

The concession first, because it makes the rest of this section credible. Microsoft 365 with Purview Audit is not on the same map as any AI email client, ours included. Purview keeps user-activity records tenant-wide with retention that reaches into years on the premium tier, ties into ediscovery and legal hold, and covers actions that never touched an AI agent. If your requirement is a compliance archive for the whole organisation — every user, every action, retained under legal hold — that is what Microsoft Purview is built for, and no AI email tool replaces it. Verify the current retention and pricing on Microsoft's own page before you commit.

This checklist is for the tool that runs alongside that archive: the AI email client that proposes and executes actions on your inbox. Its log has to answer "what did the agent do, on whose authority, and can we undo it," which is a different question from "what did every user in the tenant do this quarter." A tool that scores gold on the eleven fields above is the right layer for the AI-specific incident; Purview or its Google Workspace equivalent is the right layer for tenant-wide compliance. Both, not either.

In the AI-tool layer, AI Emaily is the pick we would defend on this criteria. We log every agent action with the actor, the authority mode (Manual, Copilot or Autopilot), the trigger, the input reference, the model and version, the proposed decision, the approver, the three timestamps, the result, and the undo state — because we could not ship Copilot and Autopilot honestly without it. Copilot approval is a mandatory checkpoint in v1; every autonomous action in Autopilot is written to the audit trail and reversible inside the undo window. Disclosure: we build AI Emaily.

Where we fall short of gold on our own scorecard: exportable is partial today. You can review the log inside the app, and undo events land as their own rows, but a machine-readable export API is on the roadmap rather than shipped. If your review requires programmatic export from day one — for a SIEM pipeline or an automated ediscovery workflow — that gap is real and worth naming. On the trial (seven days, card required, cancel before day seven and you are not billed), the log surface is what to open first. If it does not survive the worked example above, no other feature is worth evaluating. See our audit log doc, our Copilot/Autopilot overview, our security page, and pricing.