How Email Threading Algorithms Work: References vs Subject

Email threading algorithms take a flat pile of messages and reconstruct which ones belong to the same conversation. There are two families of algorithm doing this job, and they disagree in different ways. The strict, header-chain family walks the Message-ID, In-Reply-To and References fields to rebuild an ancestor tree — this is the RFC 5256 REFERENCES algorithm, descended from Jamie Zawinski's 1997 write-up. The Gmail-style heuristic family reads the same headers when they are present, then falls back to normalised subject and overlapping participants when they are not.

Neither is strictly better. Header-chain threading is precise when senders behave; heuristic threading catches conversations the headers cannot see. This post walks both algorithms, the failure modes each one produces, and where the third option — a client that reads both correctly — actually helps.

The verdict, up front#

If every message in a folder carries a well-formed References list, the RFC 5256 REFERENCES algorithm — the Jamie Zawinski algorithm — is the correct choice. It produces the ancestor tree the standard describes, it does not merge unrelated conversations, and it handles reply chains up to hundreds of messages without drift. It is the algorithm the IMAP THREAD extension standardised precisely because it has one right answer per message.

If headers are missing, truncated, or contradictory — because a mailing list stripped References, a gateway rewrote Message-IDs, or the sender's client only set In-Reply-To — a subject-plus-participant heuristic like Gmail's picks up the slack. It groups messages by normalised subject (stripping Re: and Fwd:) and overlapping From/To/Cc lists, so a thread survives one broken client mid-chain. The cost is that two unrelated threads with the same subject and shared participants can merge, and one thread whose subject changed can split.

In practice the choice is not either-or. Every modern mail client runs a hybrid: it walks the header chain first, then patches gaps with subject and participant matching. The differences worth arguing about are which signal wins when the two disagree, and how aggressively a client will merge or split when it is not sure. That is where thread bugs come from, and it is where a client's design shows.

At a glance: header-chain vs subject-plus-participant#

Both algorithms exist because email was never designed as a conversation format. Threads are reconstructed after the fact from whatever the sender's client and the intermediate hops left behind. The table below is the whole comparison in one view; the sections after it are the worked examples.

Where header-chain threading wins#

The RFC 5256 REFERENCES algorithm is what you use when you want the answer, not an answer. It is deterministic: given a set of messages, every implementation of it produces the same tree. Two clients running the algorithm on the same folder cannot disagree about which message is the parent of which reply, because there is nothing to guess.

The algorithm walks in a fixed order. It reads the References list to build the ancestor chain, treating each Message-ID as a node and each list position as an edge. Missing intermediate nodes are inserted as placeholders — the RFC calls these "dummy" or "empty container" messages — so the tree stays connected even when a middle message never arrived. In-Reply-To is consulted only when References is absent, and only for the parent link. Normalised subject is the last resort, used to attach orphaned messages to an existing tree by matching stripped subject strings.

Jamie Zawinski's original 1997 write-up is where this recipe was first spelled out step by step, and RFC 5256 turned it into an IMAP extension so mail servers could return threaded results directly. Every mail client with a proper conversation view — Thunderbird, Apple Mail, Mutt, Notmuch, most webmails outside Google — descends from this algorithm. It is the reason a well-behaved reply lands in the right thread on the recipient's side no matter which client they use.

Its win condition is a folder where senders' clients populate References correctly. In that world it produces a clean tree with the real parent-child structure intact — you can see who replied to whom, not just which messages happen to be in the same bucket. That structure matters for auditing conversations, for legal e-discovery, and for any downstream tool that needs to reason about who said what and to which prior message.

Where subject-plus-participant threading wins#

The heuristic approach is what you use when the world is messier than the standard. Gmail's conversation view is the most-used implementation and it makes several deliberate choices the strict algorithm does not.

Gmail groups messages into a conversation when they share a normalised subject and enough overlapping participants, even if the header chain does not connect them. Normalisation strips reply and forward prefixes (Re:, Fwd:, Fw:, and their translations in other languages), collapses whitespace, and often case-folds. Participants means the union of From, To and Cc across the messages — a reply that adds a new recipient still threads with its predecessor if the subject matches and the earlier participants persist.

This works well when headers fail. A mailing list that rewrites headers on the way out is the classic case: strict threading splits at the list boundary because References is gone, but subject-plus-participant heuristics rejoin the messages because the subject is stable and the list participants overlap. A reply composed in a webmail that only sets In-Reply-To gets pulled back into the thread the same way.

The failure mode is the mirror image of the win. Two unrelated threads with the subject "Meeting tomorrow" and a shared team on Cc will merge into one conversation. A thread whose subject changed from "Draft proposal" to "Signed proposal" mid-conversation may split into two, even though the References list is intact. Gmail also caps a conversation at 100 messages — past that it starts a new conversation with the same subject, which is a policy choice rather than an algorithm bug, but it looks like one from inside the client.

Outlook's conversation view is a lighter version of the same idea — it groups by normalised subject and Conversation-Index or Thread-Index (Outlook's own extension) — and it exposes a preference to split at subject changes, which the Gmail view does not. Yahoo Mail uses a similar heuristic. The proprietary side of email threading is real, and "how does Gmail group conversations" is genuinely a different question from "what does RFC 5256 say".

Cost and licensing model#

Both algorithms are free to implement. RFC 5256 is a public IETF standard, Jamie Zawinski's write-up is published openly at jwz.org, and no royalties or license fees attach to either. The IMAP THREAD extension is served by any RFC-5256-compliant mail server at no additional charge — Dovecot, Cyrus, and most modern IMAP servers advertise the THREAD REFERENCES capability out of the box.

The Gmail-style heuristic is proprietary in the sense that Google does not publish the exact algorithm — the specific weights, the normalisation rules, and the participant-overlap thresholds are internal. There is no license to buy because there is no shipped implementation to license: the algorithm runs on Google's servers and is available only through the Gmail interface, the Gmail API, and IMAP access to Gmail accounts (which threads with X-GM-THRID, Gmail's own thread identifier, alongside standard headers).

For a mail client, the real cost is engineering, not licensing. A robust threading implementation has to walk the header chain, insert placeholders for missing ancestors, deduplicate messages that arrived through multiple channels, and decide what to do when the header chain and the subject heuristic disagree. That last decision — the merge-or-split policy — is where clients differ, and it is what you are actually paying for when you pick one.

Verify the current capabilities and pricing of any mail client on its own live page before choosing — vendor packaging in this category has changed several times in the last year, and any figure printed in a blog post ages within weeks.

Who each algorithm is genuinely for#

Neither algorithm is the right default for everyone. The right choice depends on where your mail comes from and what you want to do with the conversation once it is grouped.

A third option, honestly: the client that runs both#

This post compares two algorithms, not two products, so the honest third position is the software layer that implements them. Any mail client can, in principle, run the RFC 5256 REFERENCES algorithm and layer a subject-plus-participant heuristic on top of it. The differences show up in how the client resolves disagreements between the two, whether it can dedupe a message that arrived on two accounts, and whether it exposes the underlying graph when you need to audit it.

That is where AI Emaily sits. We read Message-ID, In-Reply-To and References on every message we sync, run the RFC 5256 REFERENCES algorithm across your Gmail, Outlook and IMAP accounts in a single unified view, and apply subject-plus-participant matching as a fallback when the header chain is broken — not as the primary key. When our AI email assistant drafts a reply, it composes both threading headers correctly, so the reply lands in the same thread on the recipient's side rather than starting a new one. We build AI Emaily; the mechanics live at /docs/threading, the product page at /features/ai-email-assistant, and the overview at /. Packaging is a 7-day free trial on Pro and Autopilot — card required, $0 if you cancel before day 7 — with the current numbers at /pricing.

The concession worth naming: Gmail has built harder on its own subject-plus-participant heuristic than we have, and inside a pure single-account Gmail workflow its conversation view will recover from a broken References list more aggressively than ours does. If you live entirely inside one Gmail account and your threading complaints are mostly about mailing lists that mangle headers, Gmail's own view is still the tightest fit for that specific case. Where AI Emaily earns the switch is threading that has to work across accounts, or drafting that has to compose the right headers going out — the two cases a single-provider heuristic cannot handle at all.