Email Analytics

This document explains how email opens, clicks, and headline rates are currently calculated by WhiteBeard News Suite.

Source Of Truth

1. How Email Opens Are Classified

Open events are classified into four buckets:

• likely-human
• automated
• mpp
• unknown

Inputs Used For Classification

The classifier expects context such as:

• ua: user agent
• ip: request IP
• method: HTTP method
• has_range: whether the request sent a Range header
• secs_since_send: seconds between send time and open event
• is_human: hard override used internally when a click proves human intent

Scoring Rules

The classifier starts with bot = 0 and human = 0.

Bot-leaning signals:

• User agent contains GoogleImageProxy: bot += 10
• IP matches Apple MPP IP ranges: bot += 5
• HTTP method is HEAD: bot += 3
• Request has a range header: bot += 3
• Open happens within 5 seconds of send: bot += 5

Human-leaning signals:

• Plain GET request with no range header: human += 2
• Open happens 30 seconds or more after send: human += 2
• If the request is at least 30 seconds after send and the user agent contains GoogleImageProxy, the earlier GoogleImageProxy bot penalty is removed with bot -= 10

Hard override:

• If is_human is present and truthy, the result is forced to likely-human

Final Open Label

After scoring:

• If bot - human >= 5, label = automated
• Else if human - bot >= 2, label = likely-human
• Else if the IP is in Apple MPP ranges, label = mpp
• Else label = unknown

Open Logging Behavior

When an email is opened, one extra rule is applied on top of classification:

• likely-human opens are deduplicated for 5 minutes per message ID
• non-human opens are still logged, but they are not protected by that 5-minute dedupe path

That means repeated human opens in a short period are intentionally collapsed, while automated, MPP, and unknown opens are logged more directly. This is to eliminate repetitive automations linked to the open trigger from running.

2. How Email Clicks Are Classified

Unlike opens, clicks do not fully rely on the score-based classifier for every final label. The click path uses a simpler decision flow.

Click Classification Flow

The click label starts as likely-human.

It is changed to automated when either of these is true:

• secs_since_send <= 5
• the requester IP was previously marked as malicious by the https://roadblock/ scanner trap and cached for 24 hours

Additional guards:

• If there is no HTTP_USER_AGENT, the click is ignored
• If the user agent does not match Mozilla, the click is ignored

MPP handling:

• If the click is not already marked automated, classify($ctx) is called
• If that classifier returns mpp, the click is labeled mpp
• Otherwise the click stays likely-human

Explanation about the roadblock URL:

• A hidden link is added to every e-mail with a destination https://roadblock/
• If a click is received on this URL, the source IP is classified for 24 hours as automated.
• This helps detect clicks as bot clicks when they come from an unrecognized IP address.

Practical Click Buckets

In practice, the click logger currently emits these labels:

• likely-human
• automated
• mpp

The reporting code has room for unknown, but the click logger does not currently assign unknown as a final click label.

Human Clicks Also Force A Human Open

For clicks labeled likely-human or unknown, the class does two extra things once per message within 5 minutes:

• triggers the newsletter_click automation
• call the open tracking logic, with is_human = true, which forces a likely-human open

This is an explicit design choice: a real click is treated as proof of human engagement, even if no human open had been recorded yet.

3. How Unfiltered Open Rate And Enhanced Open Rate Are Calculated

The metrics used are:

• delivered = total - failed
• open_unique = unique opens across all open labels
• open_human_unique = unique opens classified as likely human
• clicks_mpp_unique = unique clicks classified as MPP

Unfiltered Open Rate

The unfiltered open rate is:

unfiltered_open_rate = (open_unique / delivered) * 100

This is called openrate in the output.

It is unfiltered because it uses all unique opens, regardless of whether they were human, automated, MPP, or unknown.

Enhanced Open Rate

The enhanced open rate is:

enhanced_open_rate = ((open_human_unique + clicks_mpp_unique) / delivered) * 100

This is only used when open_human_unique > 0.

If there is no human-open classification data, the class falls back to the unfiltered open rate:

enhanced_open_rate = unfiltered_open_rate

Why clicks_mpp_unique is added:

• Apple MPP can hide real human reading behind proxy opens
• a click from an MPP-classified recipient is treated as a stronger engagement signal
• the metric therefore combines human opens with MPP clicks

4. How Click Thru Rate Is Calculated

ctr = (clicks_unique / delivered) * 100

This is the value exposed as clickrate with title CTR.

Important detail:

• clicks_unique is the total unique clicking recipients across all click labels
• it is not filtered down to only likely-human clicks

So the post-level CTR is an unfiltered unique click-through rate.

Summary

• Opens can be likely-human, automated, mpp, or unknown
• Clicks are effectively tracked as likely-human, automated, or mpp
• Unfiltered open rate uses unique opens divided by delivered
• Enhanced open rate uses human unique opens plus MPP unique clicks, divided by delivered
• CTR uses unique clicks divided by delivered