New Metrics for Inbox Performance
Written by Laura Villevieille.
Laura Atkins of “Word to the Wise” recently wrote a fine article detailing the problem we have as senders: we only ever have access to engagement data through proxy measurements like “open” and “click-through” rates.
There are other metrics that senders would love to see and use to help measure engagement, for example: How many people actually read a message? Or deleted a message without reading it? Or how long a user kept an email open on their screen? All of these metrics would be fantastic data for marketers, and would help us diagnose our own deliverability issues without opening trouble tickets with the ISPs that own the architecture that records this data; sharing it with us would require they invest in development that pretty much only benefits senders, and has the additional problem of possibly even giving away part of their filtering “secret sauce”.
Surprisingly, though, a couple of trail-blazing ISPs have made this data available to senders! Mail.ru has recently rolled out a new postmaster site that includes several of these dream metrics. QQ has also made some of this data available.
Mail.ru, operator of the largest Russian webmail provider and social networking sites, has just released a new postmaster website and feedback loop. The verification process is a bit more involved than other feedback loop programs, but the access it provides is well worth the effort!
The Mail.ru system allows us to see key deliverability metrics like “number of complaints” and percent delivered/blocked/spamfoldered”, and also provides several key engagement metrics: “number of messages read”, “number of messages deleted after being read” and “number of messages deleted unread”. All of this information is visible per sending domain by day, week or month and in a build-your-own chart allowing you to compare any of the metrics by week, month or year. How cool is that?!
As Jerome Gays from Signal Spam recently shared with us, China’s largest ISP “Tencent QQ” has also made some of these metrics available. The QQ system allows you to register as the administrator for your sending domains and IPs. Once registered, reports will show “messages sent”, “messages received in the inbox”, “messages filtered”, “read rate “and “complaint rate” (shown in that order in the image below). Like Hotmail’s SNDS reports, these metrics are available by IP by day.
Unfortunately, there is no English version of the QQ system, but having access to the ISP’s data is nevertheless really valuable.
With ISP data like this from QQ and Mail.ru, we are able to see new pieces of the deliverability puzzle that were previously never available, and we are finally able to answer some long-standing burning questions:
- How many people opened the message but didn’t download images?
- What was the genuine complaint rate?
- What was the inbox/filtered rate, really?
- How does that number differ from what our seed tests show?
- How many people deleted the message without reading it?
- What does this data indicate about the mailing’s subject lines, frequency, timing and saturation?
- What does it imply about the mail stream itself (daily deal/newsletter/offers/etc.)?
As deliverability experts, we can use this information to accurately diagnose deliverability problems without engaging the ISP’s Postmaster teams. It gives us insight into the same complaint and engagement metrics that filters use. This added visibility can and should also be used to identify issues and help to monitor and stop outbound abuse on our networks.
As senders and marketers, this is really exciting data that gives us a new ability to gauge the real performance of our email campaigns, draw new insights about the relevance and reception of campaigns, and use that information to improve our sending practices thereby improving our inbox placement and ROI. That’s incredibly valuable information, and we can only hope other ISPs may follow the trail being blazed here. Thank you, Mail.ru and QQ!