Full Metal Email - Confessions of an 'Anti-Spam Zealot'
We know that spammers are the enemies of all legitimate marketers—the fear spread by malware, fraud and spoofing have made our jobs increasingly more complicated and difficult. But if we think we have it tough then the ISPs are fighting an uphill battle where the end-game hasn’t appeared on the horizon. I’m happy to bring you another fantastic interview from the spam trenches, as the title says, in email the wind doesn’t blow, it sucks. No one knows this better than Yahoo!’s Carlo Catajan who has been on the front lines and in the trenches at one of the world’s largest ISPs for almost a decade. As a member of the anti-spam operations team at Yahoo! Carlo has seen it all and now he shares a bit of his knowledge and insight with us here on the Unica blog.
L.S.—I’ve seen your business card and the title is “Anti-Spam Zealot.” You said you were given the opportunity to pick whatever title you wanted, but I’d like to know why you think of yourself as an anti-spam zealot. Can you tell me a bit about your zealotry?
C.C.—The idea for the title came about from a conversation I had with someone years ago. The person described e-mail enthusiasts who insisted that “confirmed opt-in” was the only way to build a subscribers list as “a bunch of zealots.” The description amused and resonated with me. I also thought it was apt yet playfully menacing. And while the title might come off as cutesy, you need a lot of real passion and commitment to do anti-abuse work. And I – and everyone else in the world of spam- and abuse-fighting, in fact – has to possess that kind of zeal.
L.S.—How did you come to join Yahoo’s Anti-Spam Ops Team?
C.C.—I initially joined Yahoo! as part of the abuse team for Yahoo! Clubs in 2001, which eventually morphed into what is now Yahoo! Groups. After a couple of years in that role, I wanted to try something different and transitioned over to Yahoo! Mail’s abuse/postmaster team. That opened my eyes to our dynamic and intriguing space, and there has not been a dull workday since.
In mid-2005, I left Yahoo! to join Habeas, where I did sender support services for about a year. I then rejoined Yahoo! Mail’s product team in 2006. While I’m not technically with the postmaster team anymore, I do collaborate closely with the team and am very much involved with how we provide support to external senders around the globe.
L.S.—You’ve worked on the vendor side, did that help shape your opinion or perspective about “mailers” having them as clients in the capacity of a vendor?
C.C.—Definitely. Switching to the other side (of the MTA) was a challenging and eye-opening transition. My role at Habeas was part external e-mail auditor and part consultant, so I advised other companies on best practices to deploy. As an outsider telling companies how they should run their e-mail program, that proved to be a daunting task. Enlightening clients on what they were doing wrong or what areas needed improvement in their existing practices was the easy part; actually compelling them to execute changes based on those recommendations, which often entailed negatively impacting their success metrics, was often the challenge.
L.S.—You might’ve heard this argument before: “marketers talk deliverability, ISPs talk all forms of messaging abuse” how do you think mailers and ISPs can work more closely together to curb abuse and increase deliverability?
C.C.—Legitimate senders and receivers ultimately share the same end goal: to deliver e-mail that matters to our mutual audience. We don’t view marketers as adversaries, and I think our continued participation and collaboration with other mailbox providers and commercial senders in terms of establishing best practices and technological advances in e-mail are a testament to that.
L.S.—Engagement is all the rage these days, can you tell us a bit about how engagement factors into Yahoo!’s filtering schemas and or how Yahoo! utilizes engagement?
C.C.—Solely relying on complaint rates to gauge sender reputation no longer suffices. Not only have spammers found ways to game that methodology, it also completely relies on users to actively click on the “spam” and “not spam” buttons in order to feed our data to our reputation systems.
By looking at aggregate data that reflect how recipients interact with a sender’s messages (e.g., how many of a sender’s messages are opened, how many users clicked on a URL within the e-mail, how many users had the sender in their address book, etc.), we can infer different levels of interest that a particular sender elicits from their recipients. And these additional reputation features can paint a clearer picture of a sender’s reputation than merely relying on the direct user feedback we collect from the “spam” and “not spam” buttons, a mechanism that we know not all users utilize.
L.S.—You’ve been behind the scenes for a long time, and I’m glad they’re letting you out of the bat cave. That being said, I’m sure you do a fair bit of reading, what are you favorite informational sources about mail-abuse and deliverability?
C.C.—To be honest, it was my choice to remain incognito—the proverbial student lurking at the back of the classroom. (By the way, there is, in fact, no bat phone in the cave.) Being a good student, I do follow a handful of blogs to keep me up-to-date on what’s happening in the industry:
Box of Meat – It’s an excellent amalgamation of all sorts of spam, security and other related news. It also serves a drool-worthy side order of delectable meat pics.
Word to the Wise – Laura Atkins’ blog is the go-to place for great deliverability advice; a must-read for any professional in the field.
Terry Zink’s Anti-Malware blog – Terry provides insightful commentary on botnets, spam and other forms of online abuse.
Spam Resource – As its name suggests, Al Iverson’s blog covers a gamut of hot topics and news items in the world of spam.
Deliverability.com – Despite the fact that I’m part of its list of contributors, the site is really a good resource where e-mail insiders share their thoughts and insights.
L.S.—Can you tell me about what Yahoo!’s top anti-spam priorities are today? Is phishing still as prevalent as it was a couple years ago where every other email seemed to be a Nigerian 419 scam?
C.C.—Phishing is still a very big concern. From the standard site spoofing, to keyloggers and other malware being spread via e-mail, and users’ accounts being compromised to send out spam, it’s a top priority for us to protect our users and our network from these threats.
As for the Advance Fee Fraud scams, it is obviously still around, and as prevalent as ever. No longer are they restricted to professing grandiose promises of hidden loot and lotteries to random recipients. The fraudsters have now expanded their purview to hacking social networking and webmail accounts, which enables them to send more effective pleas for funds directly from a user’s own connections.
L.S.—If you were sitting in a full lotus position and floating two inches off the ground bathed in ethereal light wearing a saffron robe, what kind of mantra would you offer mailers to help them in the hour of their need?
C.C.—At that stage of enlightenment, I’d be chanting “postmaster.yahoo.c … OM.”
Seriously, we’ve worked hard to provide senders with a plethora of useful information on the Yahoo! Mail Postmaster site at http://postmaster.yahoo.com. For mailers having issues sending to our users, that is definitely the place to start. I especially like to reference this FAQ as a starting point for any mailer, old and new alike.
L.S.—Do mobile technologies factor into your anti-spam zealotry, if so how?
C.C.—While it still obviously pales in comparison with the amount of abuse we see via the web-based or desktop e-mail client, we have seen syndicated abuse of mobile gateways and our mobile platforms. Our suite of defenses don’t change no matter the platform, though, so we can still leverage the technologies we have been continually fortifying over to the mobile arena.
L.S.—Social isn’t the next new thing, it’s the thing today, and being FB buddies I feel justified in asking this question: have you seen any kind of spam or abuse that is specifically geared toward or piggy backing off of social platforms?
C.C.—We see a steady stream of spam, phishing and malware-laden e-mails pretending to come from Facebook and other popular social networks, and we block a huge portion of that traffic. We’ve also noted some of the hardcore spammers hosting their image files on some of these sites and embedding them in their messages—running on the assumption that no mailbox provider can possibly block URLs from these popular domains.
Finally, as I mentioned earlier, the 419-type fraudsters are there trying to take advantage of users’ already-established connections. Just imagine getting an urgent plea for monetary help from one of your buddies; wouldn’t you want to help them out if they’re stranded without a phone or any money in some faraway country? You and I wouldn’t fall for that since we’re keen on such ruses, but for folks who aren’t dealing with e-mail abuse professionally, that message will tug at their hearts and will likely compel someone to wire the requested funds from their nearest Western Union branch. Perhaps this is the reason why even our spamtraps are getting Facebook invitations.
L.S.—Is there anything special Yahoo! Is doing because of this considering that rumor has it FB is planning on becoming a mailbox provider?
C.C.—Our goals remain the same: the first and foremost of which is to keep improving the core features — speed, security, accessibility and stability — that our 275+ million users have come to depend on. In addition, we are incorporating Yahoo!–specific and 3rd-party social streams to enhance the overall webmail experience. And through our OpenMail platform, we are continuing to grow the productivity tools and apps that our users can utilize to manage their mailbox.
With or without the advent of Facebook e-mail, we’re working on a lot of exciting features for our users in our quest to be the best e-mail service on the planet.
L.S.—Can you tell me the top metrics that mailers should be concerned about if they want to improve their deliverability and why?
C.C.—Bounce rates – If your mailing list includes a lot of addresses that are bouncing, you’re definitely doing something wrong. That’s a telltale sign that your existing methods of collecting addresses is flawed.
Complaint rates – Another no-brainer here. If you’re not enrolled in our complaint feedback loop program at http://feedbackloop.yahoo.net, you absolutely should as soon as you can.
Engagement Data – We’ve talked about this already, and senders should start looking at ways to keep their messaging relevant to their subscribers. Merely acquiring an interested user for a mailing list doesn’t cut it anymore, senders have to make sure that interest doesn’t wane else their reputation may suffer if the just keep mailing to all e-mail addresses they’ve collected throughout the years.
L.S.—Do mailers have to sign SPF, DK, DKIM in order to achieve inbox status with Yahoo? Is there one they should sign over another?
C.C.—It is not a requirement to authenticate your e-mails to achieve great deliverability at Yahoo! Mail, though it does have a lot of benefits. One of which is the eligibility to enroll in our complaint feedback loop program, which does require a sender to sign with either DomainKeys or DKIM.
If you’re a sender just starting to look at domain-based authentication for your e-mail stream(s), then DKIM is definitely the way to go. Using both DKIM and SPF is even better.
L.S.—How will domain reputation help or hurt mailers once Yahoo begins to rely on DKIM and measure mailers by their Domain rather than their IP?
C.C.—Domain reputation really aims to benefit legitimate senders in the long run. It enables us to identify that a mail stream is coming from an authorized source for a particular domain, and potentially reject any e-mail forging their domain if the sender prefers to have that enforced. DKIM also allows us to track the reputation of a sender’s different mail streams if they sign each with a distinct subdomain—even if they just use a single IP address to send out these different types of mailings.
Another touted advantage is the domain reputation’s portability. Whereas a good sender loses any earned reputation when they have to reassign their mail servers to new IPs, domain reputation via DKIM allows the sender to change their sending IPs whenever, as long as they continue signing with the same domain. That ensures uninterrupted deliverability for good senders, as well as lower maintenance efforts since they don’t have to contact each and every mailbox provider out there to announce their new IPs.
L.S.—I won’t ask you what mailers should do when attempting to contact Yahoo! for deliverability remediation, rather I’ll ask you this: having seen a million and one requests for unblocking, what are your top 3 reasons NOT to grant an unblock request?
C.C.—Let’s just say that mentioning the following won’t win you any automatic brownie points:
- “I spend bazillions of dollars on display ads on Yahoo!”
While this is certainly good for Yahoo! and the overall state of display marketing, it won’t get your e-mails magically displayed in our users’ inbox. In fact, it is totally irrelevant.
- “We have a 0.000001% complaint rate and we’re still ending up in the spam folder!”
Such claims are usually too good to be true. But if it is, it can mean several things:
- Almost all your e-mails are ending up in the spam folder so our users don’t get the opportunity to click spam on your messages.
- Your mailing list consists mostly of accounts you own.
- You’re not properly signing all your e-mails with DKIM and, as a result, are not getting all the complaint feedback loop reports generated by your mailings.
- All of the above.
If a company is sending messages that our users truly want, our system is reactive and smart enough to detect that. While a good sender may see bulking initially, once we have collected enough positive reputation data for them, the messages will land in our users’ inbox accordingly.
- “We have changed our spamming ways and are completely on the up-and-up now.”
It’s always nice to hear when previously wayward senders start abiding by industry best practices. However, simply claiming to have done so isn’t enough.
A sender can’t expect to see their deliverability do a complete turnaround as soon as they’ve implemented changes; reputation, online and off, is earned after all. Give it some time and deliverability will eventually improve.