Spamcop blacklists autoresponders
I learned a couple of days ago my mail server got blacklisted by spamcop.net. They don't reveal the reason for it, but it's likely that I was blacklisted for running an autoresponder, in this case my own custom challenge/response spam filter which is the oldest operating one I know of.
I understand the debate about the merit of C/R spam filters. Like all autoresponders, they can generate unwanted mail when spammers and viruses send mail with a forged From address, and the responder annoys the innocent victim. However, this is a problem common to all autresponders, and unlike the even-more-hated open-relay, it doesn't magnify the spam problem -- there is one possibly annoying response per spam, not hundreds.
I am bothered because I don't want to see anti-spam advocates fighting other anti-spam methods because they don't agree with them, or blacklists in general used to punish people you don't agree with. Spamcop should be fighting spammers, not anti-spammers.
In addition, e-mail autoresponse is an important mail tool. In fact, anti-spammers insist that mailing lists do a confirmed opt-in (also known as double opt-in), generally by autoresponse, before adding a person to a mailing list. When a mail server bounces directly delivered mail it can avoid doing an autoresponse, but if mail comes in through an MX -- a vital feature of mail -- it requires an autoresponse to bounce it. Vacation programs and many other tools use this ability.
Check to see if your mail system uses spamcop.net as a blacklist. If it does, disable it or switch to something else until they change this policy. Otherwise you won't receive mail from me, and many others.
Update: My server is no longer blacklisted. I didn't do anything (other than this blog post and a few complaints to people using the spamcop BL) so perhaps they auto remove. But it could happen again at any time until they change their policy. This is also a nasty DOS attack. Find anybody with any autoresponder, including a bounce of MX'd mail. Send forged mail to it with a From set to a spamtrap address -- and they're blacklisted. Also can be used against any sites that have you enter an E-mail address on a web page and then email that address to confirm you own it -- you can get these sites blacklisted trivially. Every web form that can enter an E-mail address is at risk.
Tue, 2006-01-03 12:17
We also run a challenge/response-based anti-spam system, and have faced similar blacklisting to what you're describing by spamcop.net.
We usea lot of intelligence on our verification process to make sure we challenge as little forged mail as possible, using ideas that you mention in your c/r best practices document, as well as SPF, anti-virus, and other heuristics.
Because our system does not always operate on the receiving smtp server (we poll some mail via pop/imap/etc), any responses to forged emails are considered backscatter. This is one of the things we strive to minimize.
We also run all of our challenges out through a single IP address, so just in case that host does get blacklisted, the rest of our customers' outgoing email is not affected.
The blacklist that has given us the most grief is SORBS. About a year ago, our entire netblock was blacklisted by them. After much effort, we were able to have them only block the single IP used by the challenge process. While that is certainly not what we wanted, that was as far as we could get with them.
However, just recently, they decided to block our entire netblock again. Why? They had not seen ANY problems with the rest of the IPs, which all require authentication, and have rate-limiting and other abuse-controls, but they felt that we hadn't done enough to "fix" our system, and wanted to exert pressure on us to do that.
What good did this do? Well, we spent a lot of time trying to convince them what a bad idea it was.. A lot of time explaining to our users why they couldn't send emails to their friends and family whose ISPs use the SPEWS list.. And a lot of time talking to ISPs, who certainly weren't expecting/intending to be blocking legitimate email by using their blacklist.
Ultimately, we were able to come to an agreement where they went back to just blocking our challenge ip again. But I have no doubt it will happen again, with spews or with someone else. Sure we can play cat and mouse, jump around various ips, etc. But that's not what we're interested in.
When we're blacklisted, we do exactly what you did: make sure everyone involved knows exactly what happened and why, and let the blocking mail provider, and the recipient who didn't get the email, decide whether using that blacklist is really in their best interest.
Tue, 2006-01-03 20:13
Spamcop runs on full auto and has been for years. They take pretty much anyone's word about whether a site is spamming, and last time I looked around their site, they don't seem much bothered by the fact that they kill a lot of legitimate e-mail that way.
They auto-age entries out of their database in a day or two of no spam reports, and they seem to believe that this is sufficiently quick for bad entries to go away.
I have no use nor respect for the spamcop system - they don't give a rat's that they are trivially used as a DOS attack, and they don't care that you a lot of e-mail goes missing because of them.
Any anti-spam solution that regularly drops real mail on the floor is NOT a solution - it's a problem of it's very own.
Wed, 2006-08-30 00:42
Where the bad links come from
I was in a forum discussion earlier today and told a guy named Kelly about a bad experience I had with SpamCop. Some spammer was putting MY site URL in their spam (along with many others) to saturate the URLs they wanted in the content of their spam. SpamCop complained to my ISP and my site came down for 12 days while I sorted it out with tech support. Some of the bloggers were calling this shotgun reporting and a menace to the Internet. I agree. SpamCop should not trust URLs in the body of e-mails as always being spam. What! Are they just grabbing all the links and e-mail addresses they can outta the spam? How wrong is that! They should do a little homework.
Tue, 2006-01-03 22:33
Join the club
As near as I can tell, GreatCircle.com is more-or-less permanently stuck on the SpamCop blacklist unless I want to stop running Majordomo. Yeah, like that's gonna happen...
Wed, 2006-01-04 00:25
Use A Filter on Top
Reducing the amount of challenges is probably the right move to take. You can reduce the likelihood of banishment in this way, as well as become less of a nuisance to the Net. In other words, try ruling out cases when messsages are rather obvious spam. It leads to lower volume of messages being dispatched, which in turn can avoid blacklisting.
I use SpamAssasin, which is active at a layer higher than challenge/response (in this case Apache with BoxTrapper). Whatever gets scored as spam will be put aside in a mail folder which is reserved for spam. Only messages not marked as spam (and not in the whitelist either) will have a challenge delivered. This cuts down the number challenges by about 70% in my case. It never entails any false positive because I set the thresholds high.
Wed, 2006-01-04 07:02
Nice blog Mr Templeton
I visited your photography page and saw the gallery of photos taken from around the world. Nice work especially on the Australia photos - keep up the good work!
Thu, 2006-01-05 06:18
I dont use the spamcop blocklist, but ..
C/R systems are more or less the smtp equivalent of what a network admin would describe as a smurf amp. As others posting here have pointed out, while they may reduce your spam levels to some extent, one thing they serve to do is to generate double the unwanted traffic when someone's being forged into spam.
That, plus lots of mailing list admins just dont like being at the receiving end of C/R systems .. Dave Farber was saying he'd unsub anybody who sent a C/R challenge back in response to IP email, for example.
Even with all the steps that are being suggested (first running spam through a bunch of filters before you run a CR bot on it as a last resort) it is not a good idea. And there's even more fun when it conflicts with another technique that I dont like too much - graylisting. Watch a c/r bot end up sending challenges back to a mailserver that uses graylisting, its a highly entertaining (!) experience, I assure you.
On a tangent from your original post, responding to Daryin. I'm too much of a fan of SPF either. Two interesting posts on circleid -
http://www.circleid.com/posts/spf_loses_mindshare/ (by John Levine)
http://www.circleid.com/posts/port_25_blocking_or_fix_smtp_and_leave_port_25_alone_for_the_sake_of_spam/ (which I wrote sometime before John wrote his article)
We were the first email provider to publicly stop publishing spf records, even conservative ones. This was back in February 2005. Earthlink followed suit a few months after we did - in July or August 2005.
Thu, 2006-01-05 12:04
I know people disagree with c/r
But autoresponding isn't going away, it's too useful in bounces after mx, for mailing list confirmation, confirmation of email addresses entered on web pages etc.
The answer is not blacklisting autoresponders, but working to fix the autoresponse to forgery problem. Autoresponses normally don't multiply spam like open relays, they just reflect it. That's not good for the person it is reflected at, of course. But the autoresponse does not advertise the product so the spammer is not interested in it. He's just putting in a forged From to get past whitelists and detectors of invalid From lines.
For autoresponses to emails, we may need to move to a regime where those who want autoresponses sign their mail. However, long before that we could move to some standization in automated responses, so that it's easy to detect autoresponses to messages you never sent out, and be rid of them.
C/R is worth protecting because it is the only system that can turn an anti-spam filter into a no-false-positive filter. The correct approach is to discard spam you are sure is spam, pass through what you are 99.9% sure is ham, and challenge the small quantity of stuff you can't figure out.
Some people say, "I don't like challenges" but if you pose the question, "would you rather have a challenge, or would you rather your mail was discarded or put into spam folder that may or may not get looked at?" -- the answer is different. I sent the mail for a reason, and I want a chance to override the spam filter if it decides not to deliver it. The message might be important.
One could also consider a flag to say "Don't bother challenging me" for those who don't want a challenge if their mail is about to be not delivered.
Non-delivery is a serious failure of the mail system. It must not go unreported. Some would argue it should be delivered to both parties. It could make sense for the sender to decide who to deliver it to, though you can't easily stop the recipient from superseding that.
If the algorithms can't figure out whether to deliver, only a person can.
Fri, 2006-01-06 04:28
Non delivery is a failure
Non-delivery is a serious failure of the mail system. It must not go unreported. Some would argue it should be delivered to both parties. It could make sense for the sender to decide who to deliver it to, though you canâ€™t easily stop the recipient from superseding that.
I agree. That's what smtp has DSNs for. Any 550 error message (which turns into a mailer-daemon notification in the sender's mailbox) we issue has something like "Mail refused, please see http://spamblock.outblaze.com/188.8.131.52" (or whatever the blocked IP is).
Similarly for blocked domains, or for other filter rules. And that's accompanied by a fairly easy auto removal mechanism that can be used just once - and a link to contact the site postmaster (me and my staff) - and we respond to tickets reporting false positive blocks within a business day, or even sooner.
Blocking is going to keep happening, and C/R bots just dont scale .. but blocking should be done responsibly and should be blocked by people who are willing to listen and respond fast to false positive reports.
Fri, 2006-01-06 13:10
CR doesn't scale?
Some badly implemented C/R schemes may not scacle, but C/R, as implemented according to my best practices, scales fine -- leaving aside, for a moment the issue of what to do about challenging mail with fake addresses.
In a world where everybody used C/R, you would only get a challenge when:
This is something I see happening perhaps a few times a week, if the whole world used C/R. That scales fine. There is an issue when you change E-mail addresses. Then you will get a new slew of challenges, temporarily. But it's not giant burden. In such cases I would recommend the adoption of tools to bypass all anti-spam including C/R because most spam filters do use some level of whitelisting, and you give up your whitelist privs when you change E-mails.
Mon, 2006-01-09 06:06
CR and scaling
> This is something I see happening perhaps a few times a week
> if the whole world used C/R.
Please, Brad. Hotmail, Yahoo, AOL, etc add something like several hundred accounts to their service an hour. There's no way CR is going to ever keep pace with it. Even after using CR as a last layer of protection against spam.
http://www.hserus.net/images/minute.png - that's about a million emails a minute rejected, and 100k emails a minute accepted / passed by our filters. Now, even if a fraction of those incoming emails were greeted by CR ... and our mail farm (with 40 million users) is barely a third the size of AOL's.
NO way CR is going to scale in such a situation.
Mon, 2006-01-09 12:25
Are you talking about the
Are you talking about the forgery problem? C/R at most doubles the volume of mail currently being handled. However, if your spam filter is already classifying 80% of the mail you get, which it had better, then C/R is only adding 20% to the total volume of mail (with most of the challenges discarded.)
I fail to see anything that doesn't scale about that. The only thing that doesn't scale is the autoresponses to forged addresses, if those victim addresses don't have their own suitable filtering. Beyond that, how can anything that simply does a small linear multiple not scale? "Not scaling" refers to something that gets harder to handle (per unit) the more units you have, not something that remains linear.
Thu, 2006-01-12 06:42
CR and scaling
I wrote a CR filter for my OS/2 mail server in 2002. It was increadibly effective but I took it offline to prevent the challanges from being classed as spam.
But I do want to explain some of the reailty of C/R as a spam threat. First off most forged spam e-mails we get are non-existing e-mails. They do not go to a single user. Secondly my (and I assume other) C/R programs will only send one C/R to any email address a week. Since spam addressed are 99% repeated that cuts C/R responces down way farther (even for us getting tens of thousands of spam a day). This does not include a whitelist (auto built from your users SMTP sending)and bayesian filter wich catches most spam before it even hits CR.
The problem is if you have a huge comercial spammer (debatable) who can hit enough C/Rs they can flood an account with a lot of C/R responces. Thus the argument by blacklists is all C/R's should be treated as so better than the original spammer.
Here is the thing. Pulling back from the knee jerk reaction for easy management. If a huge spammer is spamming THAT many addresses with a forged e-mail the forged users account is going to get flooded to beyond belief with flames and angery responces anyways making C/R the least of that account's problems.
Solution: If it was possible for this idea to come forward (without being blacklisted for even suggesting it). I think one solution would be to "standardize" for what is concidered a acceptable C/R responce. Example:
C/R: should start the subject line.
The message must be text and be no larger than 512 bytes. Only one link mail or e-mail in the message and it must go to the same domain as the sending SMTP mail server.
Reasoning: The C/R subject would allow mailers (just as bayesian preforms test emails now) to flag for testing C/R emails (even limit # recieved in X time). This format would work and be too restrictive for most spammers. However it would require the adoption of standards .
Another idea I have been kicking around (which would be easier to implement into existing systems) would be to set up a volunteer C/R server (similar in idea to the blacklists). Only legit registered servers could relay C/R through our server. Our server would then apply the "tests above" to each C/R. No more than 5 C/R's would ever be sent to one e-mail address in an hour. A delay to a ISP's message queue (time to delay queue after so many C/Rs sent at once) based on ISP size to prevent a hacked ISP from spamming (before being caught).
This server model could be distributed via a server farm or participating servers (DNS (round robin) and MX).
I feel this method would give ISP's a "safe source to whitelist" for C/Rs, prevent any abusive volume of mail from C/Rs, and safe guard spammers from abusing the system.
Thu, 2006-01-12 11:28
Such standardization is
Such standardization is already in place (there's an RFC) and in a later blog post you will find more proposals on this very topic.
Mon, 2006-06-26 20:58
Backscatter spewing - Spamarrest.com
Spamarrest.com, your IP's are bloked at my system due to many, many c/r backscatter attempts. Your Challenge mails go straight to quarantine now. Sometimes I skim over the quarantine, and if I see one, I confirm it.
Mon, 2006-07-03 12:54
What if the server your
What if the server your autoresponder is on checks against Spamcop before allowing the mail to come in and BE auto-responded to? Wouldn't that prevent the mail from a spammer from getting responded to in one of Spamcop's anonymous spam traps.
Tue, 2006-07-04 13:43
So you are saying
In order to avoid being unfairly blacklisted, use the unfair blacklist yourself, and not offer delivery to those on it?
This would possibly make sense for a better blacklist, one that follows well-established principles of justice (presumption of innocence, right of to defend yourself, right to see evidence, right of appeal etc.) but not for spamcop.
Add new comment