Spam Filter ISP Forums : Maybe exclude aol.com and other IP’s from

Maybe exclude aol.com and other IP’s from : interesting thread. we switched...

Tue, 05 Sep 2006 10:12:47 +0000

Author: MartinC
Subject: 5621
Posted: 05 September 2006 at 10:12am

interesting thread.

we switched to having soft fails blocked due to the amount coming from hotmail that got through on this check.

it may not be perfect but we had 100s getting through of hard to block spam... with the config change, all are blocked.

it would be nice if the soft fail option was configurable per domain... or something we could override so we can get the benefits of both options and a bit more granularity.

Maybe exclude aol.com and other IP’s from : I like the idea of having an SPF...

Thu, 15 Jun 2006 16:59:05 +0000

Author: Guests
Subject: 5621
Posted: 15 June 2006 at 4:59pm

I like the idea of having an SPF override to force a -all on some of these loose records. Not sure though what sort of worms that would let out of the can with regards to when they add/remove mail servers from their pool.

Maybe exclude aol.com and other IP’s from : Roman, along the idea of prospective...

Thu, 01 Jun 2006 11:38:45 +0000

Author: sgeorge
Subject: 5621
Posted: 01 June 2006 at 11:38am

Roman, along the idea of prospective enhancements in SpamFilter�s SPF filtering options, wouldn�t it be nice to be able to explicitly override an external domain�s SPF record with one of your own making?

This would allow finer-grain control to tightening lazy SPF records. We could make better use of the lazy SPF records from the Big Guys, without rejecting any and all responses of neutral and softfails.

If one could copy and paste Gmail�s existing SPF record into SpamFilter, changing the useless ?all to a �all, for example, one could stand to stop a lot of spam.

I suppose though, that the right rejection code would have to be used. Returning an error due to the �Sender�s� policy would not be accurate.

Stephen

Maybe exclude aol.com and other IP’s from : Stephen, as you wrote "neutral...

Wed, 31 May 2006 15:01:20 +0000

Author: Roman
Subject: 5621
Posted: 31 May 2006 at 3:01pm

Stephen, as you wrote "neutral ... stated that [domain owner] cannot or does not want to assert whether or not the IP address is authorized". If you explicitly use some relay for your mail (I think) you should authorize that IP for this task. If that relay will fail to separate you from spamming (and faking your authorization) customer, well, may be it is not a very good relay. I�m afraid his IP (sooner or later) will be blacklisted if not for wrong SPF but for something else.

My point is: if you authorize some server to relay your mail then you should do it explicitly. You should not inherit foreign errors in your own configuration.

I know, acc. to original meaning of SPF you are (maybe) right. But current practice of big ISPs makes it wrong (ok, only for me maybe).

Well, I have a suggestion for Roberto which could help us all:

To make an option to treat �~all�, �?all� (and maybe �+all�) as �-all�, but process explicit �neutral/softfail� responses as it does now.

In other words I�d restrict all SPF statements which says �pass|neutral the world�.

Maybe exclude aol.com and other IP’s from : Roman, I understand that you...

Wed, 31 May 2006 13:46:47 +0000

Author: sgeorge
Subject: 5621
Posted: 31 May 2006 at 1:46pm

Roman, I understand that you can have your own unique interpretation of SPF results in order to curb spam, but your interpretation doesn't match up to what's in the spec. True, neutral and softfail results are to be used in the testing phase, but that is not their only usage.

From http://new.openspf.org/svn/project/specs/rfc4408.html#op-res ult:

2.5.2.  Neutral

The domain owner has explicitly stated that he cannot or does not want to assert whether or not the IP address is authorized. A "Neutral" result MUST be treated exactly like the "None" result; the distinction exists only for informational purposes. Treating "Neutral" more harshly than "None" would discourage domain owners from testing the use of SPF records (see Section 9.1 (Sending Domains)).

...

2.5.5.  SoftFail

A "SoftFail" result should be treated as somewhere between a "Fail" and a "Neutral". The domain believes the host is not authorized but is not willing to make that strong of a statement. Receiving software SHOULD NOT reject the message based solely on this result, but MAY subject the message to closer scrutiny than normal.

The domain owner wants to discourage the use of this host and thus desires limited feedback when a "SoftFail" result occurs. For example, the recipient's Mail User Agent (MUA) could highlight the "SoftFail" status, or the receiving MTA could give the sender a message using a technique called "greylisting" whereby the MTA can issue an SMTP reply code of 451 (4.3.0 DSN code) with a note the first time the message is received, but accept it the second time.

...Neutrals aren't just used by lazy admins. I think that my case - where I used a shared hosting company that has both good and bad customers is a perfect example of where a customer such as myself would hesitate to stamp certain emails as a "pass", when same smtp server sends unsolicited email for abusive customers.

I've tested my web hosting company's smtp server, and confirmed that it is not an open relay. By getting on AOL's FBL (Feedback loop), I was able to receive a copy of all spam sent out through my hosting company. The mail headers helped me identify that the outgoing spam is coming from actual customers from the web hosting company, who are using their web sites to send out spam.

Notice that I do use a -all in my SPF record, it is only email originating from the shared web hosting company that I use that I mark as neutral.

v=spf1 +mx:mail.mydomain.com ?mail.sharedhostingcompany.com -all

If I were to avoid any neutral or softfail results, as you suggest, then I would only have the following options - none of which are acceptable to me:

SPF Record                                                    Consequence
+mail.sharedhostingcompany.com -all   Spammers on my hosting company's
                                        site can send mail on my domain's
                                        behalf. My SPF record asserts that
                                        the email is from me.
-mail.sharedhostingcompany.com -all   All email from my web hosting
                                        company's web site (including the
                                        email that I send) is marked as
                                        forgery.
(Deleting my SPF record)        I lose all benefits from SPF
                                  validation, in efforts to make sure
                                  that you can receive my mail.

I do agree with you - that neutrals and softfails are widely overused by lazy administrators. Unfortunately, the Big Boys (Hotmail, Google, etc..) are some of the worst offenders. But as you consider keeping your SPF acceptance policy more aggressive, I hope you consider that my isolated use for neutral results from one of my outgoing email servers is appropriate.

Stephen

Edited by sgeorge

Maybe exclude aol.com and other IP’s from : "but extreme pressure from...

Wed, 31 May 2006 12:54:28 +0000

Author: Roman
Subject: 5621
Posted: 31 May 2006 at 12:54pm

"but extreme pressure from our remote customers who don't give a rat's" - that's another point: WE NEED RFC.

Well, I can only suggest to move obsolete (good adj. for SmartPhones :) equipment to subdomain with wide or no SPF...

Anyway, as long as "big guys" use "?~all" I see no difference between "~?-all" at all.

Edited by Roman

Maybe exclude aol.com and other IP’s from : Dan, about your SFDB usage policy...

Wed, 31 May 2006 12:44:18 +0000

Author: Roman
Subject: 5621
Posted: 31 May 2006 at 12:44pm

Dan, about your SFDB usage policy (as long as opinions are welcomed :) . Consider this:

The main idea of SFDB is to mark WRONG relays. Let's look at 1st point. People usually (I mean IMHO. Myself.) block some domain, let's say bonbon.net not because they are/were subscribed to their mailing lists, but because spam/viruses fakes the return addr.. In other words I block some domain only if
1. SpamHits number is more than
2. The possibility of real business with it's owners is zero (and I have never seen their legitimate mail ever)
3. It has no SPF record (actually my custom error for this rule sounds like: "550 Your domain %Domain% does not have SPF record and/or is Blacklisted...")
So when I mark some server for "some@bonbon.net" there is no chance that I'd mark the REAL bonbon server - it always will be some stupid ADSL user. So it's pretty safe to use (not much or less than others).

The same applies to 5, 6 (in my config - never => I use this rule => only spammers will be marked).
May be true also for 13 and 14 (not 100% sure, needs research).
17 - always bad|stupid boy - must be marked.
20 - well, very strange for me to see this rule at all - "autoinflate" possible.
3, 15, 16 you can do it yourself as 12 - see no reason.

Edited by Roman

Maybe exclude aol.com and other IP’s from : Roman, That was a little reminiscent...

Wed, 31 May 2006 12:18:18 +0000

Author: Desperado
Subject: 5621
Posted: 31 May 2006 at 12:18pm

Roman,

That was a little reminiscent of "Who's on first?"

In principal, I really do agree with your statements and I get very irritated at servers and DNS that are configured wrong or sloppily (Wow - Spell check says that's a word). However, sometimes, it is not laziness but extreme pressure from our remote customers who don't give a rat's you know what about "right" or "wrong" and just want their mail to go through. Also, some of our own sales force want their Sprint PCS to work and their IP is dynamic and even crosses the /24 barrier so what do we do there? I have changed my SPF record over and over again to make up for that. We tried to get the stupid "Smart Phones" to use smpt auth and it fails about 30% of the time (??). I do not have a good answer but I do have a -all in my SPF. I just have to live with the problems it causes if I am claiming to be an anti-Spam crusader.

Maybe exclude aol.com and other IP’s from : Stephen, there is no "legitimate...

Wed, 31 May 2006 11:57:48 +0000

Author: Roman
Subject: 5621
Posted: 31 May 2006 at 11:57am

Stephen, there is no "legitimate reason for creating softfail and neutral spf rules" (in long term perspective)!

First of all (if I remember correctly) origionally "?" is for "testing" and "~" is for domains "where not all clients are upgraded to use correct smtp relays yet". But "smart" guys decided: "I want to participate in antispam and I want 100% deliverability, but I am very lazy... What can I do? Oh! There goes "~/?" !". This sick method is even allowed by http://www.openspf.org/whitepaper.pdf . So we have the illusion that big ISPs support antispam policy, but de facto it is ABSOLUTELY USELESS (if follow original standards). Well, if the biggest ISPs use this sick method I'm constrained to block "?/~". Anyway, what should "a:some ?all" mean? As I can see it mean "host "some" is ok to send mail, and I am in testing phase for all other IPs in the world". When are you gonna finish your endless global testing??? Sorry, this is defenetly not right.

About your particular case. Spam became the huge problem because of insecure smtp standard. But now with current "best practicies" (where the hell new RFC is ?!?) it is a problem only besause man's laziness and incompetence - spammers can only relay their crap through WRONGLY configured servers. So administration task became very clear - do everything RIGHT and block everything WRONG. By assuming that your hosting company could fail in RIGHT authentication of customers and therefore using "?" you actually say: "my hoster (presumably) has WRONG configuration in his servers, so I should also implement WRONG configuration on my side". Then please don't be disappointed when I block you.

If you use anyone's relay, please, make it "pass", make it RIGHT. Yes, if he would fail in authentication (or anything else) - he will be blocked because he did something WRONG. And then he will have a chance to do it RIGHT.

Maybe exclude aol.com and other IP’s from : Stephen, Roger that!

Thu, 25 May 2006 13:19:57 +0000

Author: Desperado
Subject: 5621
Posted: 25 May 2006 at 1:19pm

Stephen,

Roger that!