Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - SPF and Subdomains
  FAQ FAQ  Forum Search   Register Register  Login Login

SPF and Subdomains

 Post Reply Post Reply
Author
Frank Schreier View Drop Down
Newbie
Newbie


Joined: 31 January 2005
Location: Germany
Status: Offline
Points: 3
Post Options Post Options   Thanks (0) Thanks(0)   Quote Frank Schreier Quote  Post ReplyReply Direct Link To This Post Topic: SPF and Subdomains
    Posted: 31 January 2005 at 8:33am
Looks like SPF-filter (2.1.2.395) is not looking at subdomains in a common way.
If we receive mail from "yx@schulverwaltung.bremen.de" it seems the Filter is looking only for "bremen.de" and not "schulverwaltung.bremen.de".

If we check it on http://spf.pobox.com/why.html there is a message "schulverwaltung.bremen.de does not publish SPF records".
There is set up a correct SPF-record for "bremen.de" (everything is working fine with mails from xy@bremen.de), but there are lots of valid and legitimate bremen.de-Subdomains not using SPF yet.

Edited by Frank Schreier
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 31 January 2005 at 3:58pm

Frank,

If there is NO SPF Record for schulverwaltung.bremen.de, is shouls not fail because "NO SPF" passes by default.  Can you post a log entry that show a failure?

 

The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 31 January 2005 at 4:40pm
Frank,

As Dan mentioned, if there is not an SPF record for a domain, SpamFilter should ignore the SPF check for any email address from that domain.

I took a look at the domain, schulverwaltung.bremen.de, but all I see is a SOA record, I believe you're missing the NS records in your configuration. Without the NS records, the DNS subdomain is not configured correctly.

Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Frank Schreier View Drop Down
Newbie
Newbie


Joined: 31 January 2005
Location: Germany
Status: Offline
Points: 3
Post Options Post Options   Thanks (0) Thanks(0)   Quote Frank Schreier Quote  Post ReplyReply Direct Link To This Post Posted: 02 February 2005 at 8:03am
Dan,

here a log entry

01.27.05 14:51:06:745 -- (464) Connection from: 195.37.106.226  -  Originating country : Germany
01.27.05 14:51:06:955 -- (464) Resolving 195.37.106.226 - Not found
01.27.05 14:51:07:045 -- (464) found SPF record: v=spf1 ip4:194.95.254.0/23 mx ptr -all
01.27.05 14:51:07:155 -- (464) SPF query result:
01.27.05 14:51:07:165 -- (464) - SPF analysis for bremen.de done: - fail
01.27.05 14:51:07:165 -- (464) failed SPF test (fail) - Disconnecting 195.37.106.226
01.27.05 14:51:07:165 -- (464) 195.37.106.226 - Mail from: [yx]@schulverwaltung.bremen.de To: [yx]@[yx].de will be rejected
01.27.05 14:51:07:346 -- (464) Disconnect



Roberto,

your statement "Without the NS records, the DNS subdomain is not configured correctly" sounds a little "academic" to us.
It is not *our* domain or subdomain, it is one of our city council. And very important for some of our clients to receive mail from this domains. Surly, we can decide to use this SPF-Filter or not...

But my question is: It seems SPF-Filter in Logsat Spamfilter is handling subdomains in a different way than the "official" tools (http://spf.pobox.com/why.html) are doing? Is it intended or is it a bug. And if no bug, why?

Best regards, Frank
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 02 February 2005 at 5:17pm
Frank,

I believe you were indeed correct, we were wrong in how we analyzed the DNS for schulverwaltung.bremen.de.

We triple-checked the standards and you did indeed found a problem in how SpamFilter ISP handles some SPF records.

The fix is still being tested, as it required some major logic changes in how DNS queries are handled, but we are making it available as a pre-released version in the registered user area of the website. The build is 2.1.2.405.

Thank you for taking the time to report this to us and to insist as you did on making your point, as we may not have found the problem if you had not done so.

Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Frank Schreier View Drop Down
Newbie
Newbie


Joined: 31 January 2005
Location: Germany
Status: Offline
Points: 3
Post Options Post Options   Thanks (0) Thanks(0)   Quote Frank Schreier Quote  Post ReplyReply Direct Link To This Post Posted: 02 February 2005 at 5:33pm
Roberto,

thanks for your fast response.
Back to Top
Matt R View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote Matt R Quote  Post ReplyReply Direct Link To This Post Posted: 12 February 2005 at 11:32pm

Frank could have simply added the hostname or IP block to his base DNS and SPF policy and life would have been good.

The question is when is a subdomain a subdomain and when is it merely a host for the base domain.  This is an ongong debate in SPF discussions still.

When you assume every host is a valid subdomain errors occur and spammers can easily insert a bogus hostname along with using your email addresses and pass your SPF policy now.

With SpamFilter, it was decided before to remove the above loophole, thus improving over the old SPF Standard(current SPF or "classic SFP was developed before it was really used much).  Roberto used a check for an NS record to validate the subdomain and this was good until this thread began the backward steps.  I think if the old logic was put back and in addition to NS records qualifying the subdomain that the absense of a host A records also validated a subdomain, that would have solved Frank's problem and kept the superior implementation of SPF that SpamFilter used.

If we're back to assuming that every host name is a subdomain like the old SPF does because it saved an extra query, then build 405 is a serious down grade.

-Matt

 

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 14 February 2005 at 6:03pm
Matt,

We had to change the way the SPF lookups were performed, as in this case the SPF site was specifically saying "there are no SPF records for this entry" (schulverwaltung.bremen.de) while SpamFilter was instead rejecting emails from it since according to SpamFilter the SPF records for the bremen.de domain did not authorize that host. We could not go against the SPF rules, so had back down from the way we interpreted the results...
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Matt R View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote Matt R Quote  Post ReplyReply Direct Link To This Post Posted: 15 February 2005 at 7:39am

In this example there is no A record or host record for bremen.de and there is an MX record for schulverwaltung.bremen.de.  So what really needed to be done was adjust SpamFilter logic to expand on defining a valid subdomain as having an NS Record OR MX record or when not being an A record in the parent domain while having any record when doing an ALL record DNS query.

So my question is are we back to square one where every hostname is assumed to be a subdomain or is SpamFilter more intelligently determining that this host is not part of the bremen.de domain because it has not host A record?

 

 

 

 

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 15 February 2005 at 6:15pm
This is going to be a no-win battle...
Back in January on the dates this thread started, the "official" SPF site and test said specifically that "yx@schulverwaltung.bremen.de" should have been accepted. There was no hesitation in that response, forcing us to change SpamFilter's interpretation of SPF. We could not just go about on our own, rejecting emails that the SPF site specifically said were to be accepted...

But Matt is right in his last posts. With the way SpamFilter was modifed in build 405, any spammer can use a host to bypass the domain's SPF record.

THis is never going to end because... we just re-checked the official SPF site, and now the "this MUST be accepted" result for schulverwaltung.bremen.de is not so anymore... the spf.pobox.com site now is more flexible saying that depending on how SPF is implemented, anti-spam software may reject the email... this is a big change from "it passes SPF rules" or "it myst be accepted".

SPF is ever changing...

We will now once more go back to the drawing board and see what can be done to tighten SpamFilter once more....
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Matt R View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote Matt R Quote  Post ReplyReply Direct Link To This Post Posted: 17 February 2005 at 9:27pm

I think in terms of how a common email system usually gets deployed. That's why I thought the "Check for MX record" should actually check the actual subdomain or domain and not assume everything to the right of the @ symbol is a subdomain or domain. 

Originally, SPF and some others thought this was cool because it means less DNS queries, less logic to process and better performance.  I feel, however, that this accuracy is more important than saving one DNS query if needed, when we're talking about rejecting someone's valuable communications. 

The underlying question is: When is a subdomain really a subdomain and when is it a host in a domain or valid subdomain? Or how do we validate a subdomain (in the name of improved accuracy)

Robert, we originally tried to define a domain as having an NS record. So, we would test everything to the right of the @ symbol to see if it had an NS record so we knew it was a domain, if not we dropped the host name and started after the first "." and assumed that was the domain or did the same check to see if it was a domain, until we had a valid domain.  

Here's a better way that seems to have always worked: Do a DNS ALL (ANY RECORD) Query against the substring. If it has ANY record than we can say it is a valid subdomain.  If not we need to drop the host name and either test the remaining string to validate that it's a subdomain or domain or assume that it must be a domain.  Obviously it would be most accurate to continue until we have validated the domain or subdomain and then check for SPF or MX record in the case of MX record checking feature.

Back to Top
pcmatt View Drop Down
Senior Member
Senior Member
Avatar

Joined: 15 February 2005
Location: United States
Status: Offline
Points: 116
Post Options Post Options   Thanks (0) Thanks(0)   Quote pcmatt Quote  Post ReplyReply Direct Link To This Post Posted: 17 February 2005 at 9:54pm

OOPS!  CLARIFICATION OF "Here's a better way.."

If DNS ALL query returns an A record then we know it is NOT a subdomain, but a host in a domain so the domain needs to be determined and queried..  For example: www.idp.net returns it's A host record when you do a DNS ALL (ANY) query and schulverwaltung.bremen.de returns an MX record when you do a DNS ALL(ANY) query and is NOT found as a host record, thus proving that it IS a subdomain.

-Matt R.

 



Edited by pcmatt
-Matt R
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.094 seconds.