Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - AutoWhiteList Force Delivery error
  FAQ FAQ  Forum Search   Register Register  Login Login

AutoWhiteList Force Delivery error

 Post Reply Post Reply
Author
rudaf View Drop Down
Newbie
Newbie


Joined: 04 July 2007
Location: Italy
Status: Offline
Points: 38
Post Options Post Options   Thanks (0) Thanks(0)   Quote rudaf Quote  Post ReplyReply Direct Link To This Post Topic: AutoWhiteList Force Delivery error
    Posted: 05 January 2009 at 6:43pm
SFE 4.0.1.785
SQL SERVER 2000 STD
OS WIN 2KSVR SP4
 
We are subjected to domain forgery spam. months ago we successfully implemented SPF rules.
 
Now a lot of forged domains and accounts spam is passing SFE (eg. forgedaccount@forgeddomain to forgedaccount@forgeddomain) even if the SPF filter is enabled.
 
The log reported AutoWhiteList Force Delivery since in the autowhite list there was the rule *@MYDOMAIN | *@MYDOMAIN
 
Although useful to avoid intra domain false positive, we removed such a rule throuh SFE configuration control panel,  but forged mails keep to pass the filters (AutoWhiteList Force Delivery), even if the log reports SPF Fail and mail destinated to be quatantined.
 
Any idea?
 
 
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4068
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 06 January 2009 at 5:43pm
rudaf,

Did the domain "forgeddomain" actually implement SPF in their DNS? Please do note that SPF will be able to block forged spam only if the domain does have valid SPF records in their DNS.

If that's affirmative, could you please zip and email us at support at logsat.com SpamFilter's activity logfile for a day this happened, along with the to/from email addresses involved?

if the zip is over 5MB in size, I'll provide you with our FTP login info via a PM.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Ed_K View Drop Down
Newbie
Newbie


Joined: 06 January 2009
Status: Offline
Points: 3
Post Options Post Options   Thanks (0) Thanks(0)   Quote Ed_K Quote  Post ReplyReply Direct Link To This Post Posted: 06 January 2009 at 8:40pm
Hello,

We're encountering a very similar issue at one of my clients who uses SpamFilter ISP. Our OS and build are identical to the OP's except we are using Standard as opposed to Enterprise.

In this instance, the offending spam is spoofing the recipient's email address. This client's DNS provider does not allow SPF or TXT records in their DNS (I've strongly recommended that they change providers) so instead I've enabled the 'Reject if "Mail From" = "Mail To"' option under Filter Settings.

I expected that to fix it, so when it didn't, I started looking at the headers. Here is an excerpt:

X-SF-WhiteListedReason: AutoWhiteList Force Delivery
X-Rejection-Reason: 8 - 557 client's custom SPAM message

I'm not sure exactly what AutoWhiteList is, but I don't see any rule in any of our White List settings that should be allowing this mail. I found that we were not logging, so I have just turned it on and will restart the service after production hours. I'll gladly forward the logs I gather to support unless a resolution is proposed in the meantime. Thanks a lot! -Ed


Edited by Ed_K - 06 January 2009 at 8:41pm
Back to Top
rudaf View Drop Down
Newbie
Newbie


Joined: 04 July 2007
Location: Italy
Status: Offline
Points: 38
Post Options Post Options   Thanks (0) Thanks(0)   Quote rudaf Quote  Post ReplyReply Direct Link To This Post Posted: 07 January 2009 at 9:06am
Yes, the "forgeddomain" does implement spf record.
 
we are going to upload the current log in your FTP area: take a look at thread # (5808).
 
The log keeps on reporting AutoWhiteList force delivery but no white listing rule is currently enabled in order to let bybass all rules for mail where mail from = mail to or where mail from domain = mail to domain. The latter was enabled (*@mydomain | *@mydomain) but days ago has been removed with no resolution of the problem here quoted.
 
Anyway we cannot enable the rule "spam where mail to = mail from" since often users send mail to themselves; that's why we prefer SPF policies. 
 
Regards
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4068
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 07 January 2009 at 4:45pm
The email in this case was delivered due to an entry in the AutoWhitelistForceDelivery whitelist.

Any time an email is force-delivered from the quarantine area because it is blocked incorrectly, SpamFilter will automatically match the sender with the recipient, so that in the future all emails from that sender to that recipient will be automatically whitelisted. At some point, you may have force-delivered an email from forgedaccount@forgeddomain to forgedaccount@forgeddomaint. This caused the entry:
forgedaccount@forgeddomain|forgedaccount@forgeddomain
to be added to your AutoWhitelistForceDelivery.txt file, and will cause all emails from forgedaccount@forgeddomain to sforgedaccount@forgeddomain to be whitelisted, even if they are spam.

Without seing your actual file, we're only assuming right now this is the problem. If so, you can simply remove that entry from the file, SpamFilter will automatically reload it within 60 seconds.

Since spammers will often fake the sender to be the same as the recipient, we recommend *not* to force the delivery of such emails if they are in the quarantine.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Ed_K View Drop Down
Newbie
Newbie


Joined: 06 January 2009
Status: Offline
Points: 3
Post Options Post Options   Thanks (0) Thanks(0)   Quote Ed_K Quote  Post ReplyReply Direct Link To This Post Posted: 07 January 2009 at 6:07pm
Thank you very much for your assistance, Roberto, your response was correct for my issue.

I have written a FOR loop that will find entries like this and output them to a text file. Open a command prompt and change to the directory that contains your AutoWhiteListForceDelivery.txt, then run this command all on one line:

FOR /F "tokens=1,2 delims=|" %i IN (AutoWhiteListForceDelivery.txt) DO @IF %i EQU %j ECHO %i>>output.txt


The output.txt file it creates will have one entry for each of the email addresses that have one of these entries that causes the problem. You then have to manually remove the lines based on the results, but since I knew I wouldn't have that many, I did not take the time to script that. Hopefully this will save someone else some time, too.
Back to Top
rudaf View Drop Down
Newbie
Newbie


Joined: 04 July 2007
Location: Italy
Status: Offline
Points: 38
Post Options Post Options   Thanks (0) Thanks(0)   Quote rudaf Quote  Post ReplyReply Direct Link To This Post Posted: 08 January 2009 at 6:32am
Thank you Roberto, for your prompt support.
 
Originally posted by LogSat LogSat wrote:

The email in this case was delivered due to an entry in the AutoWhitelistForceDelivery whitelist.
 
[...]

Since spammers will often fake the sender to be the same as the recipient, we recommend *not* to force the delivery of such emails if they are in the quarantine.
 
ISSUE 1:
 
That's correct but I do not agree on the system's behavior and on your suggestion as defintive solution.
 
1) SPF policies and related SF filter, are implemented exactly to avoid domain forgery
2) Automatic whitelisting entries on path quarantine - force delivery - autowhitelist is a wonderful feature that solve the serious problem of false positive, a feature that a very few antispam software can offer
3) Autowhitelist can be managed by user simply forcing-delivery through quarantine web interface, while understandig that domain forgery is in conflict with autowhitelist entries is an administrator matter with issues from users looping.
E.g:
a) user force deliver mail user@domain.com to user@domain.com since quarantined for a keywords match
b) mail from user@domain.com to user@domain.com is autowhitelisted: antispam rules will be skipped
c) domain.com registrant implements SPF record to avoid forgery
d) spammers forge the domain and send spam as coming from user@domain.com to user@domain.com
e) SFE check SPF and verify that such email is coming from a not allowed IP: mail will be rejected
f) SFE check for Autowhitelist file and find that such a mail, since mach the white list rule, has to bypass all rules and than deliver the spam 
e) User claim to antispam manager that now is receiving a lot of mails from itself with a lot of spam. He's going to think that the filter unworths the cost and that SPF policies are onother waste of time
g) Antispam Manager deletes the entry user@domain.com|user@domain.com
h) User force deliver another user@domain.com locked in quarantine area due to embedded signature in a pdf
i) and movie start again......
 
Both features (Autowhite and SPF) have to work together and not one against the other.
 
That's why you should plan to modify the sequence and provide the SPF check as subsequent or with higher priority on the other one since the SPF control is the only antispam policy totally based on domain registrant wishes: if i, domain's registrant, declare that an email from such a domain has to come from certain IPs, doesn't matter if there's an entry that, despite of the SPF entry, bypass it simply upon a rule that play as spammers do since they know that mail from = mail to is often excluded from spam check.
SPF has to overhelming any other rules.
 
ISSUE 2:
 
Originally posted by LogSat LogSat wrote:

If so, you can simply remove that entry from the file, SpamFilter will automatically reload it within 60 seconds.
 
We have SFE with whitelist on DB. We removed the entry from the file but it did not affected the db entry.
We had to delete it directly from the db.
 
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4068
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 08 January 2009 at 6:35pm
Rudaf,

You're absolutely correct on the ISSUE 2 - I missed the fact that you were using SFE.

For the ISSUE 1, in most cases, the emails where the "to" address and the "from" address are exactly the same fall in two major categories.

1. The user sends an email to themselves. In this case, the user will be going thru either your mail server or thru SpamFilter, in which case the email will be delivered without problems.

2. The user goes thru a web interface to send himself an email, for example by going to an online site that allows the browser to send the article to an email address. Many (poorly managed) sites will ask for the browser's email address and for the recipient's. The website will then (administrators have no clue of what SPF is...) forge the email by sending it on the user's behalf to the user themselves. As you obviously for example will not allow CNN.COM (yes... they are one such site were the web administrators should really talk to their email administrators...) to send email on your behalf in your SPF record, the email will be blocked.

Unfortunately case #2 is very common, and we are seeing it as being a problem more often.

Ed_K's suggestion is in our opinion a very valid one, and we may actually modify SpamFilter so that in the future it won't add entries in the form user@domain.com|user@domain.com where the sender is the recipient to the autowhitelist when such emails are force-delivered.

We're absolutely open to everyone's opinions in regards to this.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 09 January 2009 at 9:57am
I agree: DO NOT ADD user@domain.com|user@domain.com where the sender is the recipient to the autowhitelist when such emails are force-delivered.
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2009 at 2:08pm
One issue with the autowhitelistdelivery mechanism is there is no verification  that the allowed domain is coming from a valid server.
 
If I have a entry:  *@yahoo.com|*@mydomain.com
 
Then anyone spoofing emails from yahoo.com will be allowed thru. if you did an SPF check after seeing this whitelisted entry and verifying the email was really coming from a valid email in that domain, it would be much more accurate. Most of the larger ISP's have SPF records for their mail servers. This would make the autowhitelistentry filter much better.
 
 
http://www.webguyz.net
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.063 seconds.