Spam Filter ISP Forums : AutoWhiteList Force Delivery error

AutoWhiteList Force Delivery error : One issue with the autowhitelistdelivery...

Sat, 10 Jan 2009 14:08:25 +0000

Author: WebGuyz
Subject: 6593
Posted: 10 January 2009 at 2:08pm

One issue with the autowhitelistdelivery mechanism is there is no verification that the allowed domain is coming from a valid server.

If I have a entry: *@yahoo.com|*@mydomain.com

Then anyone spoofing emails from yahoo.com will be allowed thru. if you did an SPF check after seeing this whitelisted entry and verifying the email was really coming from a valid email in that domain, it would be much more accurate. Most of the larger ISP's have SPF records for their mail servers. This would make the autowhitelistentry filter much better.

AutoWhiteList Force Delivery error : I agree: DO NOT ADD user@domain.com|user@domain.com...

Fri, 09 Jan 2009 09:57:43 +0000

Author: Desperado
Subject: 6593
Posted: 09 January 2009 at 9:57am

I agree: DO NOT ADD user@domain.com|user@domain.com where the sender is the recipient to the autowhitelist when such emails are force-delivered.

AutoWhiteList Force Delivery error : Rudaf, You're absolutely...

Thu, 08 Jan 2009 18:35:16 +0000

Author: LogSat
Subject: 6593
Posted: 08 January 2009 at 6:35pm

Rudaf,

You're absolutely correct on the ISSUE 2 - I missed the fact that you were using SFE.

For the ISSUE 1, in most cases, the emails where the "to" address and the "from" address are exactly the same fall in two major categories.

1. The user sends an email to themselves. In this case, the user will be going thru either your mail server or thru SpamFilter, in which case the email will be delivered without problems.

2. The user goes thru a web interface to send himself an email, for example by going to an online site that allows the browser to send the article to an email address. Many (poorly managed) sites will ask for the browser's email address and for the recipient's. The website will then (administrators have no clue of what SPF is...) forge the email by sending it on the user's behalf to the user themselves. As you obviously for example will not allow CNN.COM (yes... they are one such site were the web administrators should really talk to their email administrators...) to send email on your behalf in your SPF record, the email will be blocked.

Unfortunately case #2 is very common, and we are seeing it as being a problem more often.

Ed_K's suggestion is in our opinion a very valid one, and we may actually modify SpamFilter so that in the future it won't add entries in the form user@domain.com|user@domain.com where the sender is the recipient to the autowhitelist when such emails are force-delivered.

We're absolutely open to everyone's opinions in regards to this.

AutoWhiteList Force Delivery error : Thank you Roberto, for your prompt...

Thu, 08 Jan 2009 06:32:10 +0000

Author: rudaf
Subject: 6593
Posted: 08 January 2009 at 6:32am

Thank you Roberto, for your prompt support.

LogSat wrote:

The email in this case was delivered due to an entry in the AutoWhitelistForceDelivery whitelist.

[...]

Since spammers will often fake the sender to be the same as the recipient, we recommend *not* to force the delivery of such emails if they are in the quarantine.

ISSUE 1:

That's correct but I do not agree on the system's behavior and on your suggestion as defintive solution.

1) SPF policies and related SF filter, are implemented exactly to avoid domain forgery

2) Automatic whitelisting entries on path quarantine - force delivery - autowhitelist is a wonderful feature that solve the serious problem of false positive, a feature that a very few antispam software can offer

3) Autowhitelist can be managed by user simply forcing-delivery through quarantine web interface, while understandig that domain forgery is in conflict with autowhitelist entries is an administrator matter with issues from users looping.

E.g:

a) user force deliver mail user@domain.com to user@domain.com since quarantined for a keywords match

b) mail from user@domain.com to user@domain.com is autowhitelisted: antispam rules will be skipped

c) domain.com registrant implements SPF record to avoid forgery

d) spammers forge the domain and send spam as coming from user@domain.com to user@domain.com

e) SFE check SPF and verify that such email is coming from a not allowed IP: mail will be rejected

f) SFE check for Autowhitelist file and find that such a mail, since mach the white list rule, has to bypass all rules and than deliver the spam

e) User claim to antispam manager that now is receiving a lot of mails from itself with a lot of spam. He's going to think that the filter unworths the cost and that SPF policies are onother waste of time

g) Antispam Manager deletes the entry user@domain.com|user@domain.com

h) User force deliver another user@domain.com locked in quarantine area due to embedded signature in a pdf

i) and movie start again......

Both features (Autowhite and SPF) have to work together and not one against the other.

That's why you should plan to modify the sequence and provide the SPF check as subsequent or with higher priority on the other one since the SPF control is the only antispam policy totally based on domain registrant wishes: if i, domain's registrant, declare that an email from such a domain has to come from certain IPs, doesn't matter if there's an entry that, despite of the SPF entry, bypass it simply upon a rule that play as spammers do since they know that mail from = mail to is often excluded from spam check.

SPF has to overhelming any other rules.

ISSUE 2:

LogSat wrote:

If so, you can simply remove that entry from the file, SpamFilter will automatically reload it within 60 seconds.

We have SFE with whitelist on DB. We removed the entry from the file but it did not affected the db entry.

We had to delete it directly from the db.

AutoWhiteList Force Delivery error : Thank you very much for your assistance,...

Wed, 07 Jan 2009 18:07:51 +0000

Author: Ed_K
Subject: 6593
Posted: 07 January 2009 at 6:07pm

Thank you very much for your assistance, Roberto, your response was correct for my issue.

I have written a FOR loop that will find entries like this and output them to a text file. Open a command prompt and change to the directory that contains your AutoWhiteListForceDelivery.txt, then run this command all on one line:

FOR /F "tokens=1,2 delims=|" %i IN (AutoWhiteListForceDelivery.txt) DO @IF %i EQU %j ECHO %i>>output.txt

The output.txt file it creates will have one entry for each of the email addresses that have one of these entries that causes the problem. You then have to manually remove the lines based on the results, but since I knew I wouldn't have that many, I did not take the time to script that. Hopefully this will save someone else some time, too.

AutoWhiteList Force Delivery error : The email in this case was delivered...

Wed, 07 Jan 2009 16:45:41 +0000

Author: LogSat
Subject: 6593
Posted: 07 January 2009 at 4:45pm

The email in this case was delivered due to an entry in the AutoWhitelistForceDelivery whitelist.

Any time an email is force-delivered from the quarantine area because it is blocked incorrectly, SpamFilter will automatically match the sender with the recipient, so that in the future all emails from that sender to that recipient will be automatically whitelisted. At some point, you may have force-delivered an email from forgedaccount@forgeddomain to forgedaccount@forgeddomaint. This caused the entry:
forgedaccount@forgeddomain|forgedaccount@forgeddomain
to be added to your AutoWhitelistForceDelivery.txt file, and will cause all emails from forgedaccount@forgeddomain to sforgedaccount@forgeddomain to be whitelisted, even if they are spam.

Without seing your actual file, we're only assuming right now this is the problem. If so, you can simply remove that entry from the file, SpamFilter will automatically reload it within 60 seconds.

Since spammers will often fake the sender to be the same as the recipient, we recommend *not* to force the delivery of such emails if they are in the quarantine.

AutoWhiteList Force Delivery error : Yes, the "forgeddomain"...

Wed, 07 Jan 2009 09:06:12 +0000

Author: rudaf
Subject: 6593
Posted: 07 January 2009 at 9:06am

Yes, the "forgeddomain" does implement spf record.

we are going to upload the current log in your FTP area: take a look at thread # (5808).

The log keeps on reporting AutoWhiteList force delivery but no white listing rule is currently enabled in order to let bybass all rules for mail where mail from = mail to or where mail from domain = mail to domain. The latter was enabled (*@mydomain | *@mydomain) but days ago has been removed with no resolution of the problem here quoted.

Anyway we cannot enable the rule "spam where mail to = mail from" since often users send mail to themselves; that's why we prefer SPF policies.

Regards

AutoWhiteList Force Delivery error : Hello,We're encountering...

Tue, 06 Jan 2009 20:40:10 +0000

Author: Ed_K
Subject: 6593
Posted: 06 January 2009 at 8:40pm

Hello,

We're encountering a very similar issue at one of my clients who uses SpamFilter ISP. Our OS and build are identical to the OP's except we are using Standard as opposed to Enterprise.

In this instance, the offending spam is spoofing the recipient's email address. This client's DNS provider does not allow SPF or TXT records in their DNS (I've strongly recommended that they change providers) so instead I've enabled the 'Reject if "Mail From" = "Mail To"' option under Filter Settings.

I expected that to fix it, so when it didn't, I started looking at the headers. Here is an excerpt:

X-SF-WhiteListedReason: AutoWhiteList Force Delivery
X-Rejection-Reason: 8 - 557 client's custom SPAM message

I'm not sure exactly what AutoWhiteList is, but I don't see any rule in any of our White List settings that should be allowing this mail. I found that we were not logging, so I have just turned it on and will restart the service after production hours. I'll gladly forward the logs I gather to support unless a resolution is proposed in the meantime. Thanks a lot! -Ed

Edited by Ed_K - 06 January 2009 at 8:41pm

AutoWhiteList Force Delivery error : rudaf, Did the domain "forgeddomain"...

Tue, 06 Jan 2009 17:43:25 +0000

Author: LogSat
Subject: 6593
Posted: 06 January 2009 at 5:43pm

rudaf,

Did the domain "forgeddomain" actually implement SPF in their DNS? Please do note that SPF will be able to block forged spam only if the domain does have valid SPF records in their DNS.

If that's affirmative, could you please zip and email us at support at logsat.com SpamFilter's activity logfile for a day this happened, along with the to/from email addresses involved?

if the zip is over 5MB in size, I'll provide you with our FTP login info via a PM.

AutoWhiteList Force Delivery error : SFE 4.0.1.785 SQL SERVER 2000...

Mon, 05 Jan 2009 18:43:06 +0000

Author: rudaf
Subject: 6593
Posted: 05 January 2009 at 6:43pm

SFE 4.0.1.785

SQL SERVER 2000 STD

OS WIN 2KSVR SP4

We are subjected to domain forgery spam. months ago we successfully implemented SPF rules.

Now a lot of forged domains and accounts spam is passing SFE (eg. forgedaccount@forgeddomain to forgedaccount@forgeddomain) even if the SPF filter is enabled.

The log reported AutoWhiteList Force Delivery since in the autowhite list there was the rule *@MYDOMAIN | *@MYDOMAIN

Although useful to avoid intra domain false positive, we removed such a rule throuh SFE configuration control panel, but forged mails keep to pass the filters (AutoWhiteList Force Delivery), even if the log reports SPF Fail and mail destinated to be quatantined.

Any idea?