Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - Messages quarantined even if sender white
  FAQ FAQ  Forum Search   Register Register  Login Login

Messages quarantined even if sender white

 Post Reply Post Reply
Author
Terry View Drop Down
Senior Member
Senior Member


Joined: 06 February 2005
Status: Offline
Points: 155
Post Options Post Options   Thanks (0) Thanks(0)   Quote Terry Quote  Post ReplyReply Direct Link To This Post Topic: Messages quarantined even if sender white
    Posted: 08 August 2007 at 9:13am

We have experienced a problem where an email was quarantined even though the sender was whitelisted by the recipient.  In looking at the logs it appears this may occur because the program must check the SFDB before the whitelist...

08/02/07 15:34:32:263 -- (1104) Connection from: 65.54.246.107  -  Originating country : United States
08/02/07 15:34:32:560 -- (1104) Resolving 65.54.246.107 - bay0-omc1-s35.bay0.hotmail.com
08/02/07 15:34:32:826 -- (1104) - SFDB filter match - relevance:12
08/02/07 15:34:32:826 -- (1104) 65.54.246.107 - Mail from: vigilante1@msn.com To: Renee.Dowlin@portofportland.com will be rejected
08/02/07 15:34:33:638 -- (2464) Mail from: xiYFUTHMy@tippicanoe.net
08/02/07 15:34:33:638 -- (2464) 66.29.125.218 - Mail from: xiYFUTHMy@tippicanoe.net To: lawrej@portptld.com will be rejected
08/02/07 15:34:33:857 -- (1104) Start virus scan
08/02/07 15:34:33:857 -- (3912) Connection from: 75.80.181.25  -  Originating country : United States
08/02/07 15:34:33:966 -- (1104) Starting quarantine procedures
08/02/07 15:34:33:966 -- (1104) Created thread (844) to add email to quarantine
08/02/07 15:34:33:998 -- (1104) Blacklist cache - Added 65.54.246.107 to limbo
08/02/07 15:34:33:998 -- (1104) Disconnect

 

the "vigilante1" email address was whitelisted for the recipient already when this quarantine occurred so it should have checked the whitelist first.......

 

We are on release 3.5.4.692



Edited by Terry
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2007 at 10:30am

I am not sure if this applies but ...

// New to VersionNumber = '3.5.4.700';
{TODO -cNew : PDF image filter now scans, in addition to previous content types, also "application/octet-stream"}
{TODO -cNew : SpamFilter will now block emails that contain an empty, blank body and also a PDF attachment, the new setting in the .ini file is on by default: BlockBlankEmailsWithPDFAttachments=true}
{TODO -cFix : If an email session was whitelisted due to a whitelist keyword match, if further emails were sent during the same SMTP session, they could be blacklisted due to a falase keyword match}

Also, How did you whitelist the sender?



Edited by Desperado
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
Terry View Drop Down
Senior Member
Senior Member


Joined: 06 February 2005
Status: Offline
Points: 155
Post Options Post Options   Thanks (0) Thanks(0)   Quote Terry Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2007 at 10:35am

I don't think that those really apply.  The sender was whitelisted by the user by releasing email from quarantine weeks earlier....here is a log entry from several days earlier that shows the fact that the sender was whitelisted for this recipient

07/30/07 08:59:54:218 -- (3040) Bypassed all rules for: Renee.Dowlin@portofportland.com from vigilante1@msn.com ( AutoWhiteList Force Delivery)
07/30/07 08:59:54:281 -- (3040) Bypassed all rules for: sam.hartsfield@portofportland.com from vigilante1@msn.com

I am really think it must be the order of checking....

 

Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2007 at 10:44am

Hmmm I believe a whitelist entry will over-ride anything but there were some AutoWhiteList issues that were also resolved after the 692 build.  Or .... there may be a bug I have not experienced yet.

The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
RBarrow View Drop Down
Groupie
Groupie
Avatar

Joined: 22 September 2005
Status: Offline
Points: 45
Post Options Post Options   Thanks (0) Thanks(0)   Quote RBarrow Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2007 at 4:27pm

We have reports from our users indicating the same problem (build 700).  In researching this report, we think found a seemingly unrelated issue which may give the appearance of a whitelisted email not coming through. In situations where an email was addressed to several recipients and one or two had the sender whitelisted while the others did not, the email is sent to the whitelisted recipients (as it should be) but the email is also quarantined for ALL the recipients in the list instead of just those without a whitelist entry.

This gives the appearance of a previously whitelisted address being blocked when the user checks the quarantine later.

This situation is causing us a LOT of problems...!!!

Roy

The mail is



Edited by RBarrow
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2007 at 4:28pm
Terry,

We can't replicate this, the email should have been whitelisted. The filtering order can be found at www.logsat.com/spamfilter/forums/forum_posts.asp?TID=5171#77 76.

As you can see, most whitelists are always checked before the blacklists. If you're using SpamFilter ISP "standard", can you please edit the file:

\SpamFilter\Domains\SFI\Filters.ini

and check to ensure that WL_AuthorizedTOEmailsFileName entry has the full path to the WL_AuthorizedTOEmails.txt file (drive letter + path), and not just a relative filename by itself?
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Terry View Drop Down
Senior Member
Senior Member


Joined: 06 February 2005
Status: Offline
Points: 155
Post Options Post Options   Thanks (0) Thanks(0)   Quote Terry Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2007 at 9:23pm

Roberto, here is what I have in that line

WL_AuthorizedTOEmailsFileName=
WL_AutoWhiteListForceDeliveryFileName=d:\program files\spamfilter\AutoWhiteListForceDelivery.txt

 

does this need to be changed?

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 08 August 2007 at 10:17pm
No, that entry should be fine. Could you please email us at support at logsat dot com that AutoWhiteListForceDelivery.txt file, and the \SpamFilter\Domains\SFI\Filters.ini as well?
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Terry View Drop Down
Senior Member
Senior Member


Joined: 06 February 2005
Status: Offline
Points: 155
Post Options Post Options   Thanks (0) Thanks(0)   Quote Terry Quote  Post ReplyReply Direct Link To This Post Posted: 09 August 2007 at 9:20am
Those files have now been emailed to you...in reading the previous posts I did notice that the line in our filters.ini that was for WL_AuthorizedTOEmailsFileName=
is blank...does the other line supercede that entry?
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 09 August 2007 at 12:53pm
Terry,

We received the files, and everything does indeed look in order. We're trying to determine what happened, as that email should have indeed been whitelisted.

In regards to the "WL_AuthorizedTOEmailsFileName" entry, if you are not using the "AuthorizedTO" list, it's normal for it to be blank. That filter is used only if you wish to provide SpamFilter with a list of all the valid email accounts on your system. If you do provide it, SpamFilter will only accept emails for those users and will reject all others.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2007 at 12:00pm
We've been looking thru several days of logfiles, and have found a bug in SpamFilter which could have caused the AutoWhiteListForceDelivery.txt to become corrupted.

The following log entries do point to a possible problem:

07/16/07 07:36:36:921 -- (3400) Exception occurred during FindMatchInStringList: Invalid pointer operation
07/16/07 07:36:36:953 -- (2528) Reloading file for tblWL_AutoWhiteListForceDelivery: AutoWhiteListForceDelivery.txt


We've fixed the bug in red above with build 3.5.4.705 that has just been uploaded in the registered user area of the website.

Please note that we are still not 100% certain that the above bug was indeed the cause for the AutoWhiteList corruption. We are currently examining the other logs to see if this issue occurred more than once.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Terry View Drop Down
Senior Member
Senior Member


Joined: 06 February 2005
Status: Offline
Points: 155
Post Options Post Options   Thanks (0) Thanks(0)   Quote Terry Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2007 at 12:42pm
Thanks Roberto....I want you to know how much I appreciate the great support we have had for this product.  Whenever we have had a problem you have been right on it to either fix it or show us what we did wrong...Great job..I want you to know I have recommended this product many time to others....
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 12 August 2007 at 8:13pm
Thank you for the support!
We just finished going thru your logs, and saw the exact same erros on the 9th, 11th, 16th and 18th.

In all cases the two events I mentioned in the above post in red and green occur within 10-50 milliseconds of each other. While we still cannot replicate the issue in the lab, I can confirm that there was a bug in the code isolation within SpamFilter that could have caused it to happen. As we found 5 instances (two of them on happened on the 11th), I'm at this point pretty certain that this bug should be what caused you to loose entries in the AutoWhiteListForceDelivery.txt file. Build 705 that we uploaded should have taken care of it.

Please do let us know if you see further problems, as when there's an error we can't duplicate in the lab, there's always a level of uncertainty that we cannot avoid.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
pcmatt View Drop Down
Senior Member
Senior Member
Avatar

Joined: 15 February 2005
Location: United States
Status: Offline
Points: 116
Post Options Post Options   Thanks (0) Thanks(0)   Quote pcmatt Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2007 at 8:18am

Roberto,

This issue still exists in build 704.  Had a complaint on this today.

This is related to your "email splitting" logic.   What happens is that SpamFilter correctly splits the email for the purpose of sending only to whitelisted users and not others in the message, however, since one or more recipients are not whitelisted EVERYONE gets a copy of the item in their quarantines.  
 
So the email being "split" is processed correctly in terms of delivering the message, however, the quarantine entries are not split, causing a confusing entry in each of the whitelisted recipient's quarantines.
 
-Matt
-Matt R
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 13 August 2007 at 9:54pm
Matt,

This issue is different from the one reported by Terry. We've however patched this one too, and the fix will be included in the next released build. Please contact us if you wish to receive this intermediate build privately.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 14 August 2007 at 7:13am
Update. Due to popular demand via emails Smile we pre-released build 3.5.4.707 in the registered user area. The release notes are as usually posted on our website. This is the relevant section:

// New to VersionNumber = '3.5.4.707';
{TODO -cNew : Added option in SpamFilter.ini file: HideXSFWhiteListedReasonHeader}
{TODO -cNew : In SFE, SpamFilter is able to now startup even if the database server is unavailable}
{TODO -cFix : If a spam email is split so that it is delivered for whitelisted recipients but blocked for the rest, it was still being stored in the quarantine database for all receipients, including the whitelisted ones}
{TODO -cFix : The HTML parser used to detect blank html emails was expecting legitimate opening and closing html tags to define text, and was ignoring any text outside these tags. This could cause very short html emails to appear blank if they were not following correct html syntax. We are now auto-fixing the invalid html code to be less restrictive}

// New to VersionNumber = '3.5.4.705';
{TODO -cNew : "Exception occurred during FindMatchInStringList: Invalid pointer operation" errors could cause problems with the reloading of some black / white lists}
{TODO -cNew : Added the logging of the filesize when reloading the black / white list files}
{TODO -cNew : The SURBL blacklist is not being automatically sorted to allow user-defined order}
{TODO -cNew : Added options in [authentication settings] of SpamFilter.ini: ActiveDirectoryAuthAppendDefaultDomain, ActiveDirectoryAuthPrefixDefaultDomain to automatically append or prefix the default domain when authenticating users via SMTP AUTH}
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.078 seconds.