Spam Filter ISP Forums : Regarding Spam Filter 2.6.3.487

Regarding Spam Filter 2.6.3.487 : Yeap, we agree, thus to address...

Mon, 09 Jan 2006 07:18:22 +0000

Author: LogSat
Subject: 5420
Posted: 09 January 2006 at 7:18am

Yeap, we agree, thus to address the problem we release yesterday build 516 as follows :

// New to VersionNumber = '2.7.1.516';
{TODO -cFix : SURBL filter could take several minutes to be process emails with hundreds of URLs if DNS server is down, possibly causing re-sends of the email. Improved lookups by removing duplicate TLDs and added global timeout to avoid problem}
{TODO -cFix : SpamFilter version number and blacklist/whitelist loading indicators are missing in logs when SpamFilter service is started}
{TODO -cNew : Added "Notice - IdleDisconnectMinutesTimeout reached. Removing threadID" log entry when disconnecting idle threads}

Regarding Spam Filter 2.6.3.487 : LogSat wrote: The email you...

Mon, 09 Jan 2006 02:44:03 +0000

Author: mikek
Subject: 5420
Posted: 09 January 2006 at 2:44am

LogSat wrote:

The email you forwarded us from Sony BMG contains 321 URLs in it. This causes SpamFilter to perform 321 DNS lookups.

321 URLs with different domains? I shouldn't think so. I guess a solution would be to filter out duplicates before checking them against SURBL, or set up a cache of SURBL lookups (for 10 minutes or so).

Just my 2 cents.

Cheers!

Regarding Spam Filter 2.6.3.487 : Sara,One of SpamFilter's...

Sun, 08 Jan 2006 19:00:18 +0000

Author: LogSat
Subject: 5420
Posted: 08 January 2006 at 7:00pm

Sara,

One of SpamFilter's many filters is the SURBL check. This filter looks at every single URL inside a message, and checks each one against DNS blacklists to see if domain name has been used to spam before. If so, the email is blocked. This filter is very, very efficient and powerful.

The email you forwarded us from Sony BMG contains 321 URLs in it. This causes SpamFilter to perform 321 DNS lookups. Normally this should take about 10 seconds (we tested it on our servers just now), so it is not a problem. However, should your DNS server experience problems, or timeouts, SpamFilter has a DNS lookup timeout of 15 seconds. After 15 seconds, if a response from ther DNS server is not received, SpamFilter will ignore the lookup and continue with the next one. Unfortunately, in the BMG email, should the DNS server be unavailable, it will mean that SpamFilter will timeout once per URL, so in this case 321 x 15 seconds = 4815 seconds. This means that it could take up to 80 minutes for SpamFilter to "test" all 321 URLs should the DNS server have issues. This very well explains the 20-30 minutes we've observed in your logs that it has taken to process these emails. The remote server, seing this long delay, is most likely interpreting it as a failure in sending its email, and will thus try again multiple times. Now with hindsight, going back thru your logs we also do see several DNS timeouts, indicating problems.

While it's easy to blame a DNS server for the problems, hopefully you'll have noticed that this is not how to operate... Please allow us a few more hours to come up with a solution, so that even with DNS problems the tests won't take that long. We'll also try to improve the SURBL DNS checks by avoiding duplicate lookups. We'll be testing the new build as fast as possible, and then will provide you with a "beta" build for you to test as well.

Regarding Spam Filter 2.6.3.487 : Roberto, I sent my info by email....

Fri, 06 Jan 2006 16:37:06 +0000

Author: Desperado
Subject: 5420
Posted: 06 January 2006 at 4:37pm

Roberto,

I sent my info by email.

Regarding Spam Filter 2.6.3.487 : Dan,Sara's duplicates were...

Fri, 06 Jan 2006 16:05:37 +0000

Author: LogSat
Subject: 5420
Posted: 06 January 2006 at 4:05pm

Dan,

Sara's duplicates were different than yours in the past. If you (Dan) can send me the logs showing the duplicates, along with the to/from emails I'll be able to see if the issues are related or not.

Sara,

Unfortunately the only list I was able to susbscribe to last time still has not sent me emails, so that is a dead end so far. The missing >>QUIT
<<221 Signing Off should not be an issue, but in this case I'm not going to discard anything, and will look into it. One possibilty could in theory be that the sender never sees the "250 OK" so thinks that the email was not successful. If you can please zip and email us your SpamFilter logs from today so we can correlate your posting entries with them, that will help. We may need to prepare a special debug build for you, but we need first to understand "where" to put extra debug code. Please allow us 12/24 hours once we have your logs to try to come up with a way to better see what is causing this.

Are you familiar with either Windows' Network Monitor utility, or with Etherreal? They both perform packet sniffing on a server, and the raw TCP data may help to shed some light onto this. If you're familiar with them please let us know so we can ask you to run them on the server.

Regarding Spam Filter 2.6.3.487 : Hi This is just another instance...

Fri, 06 Jan 2006 13:45:38 +0000

Author: scharette
Subject: 5420
Posted: 06 January 2006 at 1:45pm

Hi

This is just another instance of that same email. Please let meknow if I should be providing you headers for each Debug log. This email has been sent to two of our users more than 20 timestoday. The problem seems to be getting worse......though it isreally hard to tell....

>>EHLO jasper-cc.upromise.com
<<250-8BITMIME
<<250 HELP
>>EHLO jasper-cc.upromise.com
<<250-8BITMIME
<<250 HELP
>>MAIL FROM:
<<250 Address Okay
>>RCPT TO:
>>MAIL FROM:
<<250 Address Okay
>>RCPT TO:
<<250 hillcrest@mail.inct.net Address Okay
>>DATA
<<354 Start mail input; end with .
<<250 daustrom@tpaconsulting.com Address Okay
>>Received: from bomber-cc (bomber-app.prod.upromise.com [172.24.36.140])
>>n. Save 1% for college
>>ers in-stores, online, or by phone.
>>57698096
>>DATA
<<354 Start mail input; end with .
>>alt="" border="0">
>>s/mc/spacer.gif' width='1' height='2' alt='' border='0'>
>>eight='4' >
>>Received: from bomber-cc (bomber-app.prod.upromise.com [172.24.36.140])
>>ise.com/browse.php?olmcat=home_kitchen&ax=ase_ 20051228_03_20224
>>
- Have a new credit card?
>>
>>-800-462-3966).
>>border="0">
>>://www.upromise.com/survey-mm/2503995520/ase01_200 51228">Takethis short survey &l t;/table>
>> cellpadding='0' cellspacing='0' border='0'>
>>ing free shipping... all in one place? Clickhere now.
>>'#F3F3DE'>
>>

Learnmore
Learnmore
>>order='0'>

>>80/ct?g=1.1ppbrt.8toe.rx.0.77fexo.2dyiq&i=2188 97&a=%26ux%3ddaustrom&h=1">upromise.com to learn more.

<<250 Ok
<<250 Ok

Thanks

Sara

Regarding Spam Filter 2.6.3.487 : Roberto, I too am getting 3 or...

Fri, 06 Jan 2006 11:19:49 +0000

Author: Desperado
Subject: 5420
Posted: 06 January 2006 at 11:19am

Roberto,

I too am getting 3 or 4 reports of duplicated messages in the last 2 days.

Dan

Regarding Spam Filter 2.6.3.487 : Just as I finished up the last...

Fri, 06 Jan 2006 10:26:31 +0000

Author: scharette
Subject: 5420
Posted: 06 January 2006 at 10:26am

Just as I finished up the last post....I was able to pull one debug log on a particular email that is duplicating itself......

>>EHLO jasper-cc.upromise.com
<<250-8BITMIME
<<250 HELP
>>EHLO jasper-cc.upromise.com
<<250-8BITMIME
<<250 HELP
>>MAIL FROM:
<<250 Address Okay
>>RCPT TO:
>>MAIL FROM:
<<250 Address Okay
>>RCPT TO:
<<250 hillcrest@mail.inct.net Address Okay
>>DATA
<<354 Start mail input; end with .
<<250 daustrom@tpaconsulting.com Address Okay
>>Received: from bomber-cc (bomber-app.prod.upromise.com [172.24.36.140])
>>n. Save 1% for college
>>ers in-stores, online, or by phone.
>>57698096
>>DATA
<<354 Start mail input; end with .
>>alt="" border="0">
>>s/mc/spacer.gif' width='1' height='2' alt='' border='0'>
>>eight='4' >
>>Received: from bomber-cc (bomber-app.prod.upromise.com [172.24.36.140])
>>ise.com/browse.php?olmcat=home_kitchen&ax=ase_ 20051228_03_20224
>>
>>-800-462-3966).
>>border="0">
>>://www.upromise.com/survey-mm/2503995520/ase01_200 51228">Takethis short survey &l t;/table>
>> cellpadding='0' cellspacing='0' border='0'>
>>
>>ing free shipping... all in one place? Clickhere now.
>>href="http://shop.upromise.com/browse.php?olmcat=t ravel&ax=ase_20051228_03_20069">upromise.com
>>f" size="2" color="#666666">

Buying or selling your home?
<<250 Ok
<<250 Ok

What is really odd, is that the emails that are duplicating themselvesall have one thing in common and that is that they all end in the<<250 Ok
<<250 Ok
Even though I can confirm the email has been delivered to the recipient. Instead of the standard <<250 Ok
>>QUIT
<<221 Signing Off
response which is given if the email is properly deliverd and not duplicated.

Header for the particular Debug log above.
Header Information [ HIDE ]

Received: from spam02 [192.168.50.39] by mail.inct.net with ESMTP
(SMTPD32-8.00) id A4B853505FA; Fri, 06 Jan 2006 09:54:48 -0500
Received: from 12.130.29.196 by my_mailserver.mydomain.com (LogSat Software SMTP Server) Fri, 6 Jan 2006 09:53:07 -0500
Received: from bomber-cc (bomber-app.prod.upromise.com [172.24.36.140])
by jasper-cc.upromise.com (Postfix) with ESMTP id CFC63A16C
for ; Sun, 1 Jan 2006 23:23:22 -0500 (EST)
Message-ID: <1.1ppbrt.8toe.rx.0.77fexo.2dyiq@dialog>
Date: Sun, 1 Jan 2006 23:21:09 -0500 (EST)
From: Upromise
Reply-To: accountsummary@upromise.com
To:
Subject: Douglas, Your Monthly Upromise Account Summary as of 12/24/2005
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_Part_587594_719563.1136175802828"
X-Server: LogSat Software SMTP Server
X-SF-RX-Return-Path:

X-SF-HELO-Domain: jasper-cc.upromise.com
X-RCPT-TO:
Status: R
X-UIDL: 426987481

Header from a previous instance of this email.

Header Information [ HIDE ]

Received: from spam02 [192.168.50.39] by mail.inct.net with ESMTP
(SMTPD32-8.00) id A12651F05FC; Fri, 06 Jan 2006 08:31:18 -0500
Received: from 12.130.29.196 by my_mailserver.mydomain.com (LogSat Software SMTP Server) Fri, 6 Jan 2006 08:29:42 -0500
Received: from bomber-cc (bomber-app.prod.upromise.com [172.24.36.140])
by jasper-cc.upromise.com (Postfix) with ESMTP id CFC63A16C
for ; Sun, 1 Jan 2006 23:23:22 -0500 (EST)
Message-ID: <1.1ppbrt.8toe.rx.0.77fexo.2dyiq@dialog>
Date: Sun, 1 Jan 2006 23:21:09 -0500 (EST)
From: Upromise
Reply-To: accountsummary@upromise.com
To:
Subject: Douglas, Your Monthly Upromise Account Summary as of 12/24/2005
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_Part_587594_719563.1136175802828"
X-Server: LogSat Software SMTP Server
X-SF-RX-Return-Path:

X-SF-HELO-Domain: jasper-cc.upromise.com
X-RCPT-TO:
Status: R
X-UIDL: 426987475

Hopefully this helps......

Thanks

Sara

Regarding Spam Filter 2.6.3.487 : Well Roberto, These last few...

Fri, 06 Jan 2006 10:02:50 +0000

Author: scharette
Subject: 5420
Posted: 06 January 2006 at 10:02am

Well Roberto,

These last few weeks have been more than hectic in my office, due tothe holidays and people being on vacation. So I haven't had asmuch time as I like to put towards the spam software....but forwhatever reason it is indeed still happening. Unforutantely Ihave had very little luck using the debug feature as the emailsduplicate themselves within a small period of time so by the time Ifind out about them, the duplicates have stopped. I again, gotduplicates of the BMG newsletter this month (which is the same one thatduplicated for me last month).....and I see that the wjmead address Igave you is still getting multiple duplicate newsletters.

Here is a couple headers from the BMG one that just duplicated for me this morning and last night....

Date: Thu, 05 Jan 2006 20:05:55 -0500
From: "BMG Music Service"
Subject: Your COMMON Featured Selection is here!
Reply-to: "BMG Music Service"
To: scharette@sst-ltd.net
Received: from spam02 [192.168.50.39] by mail.inct.net with ESMTP (SMTPD32-8.00) id A55182005FC; Fri, 06 Jan 2006 08:49:05 -0500
Received: from 216.74.160.10 by my_mailserver.mydomain.com (LogSat Software SMTP Server) Fri, 6 Jan 2006 08:32:38 -0500
Received: from ([127.0.0.1:51490]) by bm1-1.ed10.com (ecelerity 1.2.12.3 r(6451)) with ECSTREAM id 2B/21-01811-372CDB34 for

;

Thu, 05 Jan 2006 20:05:55 -0500
Content-Type: text/html
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Message-Id: <3762-64093-FBJOY-LQF5NB-89VSZB-H@e-dialog.com>
X-Mail-From: FBJOY-LQF5NB-89VSZB-H@bmg.bounce.ed10.net
X-RCPT-To: scharette@sst-ltd.net
X-Mailer: EDMAIL R6.00.02
X-Server: LogSat Software SMTP Server
X-SF-RX-Return-Path:

X-SF-HELO-Domain: bmg.outbound.ed10.com
Status: U
X-UIDL: 436472710

Date: Thu, 05 Jan 2006 20:05:55 -0500
From: "BMG Music Service"
Subject: Your COMMON Featured Selection is here!
Reply-to: "BMG Music Service"
To: scharette@sst-ltd.net
Received: from spam02 [192.168.50.39] by mail.inct.net with ESMTP (SMTPD32-8.00) id A0093DF005E4; Fri, 06 Jan 2006 07:18:17 -0500
Received: from 216.74.160.10 by my_mailserver.mydomain.com (LogSat Software SMTP Server) Fri, 6 Jan 2006 07:01:52 -0500
Received: from ([127.0.0.1:51490]) by bm1-1.ed10.com (ecelerity 1.2.12.3 r(6451)) with ECSTREAM id 2B/21-01811-372CDB34 for

;

X-SF-HELO-Domain: bmg.outbound.ed10.com
Status: U
X-UIDL: 436472707

Date: Thu, 05 Jan 2006 20:05:55 -0500
From: "BMG Music Service"
Subject: Your COMMON Featured Selection is here!
Reply-to: "BMG Music Service"
To: scharette@sst-ltd.net
Received: from spam02 [192.168.50.39] by mail.inct.net with ESMTP (SMTPD32-8.00) id AAC386CD05C2; Fri, 06 Jan 2006 05:47:31 -0500
Received: from 216.74.160.10 by my_mailserver.mydomain.com (LogSat Software SMTP Server) Fri, 6 Jan 2006 05:31:07 -0500
Received: from ([127.0.0.1:51490]) by bm1-1.ed10.com (ecelerity 1.2.12.3 r(6451)) with ECSTREAM id 2B/21-01811-372CDB34 for

;

X-SF-HELO-Domain: bmg.outbound.ed10.com
Status: U
X-UIDL: 436472706

I know what you are staying in regards to the X-UIDL number beingdefferent....but something just isnt adding up here.....as I wasn'tgetting duplicates like this with the much earlier version ofSpamFilter (I will have to go back and check which version itwas)......Is there any other way we can troubleshoot this, versusrelying on the debug feature (again it is been really hard to get theIP in question entered before it quits duplicating.......The BMG emailabove I have received 13 times since yesterday evening, though itstopped at 8am so I haven't been able to get the debug logs. I amtrying to capture some logs right now for another email that is beingduplicated as well....hope to have that by the end of the day today.

Regarding Spam Filter 2.6.3.487 : Sara,I hope you're joking...

Fri, 23 Dec 2005 10:38:51 +0000

Author: LogSat
Subject: 5420
Posted: 23 December 2005 at 10:38am

Sara,

I hope you're joking about the posting being rude! If we loook at all my "short" posts then I should be banned from our own forum :-)

We'll wait for the email then, and hope we'll be able to provide an answer.

Learnmore
Learnmore
>>order='0'>