Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - SF stripped attachments - bug or feature?
  FAQ FAQ  Forum Search   Register Register  Login Login

SF stripped attachments - bug or feature?

 Post Reply Post Reply
Author
yapadu View Drop Down
Senior Member
Senior Member


Joined: 12 May 2005
Status: Offline
Points: 272
Post Options Post Options   Thanks (0) Thanks(0)   Quote yapadu Quote  Post ReplyReply Direct Link To This Post Topic: SF stripped attachments - bug or feature?
    Posted: 11 November 2009 at 9:13pm
We had a new client sign up for our spamfilter service.

After about 1 week, they contact our support and say that an invoice (HTML attachment) from a vendor is being stripped off.  Not placed into quarantine, but stripped off and the rest of the message delivered.

I am not aware of any feature like this, and we have no virus scanner on the server itself other than the one included in spamfilter.

We conduct some tests, and don't find any problem to our email addresses.  The logs show the messages flowing through our server, and no issues shown in the logs either.

I ask the new client to put the MX records back to their old MX records which takes SF out of the equation.

Sure enough, after about 36 hours they contact us and say the messages are being delivered with the attachments in tact.

How can this be, and how can we track the problem down.  We have lost the client now, so probably don't have them available to do testing.  They have provided us with one of the sample attachments but we can't reproduce the issue.

Anyone else ever have stripped attachments?
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 12 November 2009 at 11:58pm
yapadu,

SpamFilter will not strip attachments when forwarding emails, and we are not aware of any issues of this ever happening in the past. If you are able to reproduce this we'll definitely look into it of course, as there is always a possibility of a bug causing unexpected behavior. Also, if you can zip and email us the section of SpamFilter's activity logfile around the time the email in question was received, we can scan it for errors.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
yapadu View Drop Down
Senior Member
Senior Member


Joined: 12 May 2005
Status: Offline
Points: 272
Post Options Post Options   Thanks (0) Thanks(0)   Quote yapadu Quote  Post ReplyReply Direct Link To This Post Posted: 17 November 2009 at 7:35am
The customer with the missing attachments came back, and we were able to be included on the list of recipients (at two different domains).

We have so far received three messages from the specific sender with the problem, and sure enough there is no attachment.

Is there any way to enable some type of advanced debugging for a specific domain or sender or anything and trap what comes in/out?

We can not reproduce the problem ourselves yet, so we can't simulate the problem on demand.
Back to Top
yapadu View Drop Down
Senior Member
Senior Member


Joined: 12 May 2005
Status: Offline
Points: 272
Post Options Post Options   Thanks (0) Thanks(0)   Quote yapadu Quote  Post ReplyReply Direct Link To This Post Posted: 17 November 2009 at 9:03am
I have been going through the logs, looking at the incoming messages from said sender.  No errors reported during the processing of the messages.

I do find this to be of interest:

(4016) EMail from email@exmple.com to "email1@abc.com" was queued. Size: 11 KB, 11264 bytes

So, 11264 bytes in size was processed at this point by spamfilter.

Here is the delivery summery to the next server:

1NACf6-0002zK-GZ => postmaster <email1@abc.com> F=<email@example.com> R=virtual_user T=virtual_localdelivery S=1324

Only 1324 bytes received from the server spamfilter passed the message onto.

When I pull the message from the mailbox the message size is exactly 1324 bytes.  It is starting to look like it is spamfilter who is eating part of the message for some reason.

Here is what remains of the message (I have three with virtually the same content) excluding some headers.

Subject: ABC INVOICE# 0961756
Content-type: multipart/mixed;
     boundary="ABC____29737919012111160901"
X-Server: LogSat Software SMTP Server
X-SF-RX-Return-Path: <ORDERS@ABC.COM>
X-SF-HELO-Domain: SM.ABC.COM
X-SF-Originating-IP: 48.69.60.107

This message contains an attachment in MIME format, which is not
legible to your E-mail application.

--ABC____29737919012111160901
Content-type: multipart/alternative;
     boundary="ABC____29737919012111160902"

--ABC____29737919012111160902

--ABC____29737919012111160902
Content-type: text/html; charset=Windows-1252



Edited by yapadu - 17 November 2009 at 9:05am
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 17 November 2009 at 10:42pm
yapadu,

We received the logs, and as you said, this is going to be an interesting issue... The only way we're going to be able to see what is happening is by actually "seeing" what the original email looks like.

SpamFilter does have a "Debug" tab to capture SMTP traffic, but it doesn't capture *all* of the email's body, so it's not going to help here. What would help would be to install either WireShark (Etherreal) on the SpamFilter server (or any other packet capture utility), and capture all SMTP traffic from the remote server (I'll send you the IP common to the 2 emails in the logs via email).
If you'd like, you could also provide our address to the vendor to copy us as well on the emails. We can then run the packet capture on our end and try to get to the bottom of this. Please let us know in advance if you'd like for us to run the trace so we can setup up our end before they send the emails.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
yapadu View Drop Down
Senior Member
Senior Member


Joined: 12 May 2005
Status: Offline
Points: 272
Post Options Post Options   Thanks (0) Thanks(0)   Quote yapadu Quote  Post ReplyReply Direct Link To This Post Posted: 25 November 2009 at 8:52pm
Roberto,

I have sent you two text files that were compiled from wireshark, the first shows the incoming conversation between servers of the delivery.

The second shows the delivery of the message to the SMTP server.

You can clearly see that the attachment was received when delivered to spamfilter, and when spamfilter delivers it the attachment portion of the message is gone.

Back to Top
yapadu View Drop Down
Senior Member
Senior Member


Joined: 12 May 2005
Status: Offline
Points: 272
Post Options Post Options   Thanks (0) Thanks(0)   Quote yapadu Quote  Post ReplyReply Direct Link To This Post Posted: 25 November 2009 at 9:05pm
I see the problem, I see the problem!

I am so excited!

Here is a snippet of the incoming message.

--ABC____30689919010211250902

Content-type: text/html; charset=Windows-1252

.

--ABC____30689919010211250902--


--ABC____30689919010211250901


Here is the last part of the sending communication.

--ABC____30689919010211250902

Content-type: text/html; charset=Windows-1252

.

250 OK id=1NDSsR-0007cd-78

QUIT

The message is cut off, as the body of the message when incoming contains a single period in between <CRLF> so spamfilter assumes that is the end of the message when incoming.

In actual fact the sending server still sends additional data.

That's gotta be it...


Edited by yapadu - 25 November 2009 at 9:05pm
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 28 November 2009 at 1:27am
We received the files, and were finally able to pinpoint the issue thanks to your wireshark capture.

Please see the first screenshot showing the incoming file to SpamFilter. What you see is not a period between the <CR><LF> separators. Wireshark shows that it is actually a NULL character (byte value of 0). This is a control character that, per RFC, should not be used in the mail body. 



SpamFilter does receive the email successfully, and however this is where the 2nd part of the problem begins (the first part is the remote server sending an invalid NULL character....). 

Please now see the second screenshot showing the outgoing email from SpamFilter to your destination SMTP server. When SpamFilter forwards the email, it tries to "fix" the invalid NULL character and replaces it, of all things, with the "0x2e" character, which happens to be the period. Since this period, (un)luck has it, is surrounded by the <CR><LF> sequence, this just happens to indicate the end-of-email sequence and thus your destination SMTP server obviously stops receiving the email.



This has got to be one of the most unlucky sequence of events we've seen...

I'll try to see if there's anything we can do to either prevent SpamFilter form autofixing the invalid NULL characters, or to at least replace them with a different, non-dot, character...!
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
yapadu View Drop Down
Senior Member
Senior Member


Joined: 12 May 2005
Status: Offline
Points: 272
Post Options Post Options   Thanks (0) Thanks(0)   Quote yapadu Quote  Post ReplyReply Direct Link To This Post Posted: 29 November 2009 at 7:49pm
It was suspected from the start this one was going to be difficult to isolate.  It certainly took some doing but I'm glad the problem has at least been identified and hope you are able to come up with a solution for this issue. 
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 02 December 2009 at 10:34pm
yapadu,

We've finally *just* been able to reproduce this. Thank you a LOT for the patience and the help. I can confirm everything is a suspected. SpamFilter is replacing the NULL character with the ".", and its placement just happened to be between the <CR><LF>.<CR><LF> sequence causing the email to be terminated.

While we come up with a permanent fix, the good news is that there is actually a very easy workaround that so far seems to avoid the problem.

In the SpamFilter.ini file, simply change the following setting to "1" (there is not even the need to restart SpamFilter after the change...)

SpoolQueueFilesToMemory=1

This forces SpamFilter to spool the incoming emails to disk rather than handling them in RAM. The spooling process actually causes SpamFilter to load the message a bit differently when forwarding it, avoiding the NULL character to be replaced.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
yapadu View Drop Down
Senior Member
Senior Member


Joined: 12 May 2005
Status: Offline
Points: 272
Post Options Post Options   Thanks (0) Thanks(0)   Quote yapadu Quote  Post ReplyReply Direct Link To This Post Posted: 02 December 2009 at 11:28pm
Hi Roberto,

Hate to make things more complex, but check out this snippet from my ini file:

;Set this to 0 to prevent queued emails to be spooled to memory, and force spooling to disk. While less efficient, spooling to disk helps allow existing antivirus software to detect and block some infected email files
SpoolQueueFilesToMemory=1


The server is already spooling to disk it seems. :-(
Back to Top
yapadu View Drop Down
Senior Member
Senior Member


Joined: 12 May 2005
Status: Offline
Points: 272
Post Options Post Options   Thanks (0) Thanks(0)   Quote yapadu Quote  Post ReplyReply Direct Link To This Post Posted: 02 December 2009 at 11:29pm
Hmmm...

After reading the setting in the INI file, I think you mean to set it to 0 right?  The ini comment says 0 will spool to disk.  Since it is already 1 and the bug is happening you must have meant 0...  I'll try that.
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 02 December 2009 at 11:33pm
Sorry... bug in the posting! I copied/pasted the setting that *breaks* the email, so yes, the correct one to use that fixes the issue is:

SpoolQueueFilesToMemory=0

..and a side-note, we already have a fix ready. TWO single lines of code were causing the problem. We'll be testing this for a few days and will make the new build available to everyone as soon as we're sure there's no side-effects.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.063 seconds.