Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - Time out causing duplication of email?
  FAQ FAQ  Forum Search   Register Register  Login Login

Time out causing duplication of email?

 Post Reply Post Reply
Author
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Topic: Time out causing duplication of email?
    Posted: 09 March 2010 at 5:13am
Hi guys,

On a number of occasions we have been informed by a few different clients that emails from particular (but varied) senders are being received multiple times - like 100's!

Looking at the logs our end, we don't see a problem - to us we just see the sending server make a connection, send an email which we forward onto the destination server. I've now been able to get hold of some sort of log file from one of the senders which are experiencing this 'duplication' of emails. From what I can gather, there system connects and sends SF the email, SF does whatever checks it does, but before SF accepts/confirms delivery, the senders end hits a time out, drops the connection and re-queues the email for another delivery attempt later. Despite the senders end timing out before our server has confirmed receipt or acceptance, our server still forwards the email - presumably because our server knows its received the email in its entirety having receiving the <CR>.<CR> instruction? So assuming the email passed spam checks, the email is sent on. 

The above course of events cause a loop, whereby our server receives and delivers the email, the senders continually times out before our server has confirmed acceptance, causing the sending server to re-queue, and re-deliver the email - and then the loop begins again.

If this is what's happening, I think either one of the following is wrong:

1) The senders end's time out is too short?
2) SF should not deliver an email if the connection is dropped/times out - just like it wouldn't if the connection was dropped part way through the DATA stage.

If number 2 is changed, so that SF does not deliver emails from which the connection is dropped even after a <CR>.<CR> is received, this will not prevent the senders end from timing out, so instead of receiving duplicates, our server would not forward any, meaning the email would never be received, and eventually a bounce back would be generated. What are your thoughts on this? And what do people thing the fix might be? 

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4068
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 09 March 2010 at 11:24pm
lyndonje,

The behavior you describe is very odd. An incoming email is not complete until the <CR>.<CR> end-of-transmission sequence is received. SpamFilter cannot process an incoming email unless that sequence is received. If the remote server is dropping the connection (or timing out) before sending that sequence, it *should* be impossible to process the email (the should is because in IT you've never ever seen it all....)

What could happen in theory is that the remote server does send the <CR>.<CR> sequence (which will cause SpamFilter to accept the email), but then for some reason does not receive the "250 OK" acceptance SMTP code from SpamFilter, which tells the remote server that the email was received correctly. Without seeing this 250 return code, it would be likely that the remote server will retry. It's however rather strange that there could be a timeout that prevents SpamFilter from successfully sending the 250 return code while at the same time allowing the incoming email to make it thru.

If you can zip us to support @ logsat.com any logs you have (even the remote server's logs - we'll try to use those as well), we'll try to see what is happening. If you have the ability to also configure a full 2-way network capture (with Wireshark or similar) of the SMTP traffic to/from the remote server in question, this would greatly help in identifying the problem.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
vbourbeau View Drop Down
Newbie
Newbie


Joined: 14 April 2010
Status: Offline
Points: 19
Post Options Post Options   Thanks (0) Thanks(0)   Quote vbourbeau Quote  Post ReplyReply Direct Link To This Post Posted: 15 April 2010 at 8:23am
I have the same problem, in my case is often because the email have an attachment more that 4mb.
 
Do you find solution on this?
 
Thank  
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4068
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 15 April 2010 at 11:28am
SpamFilter has built-in timeouts that may apply in this case. In the SpamFilter.ini file there are the following settings:

;Force disconnect of sessions after they have remained connected for this long
IdleDisconnectMinutesTimeout=15

;Force disconnect of sessions if a command has not been received within the last nn seconds
ReadTimeout=60

;Timeout when delivering emails to the destination SMTP server (in seconds)
ReadTimeoutOutgoing=60


You can try to increase them to see if allowing more time for large attachments helps. All the above settings can be changed while SpamFilter is running – they will be re-imported utomatically within a minute. Also, do you have SpamFilter configured to reject emails over a certain size? If so, this could be the problem if their message exceeds this size.


Please also note that sometimes firewalls may terminate connections if they see them idle for too long. If the sender has a slow connection, transferring large files may take several minutes to complete, and we have seen firewall/antivirus software disconnecting TCP connections while SpamFilter is busy receiving the email.

Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
vbourbeau View Drop Down
Newbie
Newbie


Joined: 14 April 2010
Status: Offline
Points: 19
Post Options Post Options   Thanks (0) Thanks(0)   Quote vbourbeau Quote  Post ReplyReply Direct Link To This Post Posted: 15 April 2010 at 12:52pm
here are my config:
 
IdleDisconnectMinutesTimeout=240
ReadTimeout=300
ReadTimeoutOutgoing=60
 
The max email size is disable
 
And I don't think it's the firewall because I received te email and attachment.
 
I'm actually in a email loop, so I can trap next network communication and send it to you? 
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4068
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 15 April 2010 at 5:25pm
Absolutely. If you're able to retrieve a network capture via Wireshark or similar, you can zip and email us the pcap file at support at logsat.com.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.125 seconds.