Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - Stuck Connections
  FAQ FAQ  Forum Search   Register Register  Login Login

Stuck Connections

 Post Reply Post Reply Page  12>
Author
jerbo128 View Drop Down
Senior Member
Senior Member
Avatar

Joined: 06 March 2006
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote jerbo128 Quote  Post ReplyReply Direct Link To This Post Topic: Stuck Connections
    Posted: 05 June 2006 at 9:38pm

Starting about a week ago, I began to have "stuck connections"  Anywere from 2 to 10 per day.  Clicking on the KILL line will not clear them.  They are not reflected in the "current inbound connections" along the status bar of the SF GUI.  The only way that I can clear them is to restart the SF service.  The screenshot below was taken at 8PM.  You can see that there are several connections that are 6+ hours old.  I pasted the log from the timeperiod of the last stuck connection.  I see nothing that sticks out other than the fact that the log never showed the "disconnect". 

I am running 3.0.1.558.  I have been running this version since the day it came out and have had no other issues at all.

As always, thanks for the help.  You have an outstanding product.

*******************************

06/05/06 14:37:14:792 -- (34704) Connection from: 200.123.153.9  -  Originating country : Argentina
06/05/06 14:37:15:370 -- (34704) - EmailTO is not in AuthorizedTOEmail list...
06/05/06 14:37:15:370 -- (34704) 200.123.153.9 - Mail from: jhqkorlqvjk@hotmail.com To: server@DOMAINNAME.COM will be rejected
06/05/06 14:37:15:808 -- (8084) Time to add Msg to Bayes corpus:0
06/05/06 14:37:16:011 -- (34704) Blacklist cache - Added 200.123.153.9 to limbo

**********************************

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 06 June 2006 at 12:13am
We've had 3 reports today with a similar issue. It's something that has rarely come up before, and we believed was solved in version 3.0.1.557. It's a strange coincidence to receive so many reports the same day, but so far we've not been able to reproduce the problem. As soon as we find soemthing more I'll let you know.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
jerbo128 View Drop Down
Senior Member
Senior Member
Avatar

Joined: 06 March 2006
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote jerbo128 Quote  Post ReplyReply Direct Link To This Post Posted: 06 June 2006 at 9:24am

Let me know if there is anything that you need - logs, screenshots, etc. 

Thanks Roberto.

jerbo128

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 06 June 2006 at 4:15pm
jerbo128,

actually a zipped copy of your SpamFilter activity log for 6/5/06, may be trimmed from 11am to 2pm, could hopefully help us finding out why those connections still show up.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
jerbo128 View Drop Down
Senior Member
Senior Member
Avatar

Joined: 06 March 2006
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote jerbo128 Quote  Post ReplyReply Direct Link To This Post Posted: 06 June 2006 at 5:31pm

Log sent to support@.....

Thanks for the help.

jerbo128

Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 13 June 2006 at 7:50am
For the record I noticed this again on my install. I hadn't checked it for a while, and there were stuck connections going back days (possible more than a week). I'm running 3.0.1.558 and as I wasn't aware of any cure just restarted the SF service.
Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 13 June 2006 at 9:24am

I'm seeing stuck connections as well - looks like they show up when load on spamfilter is heavy...

connections are no longer existant on the system (netstat) but show up on the connections tab and can not be killed there

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 13 June 2006 at 11:15pm
We have recently found an issue with connections steadily increasing in the following specific case.

1. Using MySQL and the MyISAM database type (InnoDB is fine, it does not cause the problem).
2. The MySQL database is being backed up using the :Lock All Tables" execution method.
3. The database backup process takes several minutes to complete.

Under the scenario above, MySQL will place a write lock on all tables, preventing any application from making DB updates. This will prevent SpamFilter to add new records. We have safety features that timeout the update process in case the SQL update take too long. However, due to a "missing feature" (i.e. a bug....) in the MySQL ODBC driver, the timeout function is not implemented correctly.
This causes SpamFilter to wait indefinetly until the lock is removed. The connection thread thus will not terminate until the email has been inserted in the database, and this will cause the number of incoming connections thread to increase steadily, until the lock is lifted. But if the backup process takes several minutes to complete, this may cause the "Too many connections" limit to be reached.

It is to be noted that the problem occurs only in the specific scenario above. If the MySQL database is unavailable for any other reason (DB is down, non-existent tables, incorrect permissions for example), SpamFilter will immediately see the DB connectivity issues and will simply continue processing emails without quarantining them.

We've just uploaded build 3.0.1.571 that addresses this problem and prevents it from occurring. This did require major changes to how quarantined emails are processed, so, while testing has shown no issues, it is to note that this *is* a pre-release build, and as such, may not be as stable as an official release.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2006 at 3:54am
My problem must have been something else then as I'm using MS SQL 2005???
Back to Top
jerbo128 View Drop Down
Senior Member
Senior Member
Avatar

Joined: 06 March 2006
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote jerbo128 Quote  Post ReplyReply Direct Link To This Post Posted: 14 June 2006 at 8:38am

FYI:  I am running Ms Access - and for the record have not had any more of these "stuck connections"  in the past week.

jerbo128

Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 16 June 2006 at 5:35am
I'm still seeing these stuck connections even with the newest build .571

SQL Server 2000 via OLEDB as DB Server
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 16 June 2006 at 10:14am
I'm seeing stuck connections again. ATM SF is running at 98% CPU. There are 24 Stuck connections from yesterday between 22:19:52 and 22:22:33 BST (3 minute period). They are stuck in either QUIT, QUEUEING EMAIL or RCPT TO.

I have another 3 stuck from today, at 15:02:57, 11:16:40 & 11:26:57. In NOOP, and PROCESSING DATA... Even though none of the IPs show in a netstat.

I've zipped and emailed an hours worth of logs for you from yesterday.

Regards,
Lyndon.


Edited by lyndonje
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 16 June 2006 at 1:09pm
I see anywhere from 5-12 stuck connections per day and have been restarting SFI every evening.
http://www.webguyz.net
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 17 June 2006 at 11:43am
Thanks to everyone for their logs. Unfortunately as of today we still have not been able to reproduce the problem.

We did however completely rewrite the procedure that checks for idle connections and diisconnects them. We're going in a bit blind as we don't know the cause, but we've tried to forsee as many scenarios as possible.

A new pre-release build (3.0.1.573) is available in the registered user area to attempt one more shot at adressing this.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 19 June 2006 at 4:15am
Have updated to the pre-release, will let you know what happens.

Regards,
Lyndon.
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 6:38am
So far so good.... no stuck connections yet.
Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 6:49am
Can't find build .573 on the download page... .571 is the newest version listed, although I have .572 installed.
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 6:54am
I've just checked that I'm running 3.0.2.573 and I am.

I've just logged into the registered area and can now only see:

  
SF2.6.3.487.zip
   
SF2.7.1.535.zip
   
SF3.0.1.560.zip
   
SF3.0.1.561.zip
   
SF3.0.1.567.zip
   
SF3.0.2.571.zip

Roberto, where has it gone? Must have been there before for me to download...?


Edited by LogSat
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 8:21am
lyndonje, mikek,

Yesterday we had 2 separate reports of SpamFilter freezing after just a few minutes. It affected both builds 572 and 573. While we were not able to replicate it, and there were only 2 reports, they were identical in the symptom... so that was still too many for us... We recalled those releases yesterday as a precaution.

We've just uploaded build 575 which should have fixed those issues.

Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 8:36am

My .573 had locked up once as well but I was going to see if it happened again. Too bad it happened at 4:00am and my monitoring system was able to page me and wake me up.

Will load up 575 tonite and keep my fingers crossed.

http://www.webguyz.net
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 8:48am
Funny that, just read this post and at the same time noticed SF had locked up on me too! White screen, not responding and not listening on port 25.

Just loaded 575. Will keep you posted.
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 10:07am
Bad news, running 3.0.2.575 I have 6 stuck connections.

Have emailed you further details.
Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 11:09am
already 4 stuck connections here with .575...

inform me, if you want logs...


Edited by mikek
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 11:17am
Hi mikek, out of interest do you use the 'Authorized To' filter?

If so, are the stuck connections to recipients that are either covered by *@domain or not listed in your Authorised To file?

I might be way off here, so don't want to cloud things or lead people down the wrong path, but thats what I have noticed on these recent stuck connections.


Edited by lyndonje
Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 11:21am
Yes, I do. I generate a authorizedto.txt file out of the user database of our mailserver, every time a change to that database happens. The file has about 4700 entries.
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 11:26am
You may not have seen my edited comment at 4:19, I added:

If so, are the stuck connections to recipients that are either covered by *@domain or not listed in your Authorised To file?

I might be way off here, so don't want to cloud things or lead people down the wrong path, but thats what I have noticed on these recent stuck connections.
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 11:31am
thanks for pointing that out, we'll look into that aspect right now. As we're still not able to replicate this, we're open to any suggestions / hints you all have.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 11:33am
No, all my recent stuck connections are to users which are listed individually in the authorized-to list.
Back to Top
lyndonje View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2006
Location: United Kingdom
Status: Offline
Points: 192
Post Options Post Options   Thanks (0) Thanks(0)   Quote lyndonje Quote  Post ReplyReply Direct Link To This Post Posted: 20 June 2006 at 11:35am
Oh right... thats that one out the window then  Oh well, worth a try.

Edited by lyndonje
Back to Top
mikek View Drop Down
Senior Member
Senior Member
Avatar

Joined: 22 February 2005
Location: Switzerland
Status: Offline
Points: 133
Post Options Post Options   Thanks (0) Thanks(0)   Quote mikek Quote  Post ReplyReply Direct Link To This Post Posted: 21 June 2006 at 8:46am
18 stuck connections in the last 24 hours. sorry to say that looks to me like it even got worse with .575

Edited by mikek
Back to Top
 Post Reply Post Reply Page  12>
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.108 seconds.