Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - New SpamFilter Enterprise beta v3.5.3.630
  FAQ FAQ  Forum Search   Register Register  Login Login

New SpamFilter Enterprise beta v3.5.3.630

 Post Reply Post Reply
Author
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Topic: New SpamFilter Enterprise beta v3.5.3.630
    Posted: 26 December 2006 at 10:20am
An updated beta version of SpamFilter Enterprise v3.5.3.630 is now available to registered users. Please see notes below for details on the fixes and new features available in this beta.

Please note that all new features are (and always will be) available in both SpamFilter Enterprise and SpamFilter ISP "standard".
SpamFilter Enterprise can be configured to run in "SpamFilter ISP standard" mode, which will cause SpamFilter to run pretty much as it always has, and does not require Microsoft SQL or MySQL to run.

Major Changes introduced in SpamFilter ISP Enterprise v3.5 - The new SpamFilter ISP Enterprise v3.5 allows administrators to configure most filter settings individually for each domain being hosted. Most settings are now stored in a database, making it simpler for administrators to access them via custom web applications.

Technical notes - SpamFilter ISP Enterprise can be configured to run in "Standard" mode and in "Enterprise" mode. When configured for "Standard" mode, the functionality will be the same as in previous versions. When configured for "Enterprise" mode, the new functionality is activated. The Quarantine Database must be enabled and active for the Enterprise mode.

New Database functionality - In Enterprise mode, most settings are saved to the database. However, to avoid downtime if the database server is not available, all settings are automatically exported from the database to the file system. SpamFilter uses the files located on the file system to read all settings. SpamFilter will *not* read the database directly for its settings. SpamFilter will however continuously monitor the database, and if any settings stored in the database are changed, they are automatically saved to files and re-imported within seconds.

New Database Requirements - SpamFilter Enterprise only supports Microsoft SQL 7 or higher and MySQL v5 or higher. Support for MS Access and Oracle will not be added. The new database functionality relies heavily on the use of database triggers to ensure replication of database data to the file system. Triggers where added to MySQL starting from version 5, which explains the minimum requirement for this platform. SpamFilter will try to add the necessary tables, indexes and triggers automatically. If this procedure fails, please manually execute the MicrosoftSQL-Enterprise.sql and/or the MySql-Enterprise.sql update scripts found in the database directory.

New .ini settings structure - In both the Enterprise and the Standard modes, the SpamFilter.ini file has been greatly modified. Several of the settings in the SpamFilter.ini file can now be specified individually for each domain. For this reason, these settings have been moved to a new Filter.ini file. In Enterprise mode, each domain will have an associated Filter.ini file. All files relative to a domain are now stored in a subdirectory named after that domain. Special folder names are:
.\Domains\SFI - contains all settings for SpamFilter ISP in Standard mode
.\Domains\ ALL DOMAINS - contains all settings for domains in SpamFilter Enterprise that have not been customized.

Updates since previous beta - In addition to several bug fixes, this version changes the way emails with multiple recipients are handles. Two major issues in the past have been:
(1) emails with with multiple recipients, where one of them was in the "Unfiltered" whitelist, would be delivered to everyone even if it was spam.
(2) emails where the recipients belonged to different domains, and each domain had custom destination SMTP servers.
Both of these are being addressed in this beta. The release notes for this build are:

// New to VersionNumber = '3.5.3.630';
{TODO -cFix : Solved several bugs that would cause Access Violation Exceptions while processing incoming emails}
{TODO -cFix : Emails with multiple recipients on multiple domains - if different domains have different destination SMTP servers, the email is split and delivered to each server correctly}
{TODO -cNew : Emails with multiple recipients - When a recipient is in the "Unfiltered Emails", now a spam email is only delivered to them, and not everyone else}
{TODO -cNew : Added the X-SF-WhiteListedReason header for emails that are force-delivered from quarantine}
{TODO -cFix : SFE Only - Added option to DROP Enterprise tables before creating them via the database scripts}
 

Disclaimer - This version is a pre-release beta. As such, problems are expected. Using this version in a live production environment is not recommended.

This beta will expire on March 1, 2007.




Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
NITF View Drop Down
Newbie
Newbie
Avatar

Joined: 07 November 2006
Location: United States
Status: Offline
Points: 1
Post Options Post Options   Thanks (0) Thanks(0)   Quote NITF Quote  Post ReplyReply Direct Link To This Post Posted: 27 December 2006 at 10:29pm

 

Thank you for working so hard on the Enterprise version!



Edited by NITF
Back to Top
caratking View Drop Down
Groupie
Groupie


Joined: 13 March 2006
Location: United States
Status: Offline
Points: 79
Post Options Post Options   Thanks (0) Thanks(0)   Quote caratking Quote  Post ReplyReply Direct Link To This Post Posted: 29 December 2006 at 8:29am
Thanks for the new beta, I have installed it and here are a list of bugs
I had previously reported, and their status as I see them in the new beta:

1) Country blocks per domain
- Working

2) Sorting by titles on Country Filters tab
- Not working (not sure if this is by design or not - it works in current
release)

3) Using the '\' character in keyword filters not possible (on mysql server)
- Not working (major bummer!  RegEX is difficult without this)

4) Auto whitelisting
- Not working
When I click deliver message from the quarantine, it does add the email address combination to the domains AutoWhiteListForceDelivery.txt file.

The email combination does NOT get added to tblwl_autowhitelistforcedelivery

5) CaSe IsSuEs with database names MySQL on Linux
- Have not tested yet


New Features Suggested
1) Que messages that are going to be quarantined if DB server not available
- Have not tested yet, suspect not added just yet (still hope).

I think those were the only issues I had reported, can't recall any others anyway.

Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 29 December 2006 at 9:07am

I'm trying to understand how SFE works for redundancy with a second server. Lets say I have 10 domains (or 500) and get a second copy of SFE for redundancy. Since your sharing a common database, how would you have both SFE's text files get updated with the same info?? How about 3 SFE's, how is redundancy/syncronisation accomplished in a multiple SFE environment?

 

http://www.webguyz.net
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 30 December 2006 at 12:33am
Thanks for the updates on the bugs caratking, we know the new beta has not fixed all pending issues, there were some serious flaws that took priority and had to be fixed sooner. We also had to release the new beta before the old one expired, so were forced to ignore some fixes, sorry!

WebGuyz, thanks for the observation. Your scenario is still untested, but I believe there would indeed be problems with the current beta to support multiple servers sharing the same database. We'll try to address this in the next beta in a few weeks.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
caratking View Drop Down
Groupie
Groupie


Joined: 13 March 2006
Location: United States
Status: Offline
Points: 79
Post Options Post Options   Thanks (0) Thanks(0)   Quote caratking Quote  Post ReplyReply Direct Link To This Post Posted: 30 December 2006 at 2:52am
I did not know what WebGuyz was referring to.  After Roberto's comments I now realize that the tblreloadtableinfo has nothing to indicate what server has / has not picked up the latest changes that need to be synced.

I thought I would update my previous bug comment thread:

5) CaSe IsSuEs with database names MySQL on Linux
- Is working (previously I had not retested).  I now have the SF pointing to a DB running on CentOS and MySQL 5.
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 30 December 2006 at 11:04am
Thanks for the MySQL report. Please let us know if there's anything you see in the future causing issues with case sensitivity. We had to practically rewrite all our SQL code to fix this, and may have skipped a few...

WebGuyz is, as is usually the case :-) correct. The tblReloadTableInfo contains an indication of the changes made to tables and the domains they apply to. When SpamFilter sees records in that table, it knows that changes were made, and will reload the correct records from the updated tables. After this is done though, SpamFilter will clear the temp data in tblReloadTableInfo. If there are multiple SpamFilter servers pointing to the same database, only one SpamFilter will see the changes, as by the time others check that temp table, the 1st SpamFilter will have deleted the records. We will have to figure out somehting... this is a scenario we hadn't thought about, but must of course fix it!
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
__M__ View Drop Down
Groupie
Groupie


Joined: 30 August 2006
Location: Australia
Status: Offline
Points: 75
Post Options Post Options   Thanks (0) Thanks(0)   Quote __M__ Quote  Post ReplyReply Direct Link To This Post Posted: 03 January 2007 at 3:06am
Apologies if this is a red herring but I am running V3.5.3.631 and I think that the feature to add AllowedDomainFilterMatrix.txt has been left out of \domains\SFI\filters.ini

It does appear to be in the SpamFilter.ini file however regardless of the path I specify it always tries to grab the file from the default install path of SFI.

Can anyone confirm this is the case please.

Test Enviroment: SpamFilter V3.5.3.631 BETA in Standard Mode (not Enterprise)


Edited by __M__
Back to Top
caratking View Drop Down
Groupie
Groupie


Joined: 13 March 2006
Location: United States
Status: Offline
Points: 79
Post Options Post Options   Thanks (0) Thanks(0)   Quote caratking Quote  Post ReplyReply Direct Link To This Post Posted: 03 January 2007 at 3:09am
Found another situation where cAsE on mySQL on linux is an issue:

01/03/07 08:08:00:796 -- Exception occurred during ChecktblReloadTableInfo: [MySQL][ODBC 3.51 Driver][mysqld-5.0.22]Table 'mx.tblReloadTableInfo' doesn't exist

I've been able to work around the problem by creating a view called tblReloadTableInfo that is simply a view of the lowercase table name.



Edited by caratking
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 04 January 2007 at 11:18pm
__M__ and caratking, unfortunately we confirm (just verified...) both bugs still exist in the new beta.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
__M__ View Drop Down
Groupie
Groupie


Joined: 30 August 2006
Location: Australia
Status: Offline
Points: 75
Post Options Post Options   Thanks (0) Thanks(0)   Quote __M__ Quote  Post ReplyReply Direct Link To This Post Posted: 05 January 2007 at 1:17am
Roberto, no probs mate.

Glad to be able to assist with the testing of the beta, this problem by no means is causing us any great problem.

I'm just very thankful that you guys are constantly addressing our concerns/wishes especially with major rewrite like the split mail processing.

I continue to be more than happy with my decision to rely on SFI for our filtering needs.
Back to Top
__M__ View Drop Down
Groupie
Groupie


Joined: 30 August 2006
Location: Australia
Status: Offline
Points: 75
Post Options Post Options   Thanks (0) Thanks(0)   Quote __M__ Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2007 at 6:43am
Roberto, I have been testing V3.5.3.635 today on our test server and it appears to be working fine however this evening when I upgraded our live server I discovered that our MySQL case issue is still haunting us.

We are running in "standard" mode with our Quarantine database running on a Linux MySQL database called "SpamFilter_Quarantine". The following error is occuring because I believe of the case of tblquarantine (previous versions such as x.605 retail used tblQuarantine).

Exception occurred during TDeliverQuarantineThread.Execute: [MySQL][ODBC 3.51 Driver][mysqld-4.1.8-standard]Table 'SpamFilter_Quarantine.tblquarantine' doesn't exist

Now obviously I could just rename the database table however considering our web-based Quarantine interface was designed for use with the old naming convention I think it's best that the case sensitive issue is addressed and Logsat standardizes on the naming previously used. To make life a little easier (although I'm sure you could look up old code) here is what the naming was previously.

tblLogins 
tblMsgs
tblQuarantine
tblRejectCodes
tblServers

Let me know when you have another revision ready and I'll just punch it straight onto the live server to confirm that you can put these niggly little bugs to rest.

Thanks again, Mike


Back to Top
Rifleman View Drop Down
Groupie
Groupie


Joined: 08 August 2005
Location: United States
Status: Offline
Points: 53
Post Options Post Options   Thanks (0) Thanks(0)   Quote Rifleman Quote  Post ReplyReply Direct Link To This Post Posted: 12 January 2007 at 10:52am
Is there a trick to adding domains? I cannot add more than 12 characters to the domain list. It will not let me. What am I doing wrong?
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 12 January 2007 at 4:11pm
Could you please email/post a screenshot of where this is happening?
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 16 January 2007 at 5:08pm
Mike,

We've actually converted all SQL queries in the new beta to lowercase, so this will require all table names to be changed to lowercase as well. This way to proceed was the simplest and safet way to avoid case-sensitivity problems in the future. Sorry for the delay!
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
jerbo128 View Drop Down
Senior Member
Senior Member
Avatar

Joined: 06 March 2006
Status: Offline
Points: 178
Post Options Post Options   Thanks (0) Thanks(0)   Quote jerbo128 Quote  Post ReplyReply Direct Link To This Post Posted: 19 January 2007 at 2:13pm

Roberto

I have been seeing a lot of this in the log:

Exception occurred during DoSURBLCheck: Access violation at address 005E397B in module 'SpamFilter.exe'. Read of address 2EC712EB

I have tried disabling the surbl checks by unchecking under custom domain filters, but the error is still logged.  Is this a known problem?  I am running x.x.635.  I sent you an email with the complete log of several threads with ip's and domains.

Thanks

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 19 January 2007 at 8:18pm
Got it, thanks. We had fixed this error in the new beta, but the one you reported is a different one and no, it was not a known problem. It is now!
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4066
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 20 January 2007 at 11:11am
Jerbo128,

Actually the bug you found was more serious, as the problem was occurring on practically every SURBL filter check, rendering that filter useless...

We've just uploaded a new beta with the fix (3.5.3.636). Thanks again for the report.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
 Post Reply Post Reply
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.078 seconds.