LogSat Software

Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic

   Kevin,

It's a new filter based on a suggestion by Lee ( www.logsat.com/spamfilter/forums/forum_posts.asp?TID=/5531&a mp;PN=1#7577).

A few months ago we created a new filter, the IP blacklist cache. This filter causes any IP that sends multiple spam emails in a short period of time to be added to a temporary memory cache in SpamFilter, creating sort of a local, superfast blacklist.

Starting from v3, when SpamFilter will add an IP address to this memory cache, it will also upload its information (what the IP is, why it was blocked) to our centralized database. As more users begin to deploy SpamFilter v3, this database will grow very rapidly, and will contain in essence a copy of all the blacklist caches of all the SpamFilters in the world. As the cache practically contains the IPs of repeated spammers, the centralized DB will contian a dynamic, ever changing, self-updating list of IPs that are currently being blocked by all the SpamFilters in the world. 

Every copy of SpamFilter is then able to query this database in realtime, and see if the connecting IP is listed in the database. SpamFilter will then query the database to see how many different users have reported that IP address at that time, and will apply a "confidence check" (meaning more users have reported the spammer, the less likely it is a false positive, and will decide if the email should be blocked or not.

The database is updated in realtime when an IP is added and removed from the cache, and furthermore we perfrom hourly "cleanup" on it to remove stale entries that have not been updated in the last 24 hours.

This is all experimental with this new version, and in a few weeks we'll be able to see hwo this filter performs.

Author	Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic
LogSat Members Profile Send Private Message Find Members Posts Add to Buddy List Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4106	Post Options Post Reply Quote LogSat Report Post Thanks(0) Quote Reply Posted: 03 April 2006 at 12:28pm
	Kevin, It's a new filter based on a suggestion by Lee ( www.logsat.com/spamfilter/forums/forum_posts.asp?TID=/5531&a mp;PN=1#7577). A few months ago we created a new filter, the IP blacklist cache. This filter causes any IP that sends multiple spam emails in a short period of time to be added to a temporary memory cache in SpamFilter, creating sort of a local, superfast blacklist. Starting from v3, when SpamFilter will add an IP address to this memory cache, it will also upload its information (what the IP is, why it was blocked) to our centralized database. As more users begin to deploy SpamFilter v3, this database will grow very rapidly, and will contain in essence a copy of all the blacklist caches of all the SpamFilters in the world. As the cache practically contains the IPs of repeated spammers, the centralized DB will contian a dynamic, ever changing, self-updating list of IPs that are currently being blocked by all the SpamFilters in the world. Every copy of SpamFilter is then able to query this database in realtime, and see if the connecting IP is listed in the database. SpamFilter will then query the database to see how many different users have reported that IP address at that time, and will apply a "confidence check" (meaning more users have reported the spammer, the less likely it is a false positive, and will decide if the email should be blocked or not. The database is updated in realtime when an IP is added and removed from the cache, and furthermore we perfrom hourly "cleanup" on it to remove stale entries that have not been updated in the last 24 hours. This is all experimental with this new version, and in a few weeks we'll be able to see hwo this filter performs.
	Roberto Franceschetti LogSat Software Spam Filter ISP

kspare Members Profile Send Private Message Find Members Posts Add to Buddy List Senior Member Joined: 26 January 2005 Location: Canada Status: Offline Points: 334	Post Options Post Reply Quote kspare Report Post Thanks(0) Quote Reply Posted: 03 April 2006 at 11:54am
	Roberto, can you explain SFDB a little bit more? What black lists is it based on?

Marco Members Profile Send Private Message Find Members Posts Add to Buddy List Senior Member Joined: 07 June 2005 Location: Netherlands Status: Offline Points: 137	Post Options Post Reply Quote Marco Report Post Thanks(0) Quote Reply Posted: 03 April 2006 at 9:37am
	Great work on the new 3.0 version. Will run it a couple of days to find bugs. found one allready :) - The shrunk qdb lines are back im sorry to inform you about (you know, the qdb lines reducing height to 2 pixels after a refresh)
	Anyone who is capable of getting himself made president, should on no account be allowed to do the job. D.Adams

kspare Members Profile Send Private Message Find Members Posts Add to Buddy List Senior Member Joined: 26 January 2005 Location: Canada Status: Offline Points: 334	Post Options Post Reply Quote kspare Report Post Thanks(0) Quote Reply Posted: 02 April 2006 at 5:13pm
	The image filter caught 3 messages so far today. Two were obvious spam the 3rd was a sales rep blasting out emails to his customers. kida grey area but i'm personally fine with it sitting in there. Good Job!

kspare Members Profile Send Private Message Find Members Posts Add to Buddy List Senior Member Joined: 26 January 2005 Location: Canada Status: Offline Points: 334	Post Options Post Reply Quote kspare Report Post Thanks(0) Quote Reply Posted: 02 April 2006 at 11:31am
	The config tab is MUCH better! Kevin Edited by kspare

LogSat Members Profile Send Private Message Find Members Posts Add to Buddy List Admin Group Joined: 25 January 2005 Location: United States Status: Offline Points: 4106	Post Options Post Reply Quote LogSat Report Post Thanks(0) Quote Reply Posted: 31 March 2006 at 4:43pm
	We've made available as a pre-release the new SpamFilter ISP v3.0.0.547. The major additions to this new release are: The availability of a new filter that scans images embedded in emails for spam content. A new SFDB (SpamFilter Distributed Blacklist) filter that will rely on the large network of SpamFilter users to create and maintain automatically a centralized blacklist of spammer's IP. A new web management interface is also being worked on, but due to its complexity it still has not been enabled on this beta. We hope to do so in the coming weeks. Please note that this version is to be considered as a beta, not as a pre-release, as further testing will be required. The release notes are as follows: // New to VersionNumber = '3.0.0.547'; {TODO -cNew : Added a new filter to detect spam in images embedded inside emails} {TODO -cNew : Added new SpamFilter Distributed Blacklist filter} {TODO -cNew : Cosmetic touchups to the Configuration tab GUI} {TODO -cNew : Added safety code to remove duplicates added by external apps to AllowedDomains.txt local domains} {TODO -cNew : Removed annoying "Exception occurred during OnConnect" exceptions in logs} {TODO -cFix : Sometimes Socket Errors on MX test could cause rejects (catches even more cases than in build 535)} {TODO -cFix : Regression error in build 541 caused SURBL and attachment filter to stop working}
	Roberto Franceschetti LogSat Software Spam Filter ISP

LogSat Software

Site Navigation[Skip]

Spam Filter ISP Support Forum

SpamFilter ISP v3 beta available