Print Page | Close Window

spam catch rate

Printed From: LogSat Software
Category: Spam Filter ISP
Forum Name: Spam Filter ISP Support
Forum Description: General support for Spam Filter ISP
URL: http://www.logsat.com/spamfilter/forums/forum_posts.asp?TID=6898
Printed Date: 18 October 2017 at 10:44am


Topic: spam catch rate
Posted By: dee
Subject: spam catch rate
Date Posted: 19 November 2010 at 12:24am
Hello, am new in this forum and I was just wondering if there is anyone who could help me with the formula for spam catch rate. I know that we can calculate it using no of spam blocked over no of total spam. what if I do not have the total number of spam? is there any other way to calculate catch rate without using knowing the total number of spam that the filter did not block.



Replies:
Posted By: LogSat
Date Posted: 19 November 2010 at 11:01pm
dee,

You can retrieve this information by issuing a couple of commands from an MSDOS prompt on SpamFilter's server, using the "FIND" command to count the number of lines in SpamFilter's activity logfile that contain certain strings.

For example, to count the total number of email attempts received by SpamFilter, you can look for the number of lines that contain the text "Connection from". To do this, open an MSDOS prompt, navigate to SpamFilter's logfile directory, and issue the command:

D:\SpamFilter\logfiles>find /c /i "Connection from" 20101118.log
---------- 20101118.LOG: 26031

You will see the number of matching lines in the result - 26,031 email attempts in the above example.

You can then count the number of emails that SpamFilter forwarded to your email server - these are the "clean", non-spam emails that SpamFilter identified. This is done by looking for the number of lines that contain the text "was queued". The command in this case is:
D:\SpamFilter\logfiles>find /c /i "was queued" 20101118.log
---------- 20101118.LOG: 238

In this example that number is 238.

The percentage of clean emails allowed is then (238/26031) * 100 = 0.9%
The number of emails blocked will be 100 - (% of clean emails), in the above case it's then 100% - 0.9% = 99.1%

The above numbers were actual numbers for the amount of emails we received yesterday at logsat.com. Please note that we receive a *lot* of spam, and are thus blocking a very high percentage of it. In other cases usually the amount of spam varies between 70%-90% of the total emails received (instead of the 99.1% we receive).

 





-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: yapadu
Date Posted: 20 November 2010 at 8:19pm
Hey that is cool, I ran the numbers on our servers:

Primary Server:

Connection from: 193818
Was queued: 21899

Clean mail: (21899/193818) * 100 = 11.29%

So spam = 100 - 11.29 = 88.71 spam


Backup Server:
Connection from: 116411
Was queued: 106

Clean mail: (106/116411) * 100 = 0.09%

So spam = 100 - 0.09 = 99.91% spam


I would be willing to be that the majority of those 106 messages that were let through on the backup were actually spam.  It is rare that real mail is processed by our backup, just imagine if we did not have spam protection!


-------------
--------------------------------------------------------------
I am a user of SF, not an employee. Use any advice offered at your own risk.


Posted By: LogSat
Date Posted: 20 November 2010 at 10:31pm
yapadu,

It's also possible that some of those 106 emails were actually whitelisted due to some of the whitelist filters, which would make the spam catch ratio even more impressive on your secondary Smile

Joking aside, that has to be one of the highest ratios we've seen in over 8 years!


-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: dee
Date Posted: 22 November 2010 at 4:54am
Thank you guys for your responses, however, still have a problem because I cannot access a server, have been given reports and logs to analyse the situation and finally make recommendations, These reports give me total number of emails and spam blocked, not the spam that was falsely identified as ham.

What do I do in this kind of situation.


Posted By: LogSat
Date Posted: 22 November 2010 at 11:06pm
dee,

I'm not sure what you mean by "cannot access a server". Please do note that to install and run SpamFilter you will need to have administrative rights on the server it is installed on.

To find out what spam was mis-identified as ham, the only way to be certain of that amount is to go thru each and every single email received visually, and manually identify what is spam and what is not for multiple users over a large period of time (ex. a day). Only a human being will be able to correctly determine with fair certainty (I say fair as even humans can sometimes not be sure if some emails are to be considered spam or not... ex mailing lists) what kind of emails have been received and thus their spam/clean ratio.


-------------
Roberto Franceschetti

http://www.logsat.com" rel="nofollow - LogSat Software

http://www.logsat.com/sfi-spam-filter.asp" rel="nofollow - Spam Filter ISP


Posted By: AndrewD
Date Posted: 24 November 2010 at 12:00am
Dee,
Everything above is correct, No system out there can do what you are asking, but there is another alternative.
I look at it in house on my email address and can judge it for my account. As my account is an email address that has been around for many many many years and is a published contact i receive a LOT of spam. So if i simply move the spam that does get through into a seperate box, and then look at it on a period (Monthly) basis.
 
find /c /i "to  mailto:ada@ntbm.com.au - {insert your email address} was queued." 20101118.log
 
You would need to repeat this on each log for the month.
then you will have a count of emails delivered, and also a count (From the box mentioned above) of the false ham's. This will give you a ratio, that you could use as a guide for the rest of your network. Also i use the false hams to customize my filters to improve catch rate.
 
The other issue that you need to consider is the false positives, This is a lot easier to get an idea of as everytime a user releases an email from the quarantine section it will be added to the database for the users whitelist. then you can add the entries and divide that by the total blocked to give you a false positive count.


-------------
Spamfilter web interface. www.tyrexpg.com.au

See http://www.logsat.com/SpamFilter/Forums/forum_posts.asp?TID=6883



Print Page | Close Window