Spam Filter ISP Support Forum

  New Posts New Posts RSS Feed - Beta questions
  FAQ FAQ  Forum Search   Register Register  Login Login

Beta questions

 Post Reply Post Reply Page  <12
Author
dcook View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2005
Location: United States
Status: Offline
Points: 174
Post Options Post Options   Thanks (0) Thanks(0)   Quote dcook Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 3:00pm
I know we don't have an enterprise version for greylisting -- but I see we need it.  I have had several clients say they are getting bounces.  Here is what happened:
1. First email is sent to MX-10 and greylisting starts
2. Second retry email is send to MX-20
3. No more retry's - email is non-deliverable
 
First of all, the mail server should be more persistant trying more than twice for delivery. Second if enterprise servers shared the greylist - it would not matter.
 
Since I can't change the sender, I suggest that the next release include enterprise database storage of the greylist, please. Thanks
 
 


Edited by dcook - 10 January 2008 at 3:01pm
Dwight
www.vividmix.com
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 3:09pm
Dwight,
 
The enterprise mode does greylist but shares a single file.  I am not seeing the same issue you are.  Can you expand on it a little?
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 3:12pm

I use greylisting with 2 SFE's. I thought the same thing, that there might be instances of mail not getting through, but so far (since Sunday afternoon) I have not had anyone complain.

Would be nice to share a greylist database, but the question becomes are we willing to put up with a performance penalty. And if a greylist db became a reality, I would like to see a more standard version of greylisting using triplet info.

Perfomance penalty to me means its just time to move SFE's to beefier hardware LOL
http://www.webguyz.net
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 3:39pm
My 2.5 Cents.
 
I worry about the triplet setup in the case of some of our domains that have 2-3 thousand accounts.  2 problems here ... First, I am very happy that it only took a day or so to get most, if not all of AOL's, Hotmail's and the other "big guys" IP's to populate the list and therefore not delay messages to our customers.  This would not be the case if the "triplet" setup was used.  Second, I already have well over half a million lines in my GreyListAllowed.  I can not imaging how large it would get if the triplet method was used.
 
On the other side of the fence, the triplet *seems* like it would be more effective over the long term ... seeing that Spammers always figure out what is going on and always adapt. 
 
So ... which is best?  dunno yet.
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
dcook View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2005
Location: United States
Status: Offline
Points: 174
Post Options Post Options   Thanks (0) Thanks(0)   Quote dcook Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 3:45pm

There I go assuming things again - I thought the limbo or temporary greylist was NOT shared. 

I have a client that complained about getting bounces.  They are a mortgage company and can't afford to miss clients emails.  I did a trace and found that several of their missing emails went to our first MX record (SF) and were put in the limbo grey area.  The next email went to another MX (different SF) and it too was put into the grey limbo. So the email source saw the greylist messages and said they were being bounced.  My mortgage company never got the email from the sender.

I had the mortgage company in a separate individual configuration of SFE but since the greylist is a global configuration, I could not tweak this clients email settings.  For the time being I am just forwarding all of their email unfiltered directly to their exchange server.

I have not had complaints from other clients on the greylist.

 

Dwight
www.vividmix.com
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 3:56pm
Dwight,
 
So the sender stopped attemting to send after 2 rapid fire attempts?  They should have retried *at least* for a couple of days ... even 1 day.
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
dcook View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2005
Location: United States
Status: Offline
Points: 174
Post Options Post Options   Thanks (0) Thanks(0)   Quote dcook Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 4:00pm

Some "so called" IT people can't help but tweak those mail server settings.  (smile)

Dwight
www.vividmix.com
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 4:15pm
Originally posted by dcook dcook wrote:

I know we don't have an enterprise version for greylisting -- but I see we need it.  I have had several clients say they are getting bounces.  Here is what happened:
1. First email is sent to MX-10 and greylisting starts
2. Second retry email is send to MX-20
3. No more retry's - email is non-deliverable
 
First of all, the mail server should be more persistant trying more than twice for delivery. Second if enterprise servers shared the greylist - it would not matter.


Per RFC2821, when there are multiple MX records, the sender MUST try sending an email to all servers in the list if one is unavailable. This would mean that if an attempt is made to the primary MX server, and this fails due to the graylisting, the remote server should then immediately try connecting to the secondary server. The relevant section of the RFC is:

   When the lookup succeeds, the mapping can result in a list of
alternative delivery addresses rather than a single address, because
of multiple MX records, multihoming, or both. To provide reliable
mail transmission, the SMTP client MUST be able to try (and retry)
each of the relevant addresses in this list in order, until a
delivery attempt succeeds. However, there MAY also be a configurable
limit on the number of alternate addresses that can be tried. In any
case, the SMTP client SHOULD try at least two addresses.

Now, this process will fail for the first attemp due to the greylisting. The same RFC2821 now states that the sender MUST retry a particular destination if the first attempt has failed. This means that the remote server MUST retry sending the email to the primary MX server. if you see servers that are not being able to deliver mail to you in your configuration, it's very likely that they are violating this RFC, which is considered the RFC when it relates to emails. The relevant section here is:

   The sender MUST delay retrying a particular destination after one
attempt has failed. In general, the retry interval SHOULD be at
least 30 minutes; however, more sophisticated and variable strategies
will be beneficial when the SMTP client can determine the reason for
non-delivery.

Originally posted by Desperado Desperado wrote:

Dwight,
 
The enterprise mode does greylist but shares a single file.  I am not seeing the same issue you are.  Can you expand on it a little?

Dan, I have to correct you on this one. Each SpamFilter uses its own copy of the greylist file, and that file is only imported once when SpamFilter startup. Unlike all other configuration files, this one is not reloaded by SpamFilter if modified externally.



Originally posted by WebGuyz WebGuyz wrote:

Would be nice to share a greylist database, but the question becomes are we willing to put up with a performance penalty. And if a greylist db became a reality, I would like to see a more standard version of greylisting using triplet info.
Perfomance penalty to me means its just time to move SFE's to beefier hardware LOL

WebGuyz, using the full triplet would be potentially a disaster waiting to happen in large installations. If, in addition to keeping track of individual IPs, we also kept track of the to/from email addresses for all emails regarding that IP, the number of entries in the list would row exponentially, going from the millions of entries you will be reaching now, to the billions. SpamFilter is efficient, but that may be too much for a dinky 3MB executable!
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 4:24pm
Roberto,
 
I was refering to the single file in the SFE configurating which I thought was shared by ALL domains under that same SFE instance.  Am I wrong there?  I did not mean that the file itself could be shared by differant servers instances.
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 4:29pm
Originally posted by Desperado Desperado wrote:

Roberto,
 
I was refering to the single file in the SFE configurating which I thought was shared by ALL domains under that same SFE instance.  Am I wrong there?  I did not mean that the file itself could be shared by differant servers instances.

Misunderstood you. Yes, in this case you're absolutely correct. The file will apply to the entire SE installation, for all the domains it handles.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 4:32pm
Whew!  Thought I had lost my mind! ... Well ... that may still apply!Geek

Edited by Desperado - 10 January 2008 at 4:32pm
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 4:38pm
Originally posted by LogSat LogSat wrote:


WebGuyz, using the full triplet would be potentially a disaster waiting to happen in large installations. If, in addition to keeping track of individual IPs, we also kept track of the to/from email addresses for all emails regarding that IP, the number of entries in the list would row exponentially, going from the millions of entries you will be reaching now, to the billions. SpamFilter is efficient, but that may be too much for a dinky 3MB executable!
 
Like I said, Wink way beefier hardware Wink
Think I saw a old CRAY super computer available on Ebay or CraigsList ....
http://www.webguyz.net
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 10 January 2008 at 4:48pm
I remember another poster talking about a greylisting proxy server in front of all their SF servers. Maybe that would be a better way to go then trying to share it in a SFE table. The technology definitely works, but is it scalable is the question.
http://www.webguyz.net
Back to Top
atifghaffar View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 May 2006
Location: Switzerland
Status: Offline
Points: 104
Post Options Post Options   Thanks (0) Thanks(0)   Quote atifghaffar Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2008 at 12:20am
Webguyz,

The proxy will have to proxy the connection to keep all the connection information. (so all other tests based on the ip address can be performed)

My solution was with a NATting firewall.

I still have a firewall in front of the SF boxes but it now does just the blocking based on the limbo information.

Dont know how this can be done in windows.

The rules are quiet simple.
if ip in limbo drop
if ip in greylistok forward to another ip

I can share my firewall script if someone is willing to implement it in windows.

best regards

Atif
Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2008 at 12:30am
Was going to try this transparent proxy: http://www.hermes-project.com/pages/hermes
But then I realized there still is no easy way to go from a single central greylisting proxy to 2 or more SFE servers unless you get something like a load balancer.
 
Don't really want to implement this in firewall as I may be uprading mine in the near future.  Will keep looking around and fine tuning.
 
I marvel at how much of my time is spent trying to stop spam. What a shame ...Dead
http://www.webguyz.net
Back to Top
dcook View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 January 2005
Location: United States
Status: Offline
Points: 174
Post Options Post Options   Thanks (0) Thanks(0)   Quote dcook Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2008 at 10:57am

 

 

Originally posted by Desperado

Roberto,

 

I was referring to the single file in the SFE configurating which I thought was shared by ALL domains under that same SFE instance.  Am I wrong there?  I did not mean that the file itself could be shared by differant servers instances.


Misunderstood you. Yes, in this case you're absolutely correct. The file will apply to the entire SE installation, for all the domains it handles.

 

Each spam filter server currently keeps it own independent grey list. In our case, in this spam filter build SFE-1 and SFE-2 keep their own, separate grey list that is used globally per server. The information is never shared between SFE-1 and SFE-2.

 

Roberto did you say that in a future build grey list sharing for the enterprise version using the database should be supported?

 
Dwight
www.vividmix.com
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2008 at 5:07pm
Originally posted by dcook dcook wrote:

 

Roberto did you say that in a future build grey list sharing for the enterprise version using the database should be supported?

 

Originally that was the plan, but that was during our early development stages, before we had a chance to see how well this beta worked :-)
For now we'll wait and see, as storing that list in a database does concern us due to its size and the frequency of updates to it.
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
atifghaffar View Drop Down
Senior Member
Senior Member
Avatar

Joined: 31 May 2006
Location: Switzerland
Status: Offline
Points: 104
Post Options Post Options   Thanks (0) Thanks(0)   Quote atifghaffar Quote  Post ReplyReply Direct Link To This Post Posted: 11 January 2008 at 6:19pm
Roberto,
How about an option for the admin to decide  the interval when to flush the greylist to the db. and how often it should be reloaded.
I think a "select ip from table where id>max_id_we_have" every 10 minutes will not be so resource intensive.


best regards

Atif
Back to Top
Guests View Drop Down
Guest Group
Guest Group
Post Options Post Options   Thanks (0) Thanks(0)   Quote Guests Quote  Post ReplyReply Direct Link To This Post Posted: 15 January 2008 at 7:04am
Do have a little error that seems to be returning:
 
01-15-08 12:59:25:096 -- (4340) Exception occurred during TTimerMinuteTimerThread.DoUpdateChartCountries: Access violation at address 00401D3D in module 'SpamFilterSvc.exe'. Write of address 00000001
Back to Top
LogSat View Drop Down
Admin Group
Admin Group
Avatar

Joined: 25 January 2005
Location: United States
Status: Offline
Points: 4065
Post Options Post Options   Thanks (0) Thanks(0)   Quote LogSat Quote  Post ReplyReply Direct Link To This Post Posted: 15 January 2008 at 6:12pm
sysiq, can you please let us know what build of SpamFilter you are using?
Roberto Franceschetti

LogSat Software

Spam Filter ISP
Back to Top
caratking View Drop Down
Groupie
Groupie


Joined: 13 March 2006
Location: United States
Status: Offline
Points: 79
Post Options Post Options   Thanks (0) Thanks(0)   Quote caratking Quote  Post ReplyReply Direct Link To This Post Posted: 15 January 2008 at 11:54pm
Just tossing out an idea here...

Currently configuration information is in a database and shared by all servers.

If you want to share your grey list, putting millions of IP Addresses into a database is probably going to be a BIG hit on performance.

Why not enable server to server communication directly between spamfilter machines?  There is no configuration, no changes are done by the administrator.  It is only the machines that need to share the information.  Have them do a sync directly between themselves.

Servers are registered in the database anyway, have each server update the database with its IP Address and some type of random key.  The servers can use that info to find each other and the key to ensure they are taking with who they think they are.


Thats it...


Back to Top
Simone View Drop Down
Groupie
Groupie


Joined: 06 July 2005
Status: Offline
Points: 42
Post Options Post Options   Thanks (0) Thanks(0)   Quote Simone Quote  Post ReplyReply Direct Link To This Post Posted: 23 January 2008 at 1:29pm
A little question:

On SFE the Grey Listing could be apllied to all domains or you can choose wich one use and wich one will not?

Reading your post it seems the first option the right one, but will the second be possible in the future?

Thank you,
Simone
Back to Top
Desperado View Drop Down
Senior Member
Senior Member
Avatar

Joined: 27 January 2005
Location: United States
Status: Offline
Points: 1143
Post Options Post Options   Thanks (0) Thanks(0)   Quote Desperado Quote  Post ReplyReply Direct Link To This Post Posted: 23 January 2008 at 1:36pm
Simone,
 
The GrayListing works at the SMTP level and as a result, it is applied to ALL domains on the SFE instance.
The Desperado
Dan Seligmann.
Work: http://www.mags.net
Personal: http://www.desperado.com

Back to Top
WebGuyz View Drop Down
Senior Member
Senior Member


Joined: 09 May 2005
Location: United States
Status: Offline
Points: 348
Post Options Post Options   Thanks (0) Thanks(0)   Quote WebGuyz Quote  Post ReplyReply Direct Link To This Post Posted: 24 January 2008 at 9:38am
I love getting emails like this from our customers ;-)
 
(He's referring to the amount of spam in quarantine that he has to slog thru)
 
----------------------------------------
I have noticed a huge decrease in the number of spams I get every day. It seems like about 60% have gone away.

Did you put them all in jail, or have they given up on me?

I don't want them back, I am just curious as to what has changed.

Thanks

Chuck
-----------------------------------------



Edited by WebGuyz - 24 January 2008 at 9:40am
http://www.webguyz.net
Back to Top
kspare View Drop Down
Senior Member
Senior Member


Joined: 26 January 2005
Location: Canada
Status: Offline
Points: 334
Post Options Post Options   Thanks (0) Thanks(0)   Quote kspare Quote  Post ReplyReply Direct Link To This Post Posted: 24 January 2008 at 10:39am
haha i've been getting the opposite emails. People are wondering if we've been having problems because they don't have the usual amount of spam! haha
Back to Top
 Post Reply Post Reply Page  <12
  Share Topic   

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.078 seconds.