8/15 2003 SMTP on mail1.hsphere.cc

[jgann]

Originally posted by admin
...the mail server sending it to the mail server ( If setup right ) should retry the mail again for at least 5 days. There is not much we can do if the other end is not setup to resend the emails like SMTP should this is standered for all SMTP to retry messages if not sent the first time.

OK, I did not get an important email that has cost me a serious delay in an equipment order. When I first became aware of the problem I sent periodic test emails from both inside matrix and outside. If the above is true (in terms of re-sending) then why doesn't the test emails from matrix re-send to me later? Or where those disrupted as well? Not angry - Just trying to understand and learn.

[Allergy]

Originally posted by garygpg
I think that'd me remarkably incovenient for those of us who send out regular newsletters or updates to legitimate opted-in subscribers.

It should be approved once.

[Curt]

Originally posted by alexc
munizjm: messages that don't arrive within a few hours of service being restored have either been silently swallowed by the sending system or purged by mistake after arrival. I won't take the blame for all disappearing e-mail on mail1 but I consider it probable that some of it was eaten by one of our queue sweeps too. If you submit a TT with the addresses I can check and see whether e-mail from individual senders has been recorded as arriving here or not and report back to you.

Hi Alex. Could you possibly explain to us how the queue sweeps work? If they're potentially "eating" valid and possibly important email, is there nothing that can be done to make the sweep more selective about what it should delete? Thanks for any light you can shed on the subject.

These queue sweeps are initiated manually, sometimes with quick-fix on the spot scripting, and are therefore inevitably more sensitive to human error. $DEITY knows we're not perfect. What happens in those cases (and the more often it's done the worse) is that there's a greater possibility that the filter will not be narrow enough and some legitimate mail will end up in the bin along with the junk. This is also why you may receive messages A, C and D but lose B for being stuck in the wrong queue at the wrong time.

On the other hand, the alternative is to attempt to let it blow over and let *everything* rot in the queue with 1000 unhappy resellers breathing down our collective neck... I'm loath to taking risks, even calculated ones, with people's e-mail but desperate situations call for a decisive approach to the solution. The joys of system management...

That said, we'll establish better procedures and increase preparedness for dealing with assaults of this magnitude, even though this is probably a uniquely egregious case of abuse and we don't expect it to repeat itself anytime soon. Lose/lose situations suck.

[jgann]

When these things do happen is there any way to have the mail bounce back to senders as undeliverable? I would think that better than them thinking it went through. At least then they would re-send or contact someone.

Therein lies the problem and therefrom arise the complaints. If something's unexpectedly erased after being accepted by the server and while queued for local delivery, there will be no bounce, it just disappears. Life just isn't that good and they don't make really smart MTAs yet.

[Curt]

Thanks very much for the explanation Alex.

Would it be an option in the future to lessen the agressiveness of the filter or make the search string(s) more focused? We're all pretty used to spam, so a little more getting through won't be an issue I'm sure, and this would make it less likely that legitimate mail would be bit-bucketed as well. Thoughts?

Certainly. We've already had the experience of spam control going overboard and won't be travelling down that road again anytime soon. Unlike spam filters, though, this was not a preemptive measure but a piecemeal (and obviously error-prone) cleanup of massive amounts of unwelcome data that had already been injected into the local mail delivery system while trying different methods to speed up the process and restore service ASAP.

[Brangwyn]

little OT I know, but I just read a survey today actually that claims around 38% of all legitimate mail is blocked/lost/swallowed by black holes around the Net due to "Spam Filtering", shows we still have a long way to go to find effective filtering techniques (I don't think we're ever going to stop spam altogether personally).

[Curt]

I agree Brangwyn, though I believe as Bayesian filters become more popular (and maybe even networked Gnutella-style), we will have much better filtering with fewer false positives. For those not familiar with Bayesian spam filtering, About.com has a bit more info at http://email.about.com/cs/bayesianfilters/.

Thanks again Alex for your explanation. I'm sure a number of us in here are programmers and can appreciate the situation of trying to throw together a program as fast as possible without any issues or bugs. It's generally an impossibility. I was just suggesting that if there is a next time for this type of situation (which is unfortunately quite realistic), maybe it might be better to allow more of the backlogged spam through if it means less legitimate mail gets tossed as well. ie: instead of killing every message with the word "diploma", kill every message with "diploma", "unaccredited", "life experience", "click here", and "privacy of your own home". Here's hoping that it's never an issue again anyway. Cheers! <tilting back a Coke>

[Brangwyn]

One of the best filters I used recently was Cloudmarks SpamNet which was based on a centralised database of known spam, I stopped using it when they went out of Beta and went subscription based. I'm using Spam Inspector now and thats fairly accurate certainly helps a bit anyway.

[somereseller]

Most of the messages that I have lost were sent from customers hosted at Vortech. I don't see how a sweep script that removes emails coming from outside could have removed those internal messages.

[thesourcez]

Still intermittent mail.
What is the story there?
How can a silly company like Hotmail or Yahoo always deliver the mail? Good or bad?
I just sent another test batch and NO mail for 2 hours.
Yes I did reboot and reconnect to the internet. Still nothing.
What gives? I thought the whole idea of self hosting was to have control of our accounts?

[Vixen]

Hotmail is who caused the issue in the first place. They are the ones who allowed all the mail to bomb into our server. So, yea they can get the mail delivered. And in the process back up everyone else's servers.

[admin]

Originally posted by thesourcez
Still intermittent mail.
What is the story there?
How can a silly company like Hotmail or Yahoo always deliver the mail? Good or bad?
I just sent another test batch and NO mail for 2 hours.
Yes I did reboot and reconnect to the internet. Still nothing.

I see nothing wrong on mail1 it has about 250 messages in the queue and working just fine.

Originally posted by thesourcez
What gives? I thought the whole idea of self hosting was to have control of our accounts?

Hmm, that would be if you had your own server, a reseller package is more like managed hosting, it does not give you any where close to full control like your own system still. VPS would be the next closest thing.