(this is completely non-jaguar related. However, affected parties, and
those interested in mail might want to read this anyway. Post followups to
the POLICY list only please, which is where this stuff belongs.)
At 09:25 2004-12-18 -0500, Lou Danzico wrote:
Have been off the XJ list since the 8th and could not re-subscribe until
yesterday. Got an email from Bob Grossman who says he’s the XK-Engine list
administrator and that Verizon, my server, firewalled J-L .
Is Mr. Grossman the list administrator?
For XK-Engine, yes. Now your problem is who the heck am I and why would
you believe what I say…
I note that you’re sbscribed to XK-Engine. When you sbscribe to a list,
a welcome message is sent out automatically by the server - most admins
identify themselves and provide an introduction in that message.
You could contact the listadmin directly off-list. Just prepend “owner-”
to the list address in question. A search of the list archives would also
net you an answer - advanced search, specify xk-engine for the list and
search for “grossman administrator”.
Note that the XJ list is a separate list from XK-Engine (er, meaning that
the XK list certainly seems more appropriate to post this query on, though
POLICY is still by far the most appropriate). Each list, and even
model-specific website section has a different aministrator (well there’s a
few people who tackle more than one list or manage both a list and website).
Now, as to the problem being reported…
Verizon (and a few other ISPs) recently started using a new technique for
dealing with inbound emails - they call back to the sending server DURING
the original mail transaction and attempt to authenticate the sender by
starting a separate mail transaction of their own. This is something like
you calling someone, having them place you on hold, and then calling you
back (which assumes that you have the capacity to answer another inbound
call, even though you have MANY such calls going on for other things),
going through some chatter there, then getting back to your original call
and saying “ok, where were we?”, OR rudely hanging up because their own
attempt to call you back wasn’t responded quickly enough. When you’re a
list server, you’ve got a LOT of these connections going simultaniously -
messages you’re delivering out to sbscribers, and inbound messages from
sbscribers, not to mention all the attempts to flood the server with junkmail.
Verizons implementation of this non-standard callback test is very tight on
timing, and does not allow for normal timeouts as can occur in some
connections. When Verizon doesn’t promptly get the response they want,
they respond by FIREWALLING the sending server (that’d be Jag-Lovers in
this case), rather than simply refusing the email. So, our server
continues to attempt to deliver the message every 15 minutes for FIVE DAYS,
since from a technical standpoint, the problem appears to be a network
issue with verizon, not a message refusal (which would bounce the message
and then we wouldn’t be trying to send THAT message). Note that it’s not
just one message in the queue - it’s 100’s (and even 1000’s) of messages
the server is trying to deliver to an ISP that’s just ignoring the mail
connection attempts instead of simply refusing the messages. It’s a
massive waste of bandwidth and processing cycles, which would be better put
to use for the website, etc.
Another ISP was hammering us back with multiple callback requests for EACH
sent message, and like Verizon, was still not allowing for a reasonable
timeout on their attempts (which took on the distinct appearance of a
denial-of-service attack). They even hammered other servers associated
with our mail delivery. Very, very rude. No explanation of what they were
doing was published ANYWHERE by them - the only documentation of it is from
other technical people discussing the problem and postulating on its cause.
JFTR, this timeout they’re using is considerably less than the amount of
time it takes for many large ISP mail servers to even respond to an inbound
SMTP connection, so if we were using the same tecnique with the same
undocumented timeouts, we’d be refusing their mail because their servers
are so slow (largely because of the volumes of mail they process). As it
is, they’re messing with established mail standards and not even
DOCUMENTING what they’re doing in the process. If it was a legitimate
methodology, it’d be documented, and if it was really stable and effective,
there wouldn’t be a problem documenting it – if documenting it would cause
it to become an unreliable method, then it is a faulty implementation to
begin with. In the industry, we call this “security by obscurity”. It’s
very bad mojo. We wouldn’t care if they were only causing trouble for
themselves, but that’s not the case - their poor implementation is causing
trouble for others, ourselves included.
This problem with verizon, tiscali, and some other ISPs isn’t limited to
just Jag-Lovers – lots of other hosts suffered the same mail
problems. That means that Jag-Lovers isn’t the only mail your ISP caused
you to LOSE. Permanently gone. Poof. Verizon appeared to simply be dead
in the water because their servers didn’t respond - even to pings - from a
variety of hosts, not just the Jag-Lovers server and the network it is on,
but from other networks as well. Remember @home ? They were big, and they
went off the air in the same fashion - just gone one day, all the mail
backing up on the servers that were sending it. This looked no different.
The J-L mail administrators (myself included) spent some time investigating
possible causes and isolated a connection to an external DNS server which
was taking circa 20 seconds to complete, which contributed to the delay
(and timeouts was a shot in the dark as to a possible cause, since the ISPs
responsible for the mess haven’t documented what they’re doing), it should
be noted that all deliveries and inbound mails except for these outfits
employing this new technique had no problems sending and receiving
messages, even when encountering the exact same connection delays - just
the handful of ISPs that implemented some new nonstandard mail hack
suddenly started refusing to accept messages.
The icing on the cake is that at the moment, the only reason Verizon
customers are even recieivng listmail from us is because we managed to
circumvent the Verizon firewall so that we can communicate with a verizon
mail server. Their mail firewalling is STILL in place if we were to
attempt normal mail delivery.
A futher analysis of the SMTP callback technique indicates that it is a
very brittle test AND if more ISPs adopt it, it’ll lead to widespread
sender forgery - which is all that is necessary to circumvent it, and it
isn’t as if the people sending junk care. This in turn will mean more junk
in everybody’s mailbox because of bounces and misdirected complaints being
sent to real addresses which were forged. Not good.—
Sean Straw Admin of the Jag-Lovers XJ-S list
Sonoma County, California
===================================================
The archives and FAQ will answer many queries on the XJ series…
FAQs: http://www.jag-lovers.org/xjlovers/xjfaq/index.html
Archives: Jag-lovers Forums - Jag-lovers
To remove yourself from this list, go to Jag-lovers Forums - Jag-lovers.
// please trim quoted text to context only