Zrinity Email Marketing Blog

The deliverability of your messaging gateway is highly dependent on other gateways, most often using older SMTP servers and often configured in unusual manners to address issues that have occurred over time. Until recently, administrators used a brute force approach to get the best delivery results and then often fine-tuned the number of connections and messages per connection to each of the major gateways. Newer email servers can manage a very large number of connections efficiently and often caused problems for administrators of older email servers. It’s like an unintentional denial of service attack that isn’t accepted well. The response has been countermeasures that reduce the number of connections available to servers that use that approach and with small networks, often just a hard block entered by the administrator. It’s a little embarrassing to contact the email administrator at some university about your deliverability issue and get a response that starts with, “So you’re the guy!”.

We created a Connection Authority Service in XMS to monitor connection performance with each recipient domain mail exchanger and constantly adjust the number of connections and messages per connection to achieve the best delivery results with the least amount of resources at each end of the connection. When messages are ready to send, a permit is granted or denied by the service. By marching to the beat of the other guy, we make it possible to move more messages through the pipeline by taking advantage of good connection performance periods and backing off during bad periods to allow them to recover. We also avoid tripping alarms that would put us into a longer delivery delays caused by dynamic blocks, hard blocks and crashes. The algorithm also supports the built-in load-balancing in the SMTP protocol by evaluating the connection performance of each mail exchanger when multiple mail exchangers are available for a recipient domain. There is an overriding mechanism to allow more connection resources to be allocated to domains that allow it. This is rarely used, but enables administrators who know that 75% of their email stream is to yahoo, to enable a much larger pool of connections to yahoo when allowed. You can also limit messages per hour to specific recipient domains, but that would no longer be useful, since the connection algorithm does that for you.

We found another use for the connection algorithm when we observed that some DNS servers were being swamped by XMS, causing packet loss and invalid results returned. That, in turn, caused delivery delays until the DNS cache within XMS was refreshed at a later time. By adding the connection algorithm to DNS queries, we assured better query results over time and eliminated wasted resources within XMS and the DNS server. We’ve also made the DNS server used for lookups configurable for those wishing to use a dedicated DNS server. Using a shared DNS server in a high volume gateway is often a drag on throughput, as there are sometimes thousands of queries made in a short period of time.

In the XMS Management Interface, you can see a visual representation of the current connection performance for each recipient domain. There is a list of command line errors in the same report. That list often contains a message indicating a block when action is taken to deny connections. I expect that within a few years there will be a standard for implementing sender reputation and user action feedback, using the current delivery status notification mechanism, so that gateways implementing the standard would be able to automatically make adjustments. For example, a recipient gateway could signal the sender that the sender is currently in a dynamic block condition for the ratio of deliverable to undeliverable email (a standard threshold, since it indicates a sender that does not handle feedback), and that a permanent block that would require human intervention to remove is pending. The sender gateway would read the standardized delivery status code and take action to suspend deliveries for that ip until administrator approval.

I’ve lumped a few things together here, because they are all closely related. They are all indications that you listen and respond. Your gateway is being tested for that every day. You have to use proper table manners, or you can’t come to dinner.