Ref: 11640023
Title: Optimizing 3+Mail and 3+Route to Facilitate a "Big Send"
date:  6/27/89

Copyright 3Com Corporation, 1991.  All rights reserved.

This article explains how 3+Mail and 3+Route work and discusses a
specific implementation in which large quantities of electronic
mail are to be sent quickly and reliably between servers all over
the United States.


How 3+Mail Works

When a mail message is sent from a netstation the names and groups
on the to: and cc: line are expanded to their full three-part names and
then sent to the mail server.

When the mail server receives a message it puts the main body of
the message in the MESSAGE subdirectory of 3MAIL and puts an
envelope in the FORWARD subdirectory for each mail server that the
message is to be delivered to.  The FORWARD subdirectory is made
up of many subdirectories that look like this:

   C:\3PLUS\3MAIL\FORWARD\00048532\00000000\00000463\1395876C.E

Looking at this example, the first subdirectory name after
FORWARD is the network number of the domain that the message is
to go to (00048532).  The next subdirectory name is the first
eight Ethernet address characters of the addressee's mail server.
(In this case, the subdirectory is 00000000 because the mail
server's Ethernet address is unknown.)  The first four digits of
the next subdirectory name are the last four digits of the
Ethernet address of the mail server (0000).  The final four
digits are the socket number for mail (0463).

The files in this nest of subdirectories are the envelopes for
the message.  The first eight characters of the envelope filename
is a hashed date- and time-stamp of when the message was sent.
It corresponds directly to a file in the MESSAGE subdirectory.
They have the same filename but different extensions.  The "E"
signifies that it is an envelope.

The envelope contains five pieces of information:  the error
code, where it is going, who it is from, where it came from, and
the list of users to receive the message.  For example:

                0
                00048532000000000463
                robinj:cso
                000400FC02608c0749740463  Mail:CSO:3Com
                \Tom Johnson:HQ:Comp\Jan Wong:HQ:Comp

Once the envelope is created the mail server will attempt to
forward the mail.  It will wait a maximum of MSFSLEEP seconds
before it attempts to forward the message locally, and will wait
MSFSLEEP * MSFRSLEEP seconds before it attempts to forward to a
remote location.

If the message is to go to a remote location the mail server will
attempt to find a route to that location.  First it will look in
its IDP route table to see if there is an open route for that
network number.  If there is an open route, or if one is
available to be opened, it will send a packet with the message to
that router and the packet will be forwarded from there.

    Note:  If another mail server has mail to send to the remote
    network, or if a remote mail server has mail to forward to
    the local network, all of the mail will also be sent through
    the open route.

If the IDP route table contains no entry for that network
number, the mail server will send out a RIP broadcast (socket
0001, Ethernet address fffffffffff) asking about that network
number.  All routers will respond with their information on that
network number and route.  The mail server will update its IDP
table with this information and will send the message to the
router that responded with the lowest hop count.  If all routers
respond with a hop count of F (route unavailable), then the mail
server will wait a MAXIMUM of another cycle of MSFSLEEP *
MSFRSLEEP seconds before attempting to send again to that server.
If 3+Mail cannot forward messages through all of these cycles before
the MSDEAD timeout occurs, the message will be returned to the sender
as "Failed to deliver."

    Note:  If the destination address is 00000000 because the
    mail server's Ethernet address is unknown, the packet is
    broadcast to the remote network and forwarded to the first
    mail server that answers.

Because 3+Mail has only one forwarding process, it can only
forward to one mail server at a time.  A mail message that needs
to be forwarded to a number of different mail servers will be
forwarded to one server at a time from a single mail server.
When forwarding mail, a mail server starts at the top of the
FORWARD subdirectory and steps through it in sequence.


How 3+Route Works

Once a mail server finds an open route, it passes all of its mail
for that network number through the router to the other network.

Routers are responsible for maintaining the IDP routing table.
As well as responding to mail server RIP broadcasts, routers also
broadcast their IDP table once a minute and whenever there is an
update (opening or closing a route connection).

When the mail server broadcasts for a route it can receive three
possible responses:

   1-14 (1-E) - The route is currently one with this hop count.

   15 (F)     - The route is not currently open, but is available.

   16 (10)    - The route is unavailable.

If the response is 1-E it will use the open route.  If the
response is F and no other router responds with a lower count,
this route will be opened, dropping the hop count to 2.

After the connection is made and data has been transmitted, the
router has the responsibility to close the connection.  The
router uses two timeouts, the modem timeout and the route
timeout, to close the connection.  The modem timeout is the
amount of time the modem waits while the line is inactive before
it disconnects.  The route timeout is similar, at a higher level.
Using both timeouts allows flexibility in setting up routes.  Set
the MSFSLEEP * MSFRSLEEP time to less than the lower of the two
router timeouts so that the mail server will have a chance to
check for an open route before it times out.

The modem timeouts are on either side of the route connection.
The route timeout is on only the initiator side.  The lower
number of both routers will determine the timeout.    Note:
Setting the numbers too low can cause conversation termination
before all processing is done.  This is especially important when
talking to large networks that have a lot of local traffic and
long cable distances.


A Practical Application:  Sending Many Messages on an Internet

A multi-network customer with 31 routes wants to send mail
between networks.  The routes will be used only for mail.
Delivery time must be less than three hours.  An 84 KB file
transfer takes six minutes at 2400 baud.

Under perfect conditions, it would take one mail server and one
router with one modem 186 minutes to deliver all of the messages.

Increasing the number of modems will not decrease the send time,
because a mail server can only serve one mail forwarding process.
But the send time can be decreased by adding another mail server
and another modem.  The more mail servers and routes available,
the faster the send time.

However, many factors will increase the total send time, including
noisy phone lines, mail server processing time, communication
time between 3+Mail and 3+Route servers on Ethernet, down routes,
remote networks sending mail to corporate, other local mail
servers sending mail to remote nets, and waiting for a modem to
time out before it can make another connection.

To optimize 3+Mail and 3+Route for this application, follow these
steps:

1.  Put the equipment on its own net and do not allow it to be
    used for any other purpose.  This will keep network traffic
    to a minimum and prevent other mail servers from tying up the
    routes with other business.  If you want the network
    accessible from the main network in addition to the mail
    servers and routers, you will require a 3+Name server and a
    3+NetConnect server.

2.  Install a dedicated phone line and phone number on the
    network.  Have the remote sites define the network in their
    3+Name service with a new network number, and define a route
    to that net using the new phone number.  This would allow
    undelivered mail to return but would not allow other mail to
    be delivered to the company net through this network.

3.  Redesign how and when mail is sent.  Rather than attempting
    to send all mail at once, break it up so that the East coast
    receives it first, the central states next, and the Pacific
    last.  To do this, define the windows on each route so that
    it is open for, say, 3 AM and 6 AM local time.  When the mail
    servers try to send to Los Angeles, they would be ignored
    because no route is available; only New York mail would be
    delivered.  Mail for Los Angeles and New York would not be
    sent at the same time, but both sites would get mail before 7
    AM local time.  Decreasing the MSDEAD timeout would allow you
    to see what was not sent to New York without waiting for Los
    Angeles to time out.  For example:

    TIME PST        WEST              CENTRAL             EAST
                 (11 routes)        (10 routes)        (10 routes)

    3:00                                               Open window
    4:00                            Open window        Close window
    5:00        Open window         Close window
    6:00        Close window

    Assuming ten minutes per mail transfer to allow for temporary
    delays, it will take 100 minutes to deliver to ten routes.
    Three mail servers delivering simultaneously can finish in 34
    minutes.

4.  After the time of delivery for each route is determined and
    the route window is set appropriately, divide the routes
    among the mail servers so that each one is able to forward
    when the route window becomes open.

