SMTP Mail Protocol
In this article we will introduce you to a protocol without which any mail services could not exist - Simple Mail Transfer Protocol (SMTP).
The term simple should not mislead you - you will see that SMTP is a rather complex software.
Everything described below is happening on a server level and our participation is limited to pressing the Send button on our e-mail program.
Transport of Electronic Mail
The purpose of the Simple Mail Transfer Protocol (SMTP) protocol is to reliably and efficiently transfer mail.
SMTP is independent of the particular transmission subsystem and requires a reliable data stream channel.
An important feature of SMTP is its ability to carry mail to multiple networks, commonly referred to as "SMTP mail relaying".
Using SMTP, one process can transfer mail to another process on the same network or to some other network via a relay or gateway process accessible to both networks.
In this way, the mail message can pass through several intermediate relays or gateway hosts on its way from the sender to the final recipient.
Domain Name System e-mail mechanisms are used to identify the appropriate destination of the next hop for a message being transported.
SMTP Model - Basic Structure
To effect a mail transfer to an SMTP server, an SMTP client establishes a two-way transmission channel to that SMTP server. An SMTP client determines the address of an appropriate host running an SMTP server by resolving a destination domain name to either an intermediate Mail exchanger host or a final target host.
An SMTP server may be either the ultimate destination or an intermediate relay (it may assume the role of an SMTP client after receiving the message) or gateway (it may transport the message further using some protocol other than SMTP).
SMTP commands are generated by the SMTP client and sent to the SMTP server.
SMTP replies are sent from the SMTP server to the SMTP client in response to the commands. In other words, message transfer can occur in a single connection between the original SMTP-sender and the final SMTP-recipient, or can occur in a series of hops through intermediary systems.
Once the transmission channel is established and initial handshaking is completed, the SMTP client normally initiates a mail transaction.
Such a transaction consists of a series of commands to specify the originator and destination of the mail and transmission of the message content itself.
When the same message is sent to multiple recipients, this protocol encourages the transmission of only one copy of the data for all recipients at the same destination (or intermediate relay) host.
The server responds to each command with a reply. Replies may indicate that the command was accepted, that additional commands are expected, or that a temporary or permanent error condition exists.
Once a given mail message has been transmitted, the client may either request that the connection be shut down or may initiate other mail transactions.
In addition, an SMTP client may use a connection to an SMTP server for ancillary services such as verification of email addresses or retrieval of mailing list subscriber addresses.
SMTP protocol provides mechanisms for the transmission of mail. This transmission normally occurs directly from the sending host to the receiving host when the two hosts are connected to the same transport service.
When they are not connected to the same transport service, transmission occurs via one or more relay SMTP servers.
An intermediate host that acts as either an SMTP relay or as a gateway into some other transmission environment is usually selected through the use of the domain name service (DNS) Mail exchanger mechanism.
Usually, intermediate hosts are determined via the DNS MX record, not by explicit source routing.
Mail Objects SMTP transports a mail object. A mail object contains an envelope and content.
The SMTP envelope is sent as a series of SMTP protocol units. It consists of an originator address (to which error reports should be directed), one or more recipient addresses, and optional protocol extension material.
The SMTP content is sent in the SMTP DATA protocol unit and has two parts: the header section and the body.
Senders and Receivers
The two hosts participating in an SMTP transaction are "SMTP client" (or sometimes just "the client") and "SMTP server" (or just "the server"), respectively.
Since a given host may act both as server and client in a relay situation, "receiver" and "sender" terminology is still used where needed for clarity.
Mail Agents and Message Stores
SMTP servers and clients provide a mail transport service and therefore act as "Mail Transfer Agents" (MTA).
"Mail User Agents" (MUA or UA) are normally thought of as the sources and targets of mail.
At the source, an MUA might collect mail to be transmitted from a user and hand it off to an MTA. The final ("delivery") MTA would be thought of as handing the mail off to an MUA by depositing the message in a "message store".
Buffer and State Table
SMTP sessions are stateful, with both parties carefully maintaining a common view of the current state. This state is modeled by a virtual "buffer" and a "state table" on the server that may be used by the client to "clear the buffer" or "reset the state table", causing the information in the buffer to be discarded and the state to be returned to some previous state.
Commands and Replies
SMTP commands and message data are transmitted from the sender to the receiver via the transmission channel in "lines".
The command consists of a key word followed by zero or more arguments.
- Five commands (HELO, MAIL FROM, RCPT TO, DATA and QUIT) are mandatory, and every implementation must support them.
- The other three commands RSET, VRFY and NOOP` are often used and highly recommended.
- The next six programs (TURN, EXPN, HELP, SEND FROM, SOML FROM and SAML FROM) are seldom used.
An SMTP reply is an acknowledgement (positive or negative) sent in "lines" from receiver to sender via the transmission channel in response to a command.
Responses are sent from the server to the client. A response is a three-digit code that may be followed by additional textual information. The meanings of the ﬁrst digit are as follows:
- 2bc - positive completion reply; the requested command has been successfully completed and a new command can be started.
- 3bc - positive intermediate reply; the requested command has been accepted, but the server needs some more information before completion can occur.
- 4ab - transient negative completion reply; the requested command has been rejected, but the error condition is temporary, and the command can be sent again.
- 5ab - permanent negative completion reply; the requested command has been rejected, and the command cannot be sent again.
The second (b) and the third (c) digits provide further details about the responses:
220 Domain service ready; ready to start TLS
354 Start mail input; end with <CRLF>.<CRLF>
452 Requested action not taken: insufﬁcient system storage
500 Command not recognized: command; Syntax error
Message Content and Mail Data
The terms "message content" and "mail data" are used interchangeably to describe the material transmitted after the DATA command is accepted and before the end of data indication is transmitted.
Message content includes the message header section and the possibly structured message body.
Originator, Delivery, Relay, and Gateway Systems
The four types of SMTP systems are distinguished on the basis of the role these systems play in transmitting emails.
- An originating system (sometimes called an SMTP originator) introduces mail into a transport service environment.
- A delivery SMTP system is one that receives mail from a transport service environment and passes it to a mail user agent or deposits it in a message store that a mail user agent is expected to subsequently access.
- A relay SMTP system receives mail from an SMTP client and transmits it, without modification to the message data other than adding trace information, to another SMTP server for further relaying or for delivery.
- A gateway SMTP system receives mail from a client system in one transport environment and transmits it to a server system in another transport environment.
Mailbox and Address
An address is a character string that identifies a user to whom mail will be sent or a location into which mail will be deposited. The term mailbox refers to that depository.
An address normally consists of user and domain specifications. The standard mailbox naming convention is defined to be:
General Syntax Principles and Transaction Model
SMTP commands and replies have a rigid syntax. All commands begin with a command verb. All replies begin with a three digit numeric code. In some commands and replies, arguments are required following the verb or reply code.
An SMTP session is initiated when a client opens a connection to a server and the server responds with an opening message.
Once the server has sent the greeting (welcoming) message and the client has received it, the client normally sends the EHLO command to the server, indicating the client’s identity.
There are three steps to SMTP mail transactions:
- The transaction starts with a MAIL command that gives the sender identification.
- A series of one or more RCPT commands follows, giving the receiver information.
- Then, a DATA command initiates transfer of the mail data and is terminated by the "end of mail" data indicator, which also confirms the transaction.
Forwarding for Address Correction or Updating
Servers may forward messages when they are aware of an address change. When they do so, they may either provide address-updating information with a 251 code, or may forward "silently" and return a 250 code.
Servers may reject messages or return them as non-deliverable when they cannot be delivered precisely as addressed. When they do so, they may either provide address-updating information with a 551 code, or may reject the message as undeliverable with a 550 code and no address-specific information.
Command Semantics and Syntax
The SMTP commands define the mail transfer or the mail system function requested by the user. SMTP commands are character strings terminated by <CRLF>. The commands themselves are alphabetic characters terminated by <SP> if parameters follow and <CRLF> otherwise.
Replies to SMTP commands serve to ensure the synchronization of requests and actions in the process of mail transfer and to guarantee that the SMTP client always knows the state of the SMTP server. Every command MUST generate exactly one reply.
IPv6 and MX Records
In the contemporary Internet, SMTP clients and servers may be hosted on IPv4 systems, IPv6 systems, or dual-stack systems that are compatible with either version of the Internet Protocol.
The host domains to which MX records point may, consequently, contain A RRs (IPv4), AAAA RRs (IPv6), or any combination of them.
Unwanted, Unsolicited, and "Attack" Messages
Utility and predictability of the Internet mail system requires that messages that can be delivered should be delivered, regardless of any syntax or other faults associated with those messages and regardless of their content.
If they cannot be delivered, and cannot be rejected by the SMTP server during the SMTP transaction, they should be bounced (returned with non-delivery notification messages).
Mail Security and Spoofing
The original SMTP design was unable to authenticate senders or verify that servers are authorized to send on their behalf, which leads to the ability to copy e-mail and is usually used in e-mail and phishing.
Proposals for major SMTP change and even full replacement are made periodically, but this is hardly possible due to the huge number of SMTP servers installed and currently operating.
Instead, mail servers now use a range of techniques, including DomainKeys (DK), DomainKeys Identified Mail (DKIM), Sender Policy Framework (SPF), DMARC, DNSBLs and greylisting to reject or quarantine suspicious emails.
Modern SMTP servers typically require authentication of customers through credentials before allowing access rather than limiting access to the location as described above.
This more flexible system is friendly to mobile users and allows them to have a fixed choice of configured outgoing SMTP server.
SMTP Authentication, often abbreviated SMTP AUTH, is an SMTP extension to log in using an authentication mechanism.
STARTTLS is an extension to the SMTP (Simple Mail Transfer Protocol) service that allows an SMTP server and client to use TLS (Transport Layer Security) to provide private, authenticated communication over the Internet.
This gives SMTP agents the ability to protect some or all of their communications from eavesdroppers and attackers.
The STARTTLS extension to SMTP is laid out as follows:
- the name of the SMTP service defined here is STARTTLS;
- the EHLO keyword value associated with the extension is STARTTLS;
- the STARTTLS keyword has no parameters;
- a new SMTP verb, "STARTTLS", is defined;
- no additional parameters are added to any SMTP command;
The STARTTLS Keyword The STARTTLS keyword is used to tell the SMTP client that the SMTP server is currently able to negotiate the use of TLS. It takes no parameters.
The STARTTLS Command The format for the STARTTLS command is: STARTTLS with no parameters.
Usage Example The following dialog illustrates how a client and server can start a TLS session:
S: <waits for connection on TCP port 25> C: <opens connection> S: 220 mail.imc.org SMTP service ready C: EHLO mail.example.com S: 250-mail.imc.org offers a warm hug of welcome S: 250-8BITMIME S: 250-STARTTLS S: 250 DSN C: STARTTLS S: 220 Go ahead C: <starts TLS negotiation> C & S: <negotiate a TLS session> C & S: <check result of negotiation> C: EHLO mail.example.com S: 250-mail.imc.org touches your hand gently for a moment S: 250-8BITMIME S: 250 DSN
SMTP is one of the most widely used and implemented application.
With the explosively growing reliance on electronic mail for commercial and personal services, there grows the demand of authentication and confidentiality.
To complement the weak security feature of SMTP industry use Pretty Good Privacy (PGP) or Secure/Multipurpose Internet Mail Extensions (S/MIME).
Still there is need of implementing the measures to eliminate spam and other security breaches.
Complete scenarios of several types of SMTP sessions.
In the examples,
indicates what is said by the SMTP client, and S:` indicates what is said by the SMTP server.
A Typical SMTP Transaction
S: 220 foo.com Simple Mail Transfer Service Ready C: EHLO bar.com S: 250-foo.com greets bar.com S: 250-8BITMIME S: 250-SIZE S: 250-DSN S: 250 HELP C: MAIL FROM:<Smith@bar.com> S: 250 OK C: RCPT TO:<Jones@foo.com> S: 250 OK C: RCPT TO:<Green@foo.com> S: 550 No such user here C: RCPT TO:<Brown@foo.com> S: 250 OK C: DATA S: 354 Start mail input; end with <CRLF>.<CRLF> C: Blah blah blah... C: ...etc. etc. etc. C: . S: 250 OK C: QUIT S: 221 foo.com Service closing transmission channel
Aborted SMTP Transaction
S: 220 foo.com Simple Mail Transfer Service Ready C: EHLO bar.com S: 250-foo.com greets bar.com S: 250-8BITMIME S: 250-SIZE S: 250-DSN S: 250 HELP C: MAIL FROM:<Smith@bar.com> S: 250 OK C: RCPT TO:<Jones@foo.com> S: 250 OK C: RCPT TO:<Green@foo.com> S: 550 No such user here C: RSET S: 250 OK C: QUIT S: 221 foo.com Service closing transmission channel
S: 220 foo.com Simple Mail Transfer Service Ready C: EHLO bar.com S: 250-foo.com greets bar.com S: 250-8BITMIME S: 250-SIZE S: 250-DSN S: 250 HELP C: MAIL FROM:<JQP@bar.com> S: 250 OK C: RCPT TO:<Jones@XYZ.COM> S: 250 OK C: DATA S: 354 Start mail input; end with <CRLF>.<CRLF> C: Date: Thu, 21 May 1998 05:33:29 -0700 C: From: John Q. Public <JQP@bar.com> C: Subject: The Next Meeting of the Board C: To: Jones@xyz.com C: C: Bill: C: The next meeting of the board of directors will be C: on Tuesday. C: John. C: . S: 250 OK C: QUIT S: 221 foo.com Service closing transmission channel
Relay Host to Destination Host
S: 220 xyz.com Simple Mail Transfer Service Ready C: EHLO foo.com S: 250 xyz.com is on the air C: MAIL FROM:<JQP@bar.com> S: 250 OK C: RCPT TO:<Jones@XYZ.COM> S: 250 OK C: DATA S: 354 Start mail input; end with <CRLF>.<CRLF> C: Received: from bar.com by foo.com ; Thu, 21 May 1998 C: 05:33:29 -0700 C: Date: Thu, 21 May 1998 05:33:22 -0700 C: From: John Q. Public <JQP@bar.com> C: Subject: The Next Meeting of the Board C: To: Jones@xyz.com C: C: Bill: C: The next meeting of the board of directors will be C: on Tuesday. C: John. C: . S: 250 OK C: QUIT S: 221 foo.com Service closing transmission channel
Verifying and Sending
S: 220 foo.com Simple Mail Transfer Service Ready C: EHLO bar.com S: 250-foo.com greets bar.com S: 250-8BITMIME S: 250-SIZE S: 250-DSN S: 250-VRFY S: 250 HELP C: VRFY Crispin S: 250 Mark Crispin <Admin.MRC@foo.com> C: MAIL FROM:<EAK@bar.com> S: 250 OK C: RCPT TO:<Admin.MRC@foo.com> S: 250 OK C: DATA S: 354 Start mail input; end with <CRLF>.<CRLF> C: Blah blah blah... C: ...etc. etc. etc. C: . S: 250 OK C: QUIT S: 221 foo.com Service closing transmission channel message)
Reference: RFC5321 – Simple Mail Transfer Protocol Reference: RFC3207 – SMTP Service Extension for Secure SMTP over Transport Layer Security
#email #server #overview