File Transfer Protocols

06.621 Data Communications and Networks 2
Week 2 – File Transfer Protocols

Learning Outcomes

By the end of this session the student should be able to understand the concept of file transfer protocols. The file transfer protocols that the student should be able to describe include: XON/XOFF, X Modem, Kermit, and Z Modem.

Readings

Ch 15 p135ff
Questions 1 - 30 p160 - 161

Introduction
In week one we introduced the concept of serial communications and concentrated on the physical layer of the OSI model. This week we look at how data is transferred between computers and move up into the data link layer of the OSI model.

The data link layer is responsible for getting a message from one computer to another without errors. The data link layer also accepts streams of bits from the physical layer and organises them into coherent messages that is passes to the network layer.

Both the sender and receiver have to agree on a set of rules that govern how their data link layers will communicate with each other. A data link layer provides three functions:

controls when computers transmit (media access control)

Detects and corrects transmission errors (error control)

Identifies the start and end of a message (message delineation)

Media Access Control
Media access control (MAC) refers to the need to control when devices transmit. MAC becomes important when several computers share the same communications circuit to ensure no two devices attempt to transmit data at the same time. Most computer networks managed by a host mainframe computer use controlled access.

X-ON/X-OFF
X-ON/X-OFF is one of the oldest MAC protocols dating back to the days of the teletype. X-ON/X-OFF is used on half-duplex, point-to-point circuits, and is only used for the transfer of text only messages. The X-ON signal indicates a system ready to receive messages, the X-OFF signal indicates a system too busy to receive.

In operation the receiving computer sends an X-ON signal telling the sender to start sending, the sender then sends continuously until the transmission finishes or until the receiver sends an X-OFF signal. If the receiver sends an X-OFF the sender waits until another X-ON is sent before sending any more data.

Since an X-ON or X-OFF signal can easily be lost during transmission the X-ON/X-OFF protocol can lead to confusion, as a result it is rarely used today.

Polling
Polling is the process of sending a signal to a client that gives it permission to transmit or to ask it to receive. With polling, the client computers store all the messages that need to be transmitted. Periodically the host computer polls the client to see if it has data to send. If it has it sends data, if it does not it responds negatively and the host moves on to the next client.

There are several types of polling, the most commonly used forms are:

Roll call polling, the FEP works consecutively through a list of clients, first 1, then 2 etc. until all are polled.
Hub go-ahead polling (a.k.a. token passing) One computer starts the poll and passes it to the next system on the multipoint circuit, which sends its message or passes it to the next system, and so on.

Contention
Contention is the opposite of controlled access. Computers wait until the circuit is free and then transmit whenever they have data to send. Contention requires a technique to handle situations when two devices try to transmit at the same time (Collision). In general, contention approaches work better than controlled approaches for small networks that have low usage. In high volume networks, many devices want to transmit at the same time, and a well-controlled circuit prevents collisions.

Error Control
There are two types of errors associated with networks:

Human errors, and

Network errors

There are two categories of network errors:

Corrupted data (that have been changed)

Lost data

Network errors are a fact of life in data communications networks. Normally errors occur in bursts, and in a burst error, more than one data bit is changed by the error-causing condition. Errors are not uniformly distributed, regardless of error rate statistics, and dial-up lines are more prone to errors because they have less stable parameters.

There are many ways to prevent errors:

Shielding (adding insulation)
Moving cables away from noise sources
Changing multiplexing type (FDM® TDM)
Tuning transmission equipment and improving connection quality
Using amplifiers and repeaters
Leasing conditioned circuits

It is possible to develop data transmission methodologies that vibe very high error detection and correction performance.

The only way to do error detection and correction is to send extra data with each message. In general, the larger the amount of error detection data sent, the greater the ability to detect an error.

There are three common error detection methods:

Parity Checking, in which additional bit is added to each byte in the message. The value of the additional parity bit is based on the number of 1s in each byte transmitted. Parity checking has a 50% success rate.

Longitudinal redundancy checking (LRC). LRC adds an additional character, called the block check character to the end of a message. The value of the BCC is determined in the same manner as the parity bit, but by counting longitudinally through the message rather than by counting vertically through each character. LRC has a 98% success rate.

Polynomial checking. Like LRC polynomial checking adds a character or a series of characters to the end of a message based on some algorithm. There are two common forms of polynomial checking:

Checksum, where the additional character is calculated by adding the decimal value of each character in the message and dividing it by 225, and keeping the remainder. Checksum has a 95% success rate.
Cyclic Redundancy (CRC), the most popular error checking scheme. With CRC the entire message is treated as one long binary number, the result is divided by a fixed binary number, and the remainder is stored and transmitted. CRC has a 99.99% success rate with 16-bit CRC.

Error Correction via Retransmission
The simplest, most effective, least expensive, and most commonly used method for error correction is retransmission. A receiver that detects an error simply asks the sender to retransmit the message until it is received without error. (called Automatic Repeat reQuest (ARQ)).

With Stop and Wait ARQ the sender stops and waits for a response from the receiver after each message or data package. Responses are:

Acknowledgement (ACK)
Negative acknowledgement (NAK)

With Continuous ARQ the sender does not wait for acknowledgement before sending next message. If it receives an NAK, it retransmits the needed messages.

Forward error correction uses codes containing sufficient redundancy to prevent errors by detecting and correcting them at the receiving end without retransmission of the original message.

Hamming code
Hagelbarger code (corrects up to 6 consecutive bit errors)
Bose-Chaudhuri code

Asynchronous Transmission
Asynchronous Transmission is often referred to as start-stop transmission because the transmitting device can transmit a character whenever it is convenient, and the receiving device will accept that character. Each character is transmitted independently of all other characters. To accomplish transmission a start bit (0), and a stop bit (1) are added to each character. The recognition of the start and stop of each message is called synchronization.

asynch.gif (11859 bytes)

In general, microcomputer file transfer protocols are used on asynchronous point-to-point circuits, typically across telephone lines via a modem. The best known microcomputer file transfer protocols include:

XMODEM
- XMODEM-CRC (CRC-8)
- XMODEM-1K (CRC+1K blocks)
YMODEM(CRC-16)
ZMODEM (CRC-32)
KERMIT (CRC-24)

XMODEM
XMODEM is generally offered by most BBS systems and is a safe bet if you are not sure of what to use. Introduced in 1977 by Ward Christensen, this protocol is slow, but reliable. Two variants of Xmodem are used: Checksum and CRC. The CRC method is more reliable at detecting errors during transmission and should usually be selected if given a choice. Mismatching the two variants of XMODEM during file transfers is usually the reason for transfer problems. Xmodem-1K refers to the XMODEM CRC protocol method but with data blocks 1024 bytes in size. Many BBS systems can automatically detect XMODEM variations and may simply present you with a single Xmodem choice. Newer protocols are faster and more dependable, but XMODEM will usually do the job. Trial and error or simply logging onto the BBS system and then inspecting the menu of protocols available will give you more detailed information and clues about other possible protocols.

YMODEM
YMODEM offers a faster transmission rate than XMODEM, better data security through a refined CRC checksum method and finally a method for "batching" or sending/receiving several files one after another unattended. Choose Ymodem over Xmodem for increased speed and data security if both your software and the host computer can provide it and if telephone lines are relatively noisy. In reality, basic Ymodem (without the batch file feature) is CRC XMODEM with 1024 byte data packets. Or simply XMODEM-1K. Ymodem with batch file handling is sometimes also called "true Ymodem.

ZMODEM
ZMODEM is faster still, somewhat more secure with respect to data loss, offers batch transmission of files and additionally allows part of a file to be stored if transmission connections are lost and then completed when the connection is reestablished. If your transmission is interrupted and you have transferred only part of the file, you can call up the host computer and continue the file transfer from where you left off. Finally, ZMODEM can detect excessive line noise and automatically drop to a shorter, more reliable data packet size when necessary. Data integrity and accuracy is assured by the use of reliable 16 bit CRC (cyclic redundancy check) methods rather than less reliable CRC checking of YMODEM and XMODEM.

KERMIT
KERMIT is a very popular protocol (named after Kermit the frog). The KERMIT protocol was developed by Columbia University, which also developed the software package with the same name.

KERMIT is a very flexible protocol that can be adjusted to support a variety of different packet sizes and error detection methods. KERMIT typically uses a 100 byte packet size with CRC-24, but these are adjusted during transmission to optimise performance. It uses either stop-and-wait ARQ or continuous ARQ. It is specially suited to microcomputer-to-mainframe connections, but works equally well with mainframe-to-mainframe connections and microcomputer-to-microcomputer connections.