Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

The efficacy of a store-and-forward file transfer system Sadowski, Edward Richard 1984

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


831-UBC_1984_A6_7 S23.pdf [ 3.61MB ]
JSON: 831-1.0051851.json
JSON-LD: 831-1.0051851-ld.json
RDF/XML (Pretty): 831-1.0051851-rdf.xml
RDF/JSON: 831-1.0051851-rdf.json
Turtle: 831-1.0051851-turtle.txt
N-Triples: 831-1.0051851-rdf-ntriples.txt
Original Record: 831-1.0051851-source.json
Full Text

Full Text

C l T H E EFFICACY OF A STORE-AND-FORWARD FILE TRANSFER SYSTEM by EDWARD RICHARD SADOWSKI B.Sc. The University of British Columbia, 1 9 8 1 . A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in T H E F A C U L T Y OF GRADUATE STUDIES DEPARTMENT OF COMPUTER SCIENCE We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA April, 1 9 8 4 ©Edward R. Sadowski, 1 9 8 4 In presenting t h i s thesis i n p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree at the University of B r i t i s h Columbia, I agree that the Library s h a l l make i t f r e e l y a v a i l a b l e for reference and study. I further agree that permission f o r extensive copying of t h i s thesis fo r s c h o l a r l y purposes may be granted by the head of my department or by h i s or her representatives. I t i s understood that copying or pub l i c a t i o n of t h i s t h e s i s fo r f i n a n c i a l gain s h a l l not be allowed without my written permission. Department of Cft^^ X -p r -^C \ t?iA C^, The University of B r i t i s h Columbia 1956 Main Mall Vancouver, Canada V6T 1Y3 Date ^~Tyf'\ Abstract Most existing File Transfer Systems are connection-based, where the actual transfer of a file immediately follows the transfer's specification. Few systems allow Three-Party Transfers, where a person on one computer can request the transfer of a file residing on a second computer to a third computer. A Store-and-Forward File Transfer System separates the specification and transmis-sion phases of a file transfer. A Store-and-Forward File Transfer System has been designed and implemented using as its medium of transmission a Computer Based Mes-sage System based on the current CCITT proposal. This File Transfer System allows genuine Three-Party Transfers. Results indicate that such a system is both feasible and practical. ii Table of Contents Chapter 1 - Introduction . 1 Chapter 2 - A Survey . 4 2.1 Network Architecture 4 2.2 FTP Architectures 4 2.2.1 The Virtual Filestore 7 2.3 Actions 11 2.3.1 Checkpointing and Error-Recovery 16 2.4 Implementation Architecture 17 2.5 The CCITT/ISO Message Handling System Model 18 2.5.1 Overview 18 2.5.2 Physical Mapping 20 2.5.3 Naming and Addressing 20 2.5.4 Layers Representing the MHS Model 22 2.5.5 The Message Transfer Layer 23 2.8 Store-and-Forward File Transfer 25 Chapter 3 - A Store-and-Forward File Transfer System 26 3.1 Motivation 26 3.2 Design Issues 27 3.3 Design and Implementation Outline 27 3.3.1 EAN and The CCITT CBMS Model 27 3.3.2 Unique Design Features 28 Three Party Transfers 28 Multiple Recipients 29 Reduction in Code Size 29 Elimination of Some Concerns 29 3.4 General Design 30 3.4.1 Functional Organization 30 3.4.2 Data Flow 32 3.4.3 Control Flow 33 3.4.4 Peer Protocol 34 Chapter 4 - The Implementation 39 111 iv 4.1 The User Interface 39 4.2 Virtual Files 40 4.3 Transfer Identifiers 42 4.4 Building of Protocol Data Units 43 4.5 Use of the UA - SDE Interface of EAN in EANFTP 44 4.5.1 UAL Interface Primitives 46 4.6 A Walk-through a File Transfer 48 4.7 Keeping Track of Transfers in Progress .'. 52 4.8 Forwarding of Receipt Notifications 58 Chapter 5 - Results and Evaluation 57 5.1 Results 57 5.1.1 Two Party Transfers 57 Pulling a File 57 Pushing a File 59 5.1.2 Three Party Transfers 60 5.1.3 Multiple Recipients 61 5.1.4 When Things Go Wrong 62 5.1.5 Status of Transfers in Progress 64 5.2 Evaluations 64 Chapter 6 - Conclusions 68 6.1 Further Improvements 68 6.1.1 Management Functions 68 6.1.2 The Virtual Filestore 69 6.1.3 Automatically Initiated Jobs 70 6.1.4 Cancellation of In-progress Transfers 70 6.1.5 Bundling of Source Files 70 6.1.6 Encryption 70 6.1.7 PartiaJ Transfers 71 6.2 An Interesting Application 71 References 72 Appendix I - FTP User Manual 73 List of Figures 2.1 - ISO OSI Network Architecture Layers 5 2.2 - File Transfer Entitles 6 2.3 - Two Host Configurations 7 2.4 - Virtual Filestore Representation of a Hierarchical File 11 2.5 - Interaction of Service Primitives 13 2.8 - Sender Active File Transfer 14 2.7 - Receiver Active File Transfer 15 2.8 - Functional View of the MHS Model 19 2.9 - Combinations of Co-resident and Stand-alone UAs and MTAs 21 2.10 - Layered Description of the Message Handling System 23 3.1 - Functional Organization of EANFTP 31 3.2 - Data Flow In EANFTP 32 3.3 - Layered Description of the File Transfer System 34 3.4 - EANFTP F T A Protocol Data Units (CCITT X.409 syntax) 36 3.5 - Logical Structure of the EANFTP SendRequest PDU 37 4.1 - Syntax of FTAUEP request 41 4.2 - Transfer Identifiers In EANFTP 43 4.3 - Definition In C of ENODE - the PDU node 44 4.4 - Layered Configuration of EANFTP 45 4.5 - Portion of CCITT definition of EM - UAPDU 46 4.6 - UAL Primitives State Transition Diagram 49 4.7 - Relationship Between PFPDUs and Agents in EANFTP 53 4.8 - Logical Structure of the Transaction Data Base 55 5.1 - Connection Asymmetry 65 v CHAPTER 1 Introduction Lars Posrsena of Clusium By the Nine Gods he swore That the great house of Tarquin Should suffer wrong no more. By the Nine Gods he swore it, And named a trysting day, And bade his messengers ride forth East and west and south and north, To summon his array. Lays of Ancient Rome Thomas Babington, Lord Macaulay The ability to transfer data files from one computer system to another is one of the most obvious applications of computer networking. This ability is so desirous that it had virtually spurred the inception of computer networks. In the infancy of computer communication there were many incompatible networks and each network had its own file transfer protocol which in most cases was incompati-ble with the protocol of any other network. From the mid-1970s to present an effort has been undertaken to standardize both the networks and their application programs (such as file transfer) in an attempt to achieve universal accessibility or Open Systems Inter-connect (OSI). The International Standards Organization (ISO) has sponsored such a standardiza-tion. Its current proposed standard for file transfer stipulates that it should be connection-based. That is, even before a file transfer can be specified, the file transfer program must set up a network connection between the source and destination comput-ers, whereupon the transfer will take place immediately. The ISO standard makes no provision for a Store-and-Forward (SAF) file transfer. That is, a transfer that takes Chapter 1 1 Introduction Chapter 1 2 Introduction place some time after its specification. Potential advantages of SAF file transfer are: 1. The potential better use of network resources by, for instance, the possible bundling of requests before a network connection is established between the source and destination computers thereby minimizing the overhead of estab-lishing such connections. Savings could be realized by establishing the con-nections during favourable rate periods or during periods of less network congestion. 2. Its "off-line" nature makes it easier to address such concerns as synchronization and error-recovery. 3. Common elements of other SAF applications such as Computer Based Message Systems (CBMS) can be used in the design and implementation. ISO has been in the process of standardizing CBMSs and it would be desirable for a SAF file transfer program to take advantage of aspects of the design of the ISO CBMS. This thesis will deal with the design, implementation, and evaluation of a Store-and-Forward File Transfer Program. This thesis is organized in the following manner: Chapter 2 is a survey of concepts and previous work in File Transfer Sys-tems. It also deals with an overview of the Comit'e Conaultatif Internationale de Telegraphique et T'elephonique (CCITT) Computer Based Message Sys-tem model on which the SAF File Transfer System, implemented as part of this thesis, is based. Chapter 3 presents the objectives in designing and implementing a SAF File Transfer System (SAFFTS). It also presents the general design of EANFTP, Chapter 1 3 Introduction the implementation of a SAFFTS. Chapter 4 describes the implementation of EANFTP, paying close attention to the information passed between entities in the system. Chapter 5 gives some examples of some actual file transfers accomplished on EANFTP. The chapter also assesses EANFTP's success in achieving the goals stated in Chapter 3. Chapter 6 gives general conclusions of the above enterprise as well as present-ing areas of needed further research and work. Appendix A is the User Manual for EANFTP. CHAPTER 2 A Survey 2.1. Network Architecture Before the advent of networking, the most reliable way of transferring a file was to copy the file to magnetic tape or some other physical medium and physically transport it to the destination system. Even this most rudimentary service required the parties of a transfer to agree on matters of format. They had to agree, or at least be aware of, the density of the recording, the character set used, the blocking factor, etc. of the tape. One can imagine the number of issues that must be addressed in order to realize this transfer on a computer network. In part, to make this task (along with other network-ing applications) more tractable, the International Standards Organization (ISO) devised a layered architecture model. In this model, services and their related protocols common to all applications, characterized by a common function, are grouped together in a hierarchical manner. They are organized as in Figure 2.1. Readers unfamiliar with the above model should see [ISO-80]. Since transferring a file over a network is an application, one might intuitively guess that the majority of a File Transfer Protocol (FTP) would reside in the applica-tion layer. However there are aspects that are usually associated with the presentation and session layers. These aspects will be discussed later in this chapter. 2.2. FTP Architectures A seminal approach to handling network file transfers was through the medium of an emulated interactive terminal. This is known as the Virtual Terminal Approach Chapter 2 4 Survey Chapter 2 5 Survey application presentation session transport network data link physical Figure 2.1 - ISO OSI Network Architecture Layers (VTA) (see [TELE-73]). A universally understood model of a terminal function is chosen. The receiver would have the identical characteristics of physical terminal and the sender of the file would "login" and "enter" the file through this terminal. Draw-backs in this approach include: - the inability to modify a transfer already in progress - the need to "login" to the remote system and thus requiring operational knowledge of the remote computer system, accounts, and passwords - inability to specify partial transfers - inability to specify services particular to or singularly helpful for a file transfer The shortcomings of VTP prompted the design and implementation of a separate service expressly designed for file transfer. Chapter 2 6 Survey An FTP can be split into two parts: the Control Service and its corresponding Pro-tocol (CSP) and the Transfer Service and its corresponding Protocol (TSP). The CSP is responsible for the initiation, specification, and supervision of a particular file transfer. The TSP is responsible for the actual transfer of a file. In its most general configuration a file transfer involves at least three parties (or hosts): the controlling or initiating host -who specifies and coordinates the transfer; the sending host - the host where the source file resides; one or more receiving hosts - the host(s) where the file is to be sent. The relationships between the hosts and the protocols are shown in Figure 2.2. To make the coordination and control of the transfer easier it is assumed in many FTPs that the controlling host and one of the sending or receiving hosts are one and the same. With this configuration the host containing the CSP is known as the active host and the other known as the passive host. Configurations are shown in Figure 2.3. These configurations are restricted to either "pulling" a file (a file from a remote source is transferred to a local system) or "pushing" a file (a file from the local system is transferred to a remote system). Figure 2.2 - File Transfer Entities Chapter 2 7 Survey active host passive host passive host active host control control ® data or data Figure 2.3 - Two Host Configurations In most FTPs, the CSPs and TSPs utilize the same connection between the sending and receiving systems. The CSP initiates a connection between the two hosts and nego-tiates the file transfer. Once this is accomplished the TSP uses the same connection to transfer the file. The two phases can overlap: the CSP synchronizing and interrogating an in-progress transmission. This approach is known as a connection-oriented FTP. The ARPA.FTP [NEIG-73] utilizes two connections at the same time: one for the CSP and the other for the TSP. Alternatively, the CSP and the TSP can be treated as two distinct phases each requiring its own connection. This method does not require the actual transfer to take place immediately. Instead the request is queued for later delivery thus allowing the person who had specified the request to do other tasks. This approach is known as a transaction-oriented or Store-and-Forward File Transfer (SAFFT). 2.2.1. The Virtual Filestore The architecture and organization of files on various systems are legion. Byte-size, character sets, handedness, etc. all contribute to a veritable Babel of file types. Similar to a system wide definition of a virtual terminal protocol, so too do most FTPs stipulate a virtual terminal filestore. A filestore is a logical collection of physical files. A virtual filestore is a universally known and understandable abstraction of any particular Chapter 2 8 Survey physical filestore. The mapping of the actual file to its virtual file representation - a presentation layer task - is a local matter and is not a concern in the definition of the FTP. What is important is that once this mapping is done, it appears as a properly formed virtual file to the FTP. To describe a virtual file, a number of attributes are associated with it. In the ISO proposal [ISO-82], these attributes can be classified as either: file attributes - a set of which describes and identifies a particular virtual file or activity attributes - a set of which describes and identifies a particular file transfer This and any other concrete FTP example in this section, unless otherwise stated, are from the ISO FTP model. The file attributes include: Filename: This is essential to any FTP. The "filename attribute is usually the physical filename prefaced with the host's Service Access Point (SAP) address. A SAP is a universally known or univer-sally obtainable address. It is unique to the domain of the partic-ular network. All requests for a particular service are channeled through its SAP. The ISO FTP uses a session layer SAP. The combination of the SAP and the local filename is enough to uniquely identify a file throughout a network. The FTPs do not specify any syntax for mapping a local filename into its virtual filename attribute other than stipulating that it be prefaces by its Chapter 2 9 Survey SAP. Therefore it is up to the parties involved in a particular transfer to agree on its representation. The filename attribute may or may not reflect the same structure as its local operating system filename syntax. This is an area of further research. Access Control Lists & Passwords: One of the disadvantages of the virtual terminal approach (at least in some of its manifestations) is that it requires one of the transfer parties to "log-in" to the other's computer. This necessi-tates the knowledge of passwords. By keeping an explicit access list, the FTP can ascertain, without the need of log-ins, whether the particular parties are allowed their desired transfer. For secu-rity purposes, the access lists could contain passwords associated with the virtual file rather than with an account. Account: Indicates against whom the filestore charges should be charged. This could include the cost of transferring this file anywhere. Date and Time of Creation, Last Modification, Last Access, Identity of Creator, Last Modifier, Last Accesser: Various biographical data of the file. File Availability: Indicates whether any delay could be expected before the file can be opened. This attribute can be used to avoid deadlock - the attribute could be interrogated and if a delay is indicated the transfer could be cancelled. A delay could also warrant the use of a SAFFTP. Chapter 2 10 Survey Device Type: The frequency of checkpoints (whose function will be described below) could be influenced by the volatility of the device of the virtual file. For example, a file to be transferred residing on disk would require fewer checkpoints than a file to be read in from a card reader as the disk is less prone to malfunction. Data Types: A file can contain mixed data: some character, some binary, different character sets. In the Autodin II [BBN-80] and ECMA [ECMA-80] FTPs, mixed data types are not allowed, limiting the utility of those FTPs. Encryption Algorithm Name: The algorithm used to encrypt the file. Current and Future File Sizes: If the file is very large it might be incapable of being transferred as a unit. Before the file could be transferred it would have to be disassembled into components small enough to be transferred. These sub-files would then be transmitted and collected at the destination, where the components would be reassembled. Structure Type: Files may be organized in many ways; they could be "unstruc-tured", "flat", "hierarchical", or some other structure. This diversity must be represented in some standard way. The ISO FTP provides a syntax for describing hierarchical files (such as Indexed Sequential Access Method (ISAM) files). The file is Chapter 2 11 Survey represented as an oriented tree. A node of such a tree can be given an unique (to the file) identifier. The subtree rooted by this node can be uniquely identified and accessed. The root of the entire file will, of course, be labelled with the virtual filename attribute. The structure is shown in Figure 2.4. No FTP has yet been designed that has a similar abstraction for multi-keyed files or files of non-hierarchical structure. Activity attributes include: T y p e of access: "Read", "Write", "Management". Management access allows the filestore attributes to be changed. 2.3. Actions By means of a virtual filestore an FTP should be able to perform a number of operations on a file such as: select (to locate, interrogate, but not open), open or close, read, write, append, update, create, and destroy. root node (entire file) block node block node record node record node ... record node Figure 2.4 - Virtual Filestore Representation of a Hierarchical File Chapter 2 12 Survey As part of the selection and specifications of a file transfer, the FTP should have the ability to read, negotiate, and modify file and activity attributes. This is known as management access. ISO has not yet designed a mechanism for negotiating the parame-ters of an attribute. The parties can either agree or disagree with the proposed change of an attribute. NIFTP [NPL-81] however has a more elaborate negotiating scheme. In this scheme, each party has a capability and a requirement for each attribute. A capa-bility is the upper bound, the requirement the lower bound. Updating - the ability to replace a part of a file - can be relatively easily realized through ISO's (or similar) hierarchical tree representation outlined above. A connection-based file transfer follows this characteristic sequence: 1) application connection: the source and destination FTPs are connected via their respective SAPs. 2) file selection: the particular files are specified and located. 3) file opening: 4) data transfer: actual transfer of file data. 5) file closing: the connection to the file is still kept but any read/write locks are released. 6) file deselect: lose knowledge of the file. 7) application disconnection: the connection is closed. Between steps 1 and 2 the FTP can give management information such as file directories of a particular filestore. Between steps 2 and 3 the FTP can access the attri-butes of a particular file. Any interrogation of a file transfer in progress takes place dur-ing step 4. Chapter 2 13 Survey The two parties privy to transfer communicate through the intermediary of the File Transfer Service Agent (FTSA). The FTSA is charged with the routing, establish-ment, and forwarding of communication of the transfer parties. Most such communica-tion between FTSAs follow the pattern shown in Figure 2.5. The actual transfer of file data is itself communication - a typical transfer might follow that shown in Figures 2.6 and 2.7. Notice how, in Figures 2.6 and 2.7, the actual file data communication departs from the usual Request - Indication - Response -Confirmation (RIRC) dialogue shown in Figure 2.5. Only requests and indications are used. This is an efficiency concession. Notice the slight asymmetry (*) in the flow between when the sender or the receiver is the active component (Figure 2.6 vs. Figure 2.7). The file data might have to be "massaged" before transmission. That is, the file might have to be encrypted or compressed. These desirable facilities would normally be placed in the presentation layer. The ISO layer model maps out a logical demarcation of functions. The functions, services, and applications particular to a layer do not neces-sarily have to physically reflect this characterization. In an FTP, for example, the code for the desirable presentation services could well be with the code for the application ser-Figure 2.5 - Interaction of Service Primitives Chapter 2 14 Survey W i t h t h e s e n d e r a s t h e a c t i v e c o m p o n e n t : s e n d e r ( a c t i v e ) c o n n e c t r e q u e s t -c o n n e c t c o n f i r m s e l e c t r e q . — . o p e n c o n f . < < -w r i t e r e q . . w r i t e c o n f . < £ -d a t a r e q -d a t a r e q . d a t a r e q . e n d - f i l e r e q . e n d - f i l e c o n f . c l o s e r e q . c l o s e c o n f . d e s e l e c t r e q . d e s e l e c t c o n f . ^ — d i s c o n n e c t r e q . . d i s c o n n e c t c o n f . F T S s e l e c t c o n f . < ^ • o p e n r e q . ; j s , 4r r e c e i v e r ( p a s s i v e ) - ^ - c o n n e c t i n d i c a t i o n c o n n e c t r e s p o n s e — 7 - s e l e c t i n d . ^ - s e l e c t r e s p . o p e n i n d . o p e n r e s p . ^ w r i t e i n d . / w r i t e r e s p . — > d a t a i n d . r > d a t a i n d . -9> d a t a i n d . - > e n d - f i l e i n d . / — e n d - f i l e r e s p . c l o s e i n d . — c l o s e r e s p . d e s e l e c t i n d . d e s e l e c t r e s p . ( t i m e ) " ^ d i s c o n n e c t i n d . d i s c o n n e c t r e s p . F i g u r e 2.6 - S e n d e r A c t i v e F i l e T r a n s f e r Chapter 2 15 Survey With the receiver as the active component: receiver (active) connect request FTS connect confirm <^— select req. select conf. open req. open conf. read req. read conf.. <~~ data ind. ^  data ind. <£.—-data ind.*^ end-file ind.^~ end-file resp end-read ind.-close req. - i s , close conf.^— deselect req. _ deselect conf disconnect req. ^ disconnect conf.^-sender (passive) —^ > connect indication connect response select ind. • select resp. ^> open ind. open resp. read ind. — read resp. - data req. . data req. data req. - end-file req. V (time) -—> end-file conf. — end-read req. (*) ^ close ind. <C — close resp. deselect ind. deselect resp. disconnect ind. disconnect resp. Figure 2.7 - Receiver Active File Transfer vices. An explicit boundary and protocol need not be necessary. As such, ISO has stan-dard services and protocols (existing and proposed) for each layer except the application. What should be recognized is that some of these protocols may not be suitable for the unique needs of a particular application. The application must then enhance or replace the unsuitable service and its corresponding protocol. Chapter 2 16 Survey 2.3.1. Checkpointing and Error-Recovery The world is not perfect and especially so networks. If a connection were broken in the middle of a transfer it would be desirable, once the connection was reestablished, to continue where the transfer had left off. In the examples of typical file transfers, it was noted that the data communication was not explicitly acknowledged (no response and confirmation). Most FTPs have a checkpointing scheme to accommodate both efficiency and restartability. At "appropriate" sections of the file, a "unique" checkpoint is sent. The sender expects the checkpoint to be acknowledged by a regular RIRC dialogue. An acknowledged checkpoint indicates to the sender that all file data sent previous to the sending of the checkpoint have been received and stored properly. That is, the receiver has now taken over "responsibility" of the data. This scheme assumes that the tran-sport layer service can deliver the data undamaged and in order. "Appropriate" places are determined by the sender. They may include logical demarcations of the file such as page boundaries, or they may reflect the volatility of the devices (As ascertained by the virtual file device attribute (q.v.)) The more volatile a device is the more frequent the checkpointing should be. The sender keeps a "window" of unacknowledged checkpoints and will suspend transfer if this window is exceeded until some of the outstanding checkpoints are acknowledged. Upon reestablishment of a previously lost connection, the sender can start sending from the last acknowledged checkpoint. Care must be taken in chosing the size of the window. Large windows pose buffering problems; small windows cause needless delays. As with checkpointing schemes in lower layers, the FTP must insure that the checkpoint is unique in the window. It would be desirable (and, depending on the sever-ity of the disconnection, absolutely necessary) to have some way of uniquely identifying Chapter 2 17 Survey a particular file transfer. If a connection were lost, this could aid in the reestablishment of the transfer. NIFTP advocates combining the virtual filename, the SAP of the other host involved in the transfer, and a unique identifier of sufficiently long period to avoid duplicates. The possible problems with this approach are similar to the identification of a unique transport connection. 2.4. Implementation Architecture In most FTPs there is some differentiation between kernel, base, and extended ser-vices. The kernel services are absolutely necessary in every implementation of that FTP - services without which it would be impossible to transfer any files. The base services are what are stipulated by the signatories of an FTP to be mandatory features of every implementation. The extensions are optional features. FTP designers wish to minimize the spread of extensions in an FTP as extensions tend to encourage the formation of "cliques" - groups of users with whom no one else can communicate. On the other hand, the base services cannot be too complex as to make it nearly impossible to implement on smaller machines or to be prohibitively expensive. There is a question of which layers should handle transfer errors. If the FTP makes use of a standard session layer protocol should the session layer handle the errors itself? That is, the FTP assumes an error-free session layer service. In this case would the FTP need to concern itself with checkpointing? What are the efficiency tradeoffs? How can one handle concurrent access to a file? No FTP, as yet, handles this with anything more sophisticated then the ISO FTP's virtual file availability attribute. Access is exclusive. How would a partial ordering of locks and access be designed into the FTP? Chapter 2 18 Survey 2.5. The CCITT/ISO Message Handling System Model 2.5.1. Overview All the FTPs described above are connection-oriented. The CCITT is in the pro-cess of specifying store-and-forward Computer Based Message Systems (CBMS). [CCIT400-83]. Although the specification does not deal with file transfer per se, many of the specified services and protocols can be and, as will be shown in this thesis, have been used in a store-and-forward File Transfer System. The CCITT/ISO Message Handling System Model (MHSM) is organized as in Fig-ure 2.8. A person wishing to communicate with another person in the Message Handling Environment does so through his User Agent (UA). The UA is responsible for the proper formulation or reception of the message. The Message Transfer Agent (MTA), receives a properly formulated message from a UA and is responsible for the actual rout-ing and transmission of the message to the recipient's UA. The recipient's UA accepts the message from its MTA and makes the message available to the recipient. The collec-tive domain of the MTAs is known as the Message Transfer System (MTS). It is through the medium of the MTS that the UAs communicate with one another. The col-lection of UAs and MTAs is called the Message Handling System (MHS). A message consists of an envelope and its content. The envelope contains, among other things, the addresses of the recipients. Normally, the envelope is the only informa-tion the MTAs are allowed to look at. The content is the actual information the sender wishes to convey. Its architecture and composition are transparent to the MTAs. It is this transparency that can be utilized in a File Transfer System. Chapter 2 19 Survey Message Handling Environment User Denotes an interaction Figure 2.8 - Functional View of the MHS Model [CCIT400-83] Once an MTA has accepted responsibility1 of a message it will try to deliver the message to the appropriate MTA of the recipient. This might involve the relaying of the message by other MTAs in the MTS. Routing, the determination of which path through the MTS that a message will take, is the responsibility solely of the MTAs. lThis interaction is specified in (CCIT41l| Chapter 2 20 Survey 2.5.2. Physical Mapping The UA and its MTA need not reside on the same machine. That is, a "stand-alone" UA can communicate with its MTA that resides on a physically separate com-puter. This is particularly effective in a Local Area Network (LAN) environment where one component of the network could be the MTA for the whole LAN. However nothing precludes a UA and its MTA from residing on the same system. Some possible configurations are shown in Figure 2.9. 2.5.3. Naming and Addressing Within the MHS model there must be some mechanism by which one can uniquely identify the entities of the system. In particular, one must be able to specify uniquely the recipient of a message. A name is that which identifies an entity. An address is where its associated entity is located in the MHS. There are two types of names: primitive - unique to a some sub-domain of the MHS but not necessarily unique to the whole MHS. descriptive - which is unique to the whole MHS. Users of the MHS are not part of the definition of the MHS and as such are identified through their associated UAs. Users wishing to communicate send their mes-sages to the UA of their intended recipient. The descriptive name of the user's UA is known as its Originator/Recipient (0/R) Name. An 0/R Address is a descriptive name for a UA that in addition has certain information that helps the MTS in locating that UA. Hence every O/R address is an O/R name but not vice versa. In general, O/R Chapter 2 21 Survey I/O Device Processing System Figure 5/X.400. Co-resident UA and MTA O A A UA 1 MTA Intelligent Terminal Processing System liSiuATijiaL i;—u MTA • - -3 8 Processing System I/O Device 0 /K A Figure S/X.400-. Co-resident and Stand-alone UA and MTA A uo . Devices Processing System Processing System 'Intelligent" Terminal o . A A Figure 2.9 - Combinations of Co-resident and Stand-alone UAs and MTAs [CCIT400-83] names are more acceptable to the users of the MHS, whereas O/R addresses are more Chapter 2 22 Survey easily handled by the entities of the MHS. The 0/R name carries no explicit indication of routing. Routing is the responsibil-ity of the MTS. Upon receipt of the message from a UA, the MTA determines only the next MTA in the ultimate path of MTAs through which the message will travel in the MTS before being delivered to the destination MTA. It should be noted that the CCITT CBMS draft proposals use the terms O/R name and O/R address inconsistently. In most cases, especially in the definitions of the proto-cols, O/R name is used where, according to its own definitions, O/R address should be. 2.5.4. Layers Representing the MHS Model Described below is the physical implementation of the functional model described above. The layering strives to be consistent with the OSI Reference Model [OSI-80]. The functions in the Message Handling System Model are divided into two major layers: 1. The User Agent Layer (UAL). 2. The Message Transfer Layer (MTL). These layers can be configured as in Figure 2.10. In this diagram: 51 is a system with only UA functions. 52 is a system with only MTA functions. 53 is a system with both UA and MTA functions. Associated with each function is its functional entity, where, in this case, the UAE and MTAE are the entities for the UA and MTA respectively. A functional entity can be thought of as the physical implementation of the functions specified of its correspond-Chapter 2 23 Survey MHS Layers within the / Application^, Layer of the OSI Reference Model UAL UAE MTL MTAE | S3 PT MTAE -4 • > 11 UAE II SDE 1*: 7**rRys:-e; * S2 SI Figure 2.10 - Layered Description of the Message Handling System [CCIT400-83] ing model. The Submission and Delivery Entity (SDE) is the means by which a UAE in a Sl-type system can access the services of the MTL. The SDE acts as the interface t between the UAE and its corresponding MTAE. Px are the various peer protocols. PI is the MTAE-MTAE protocol. P3 is the SDE-MTAE protocol. Pc is the UAE-UAE protocol and will reflect the functionality of the UA. 2.5.5. The Message Transfer Layer The MTL provides the UAs the necessary services by which they can send mes-sages. A criterion in the design of the MTL is that it be both general and functional enough to support a variety of customized UAs. The definition allows UAs that might not even be concerned with the handling of personal mail messages per se, as will be Chapter 2 24 Survey shown in this thesis. The services the MTL provide are characterized by a number of primitives. These primitives can be grouped into the following categories: 1. Those primitives dealing with the initiation and termination of a dialogue between a UA and the MTL. 2. Those primitives dealing with the parameters maintained by the MTL that deal with that particular UA. 3. Those primitives dealing with the temporary specification of the nature of the types of messages that the UA will accept from the MTL. 4. Those primitives dealing with the submission of messages from a UA to its MTA. 5. Those primitives dealing with the determination of the delivery of a message. 6. Those primitives dealing with the actual delivery of the message from the recipient's MTA to his UA. <» 7. Those primitives dealing with the notification of the inability to deliver a mes-sage. 8. Those primitives dealing with the explicitly requested notification of delivery of messages. 9. Those primitives dealing with the cancellation of messages that had been queued for deferred delivery. It is upon these set of service primitives that a UA maps its functionality. Chapter 2 . 2 5 Survey 2.6. Store-and-Forward File Transfer As mentioned above, it may not be necessary to have the transfer of a file take place immediately. In a Store-and-Forward File Transfer Service (SAFFTS), the initia-tor of a transfer need only specify the transfer, leaving him free to do other tasks since the actual transmission of the file would take place on another connection at a (though not necessarily) different time. This is analogous to a CBMS2. Would it be possible to use a CBMS to realize an SAFFTS? In this case the "message" would be a file (of potentially enormous size). This thesis will address the design, implementation, and cri-ticism of an SAFFTS. Realizing the similarity between a CBMS and an SAFFTS, we will try to determine the common elements of both applications. ^ o r an example of an implementation of a C C I T T CBMS see [NEUF-83]). CHAPTER 3 A Store-and-Forward File Transfer System As was mentioned in Chapter 2, there has been little work done in the area of Store-and-Forward File Transfer Systems (SAFFTS). This thesis will attempt to design and implement such an FTP (herein to be referred to as EANFTP) and will address the efficacy of the undertaking. 3.1. Motivation Why have a Store-and-Forward File Transfer System? One might suppose, at first glance, that it is a step backwards. After all, since a SAFFTS is not connection-oriented, transfers do not take place in "real time". Historically, most applications have striven to provide real-time service. However a SAFFTS is especially suitable for the fol-lowing: specifying file transfers from machines with no instantaneous access to the specified files the distribution of files to multiple destinations the transmission of very large files over slow, unreliable media - thus FTP is useful for those people who do not want a terminal monopolized by such a transmission third party transfers (where both the source and destination files are on remote computer systems) transfer of files where transmission costs are a major concern broadcasting - e.g. the distribution of new releases of a document or a pro-gram to several subscribing destinations. Chapter 3 26 A SAFFTS Chapter 3 27 A SAFFTS 3.2. Design Issues A designer and implementor of an SAFFTS must address some major issues: 1. The functionality of the system: What capabilities will the SAFFTS have? Will they differ substantially from those of a connection-oriented FTS (COFTSf. By its very nature, an SAFFTS will not necessarily be able to access immedi-ately a remote computer system. This must influence the capability of the FTS. In particular how will this inability affect the management requests such as remote directory listings commonly allowed in a COFTS? 2. The underlying medium of transmission: Will the SAFFTS have its own session and presentation layers? Alternatively, the session, presentation, and even some of the application layers of other networking applications might be util-ized. 3. Application overlap: If the underlying medium of transmission is going to be from part of another networking application then from which applications should it be? How can the SAFFTS enhance its functionality by utilizing the capabilities of the applications from which it is derived. 3.3. Design and Implementation Outline 3.3.1. E A N and The C C I T T CBMS Model This thesis will describe the approach we have chosen. EANFTP will use as its medium of transmission a Computer Baaed Message System (CBMS). In particular, EANFTP will be using the EAN CBMS [NEUF-83]. EAN is an implementation, developed at the University of British Columbia, of the CCITT CBMS model At this Chapter 3 28 A S A F F T S stage it is capable of operating on machines supporting Berkeley UNIX1 version 4.2 as well as on DEC VAXs under the VMS operating system. Because the desirability of Computer Based Messaging is manifest, most computer systems already have or will have in the near future a CBMS. Since EAN adheres to the CCITT CBMS model (in the model's current incarnation) it should be compatible with systems supporting the ISO-OSI networking model. Ideally the support for the ISO model will/in the future, become almost universal. It is to be hoped then, that CCITT CBMSs will become pervasive. Those systems that do support a CCITT CBMS will then be capable of implementing EANFTP. This makes the choice of a CCITT CBMS as a transmission medium both sound and attractive. EANFTP will make the most use out of the Message Transfer Layer (MTL) of EAN. (Refer to the description of the CCITT model in Chapter 2). The MTL encom-passes the session and presentation layers of the ISO-OSI model. 3.3.2. Unique Design Features How will the utilization of the MTL of a CBMS influence the design of the FTP? The following are some features made possible and easier. Three Party Transfers As outlined in Chapter 2 (Figure 2.2), in its most general form, a file transfer involves three parties: the controlling or initiating host the sending or source host the receiving or destination host(s) 1 UNIX is a Trademark of Bell Laboratories. Chapter 3 29 A SAFFTS It was also mentioned that the synchronization of these three parties in an FTP presented a non-trivial task. So much so that many FTPs require a Two Host Configuration (See Figure 2.3). This thesis will investigate the ability of EANFTP to synchronize a generalized third party transfer. The problem of synchronization in a COFTS is in maintaining three concurrent network connections. In an SAFFTS, not being connection-oriented, these connections do not have to be concurrent, thus eliminating much of the synchronization problem. Multiple Recipients One of the requirements of a CBMS is that it be able to deliver a message to a list of recipients. It will be shown in this thesis how this requirement will be utilized by EANFTP. The ability to specify multiple recipients in a COFTS is either rare or unk-nown. Any synchronization problems posed by a Three Party Transfer is only com-pounded by multiple recipients. The MTS will automatically "bundle" some messages. That is, if a source file is destined for two different FTAs that share the same MTA then the MTS sends only one copy of the source file to that destination MTA. Reduction in Code Size The MTAE need not be co-resident with its associated UAEs (as shown in Figure 2.9). It is possible to configure EANFTP correspondingly, resulting in programs of smaller size which might be important for small machines. Elimination of Some Concerns By taking advantage of the MTL, EANFTP will not have to concern itself with issues such as: 1. Checkpointing: as this is handled in the session layer of the MTL Chapter 3 30 A SAFFTS 2. Accounting: files will essentially be sent as messages and therefore the charges that should accrue can be handled by the accounting system (if any) of the MTL 3. Managing its own address space: EANFTP will use the name space of the CBMS on which it is based. The MTA notifies its UAs of any invalid addresses 4. Sequencing of requests: the MTA delivers messages to the UA one at time and therefore acts as a sequencer for the UA 5. Connection failures: the MTL is responsible for the reestablishment of lost con-nections and for the retransmission of effected messages. 6. Priorities of Service: High, Low, and Normal priorities are supported and managed by the MTL. 3.4. General Design 3.4.1. Functional Organization EANFTP is organized as shown in Figure 3.1. The CCITT CMBS Model has two major entities: the Message Transfer Agent Entity (MTAE) and the User Agent Entity (UAE). EANFTP will make use of its own specialized UAEs that deal almost exclusively with files. To these UAEs, "messages" will be properly couched files. There are two types of UAEs: the entity corresponding to the File Transfer User Agent (FTUA) which is the agent with whom the user interacts. That is, the FTUA Entity (FTUAE) is the entity responsible for the proper formulation of a file transfer request which is then relayed via its MTAE to the File Transfer Agent Entity (FTAE). The FTA is responsible for the supervision of the file's transmission via the MTS to the Chapter 3 31 A S A F F T S Figure 3.1 - Functional Organization of EANFTP file's destination(s). This supervision entails the sending and receiving of the file as well as the handling of delivery confirmation which is received either from other FTAs or, as in the case of an invalid host name, from the FTA's associated MTA. Similar to the configurations shown in Figure 2.9, EANFTP's file transfer agents can either be co-resident with its associated MTA or stand-alone (where an MTA can. be associated with file transfer agents from other systems). Chapter 3 32 A SAFFTS 3.4.2. Data Flow A transfer of a file in EANFTP follows the steps as shown in Figure 3.2. 1. A user, on some initiating host machine, specifies, through an FTUA, that a file be sent from some particular source host machine to some other host machine. (The user is allowed to specify more than one recipient machine for the file). Figure 3.2 - Data Flow in EANFTP Chapter 3 33 A S A F F T S 2. This request, once properly formulated by the FTUA, is sent via the associated MTA and the MTS to the FTA at the source host machine. 3. The FTA sends a copy of the file via FTA's associated MTA and the MTS to the FTA of the recipient host machine. 4. This FTA deposits the file in the requested location and then sends a reply via the MTS to the FTA of the source host. 5. The FTA of the source host relays this acknowledgement back to the FTA of the initiating host machine. 6. If the user had requested delivery notification then the FTA of the initiating machine would send a computer mail message, through the FTA's associated MTA and the local UA, to the user. 3.4.3. Control Flow There are explicit boundaries of "responsibility" in the FTS made necessary by its Store-and-Forward nature. As the file goes from one layer of the system to the next and from one machine to the next, responsibility for the message (in this case a file) has to be taken by the entity who possesses it. An entity is "responsible" for a message when it takes "possession" of the message and will ensure that it will get delivered to its destina-tion (be it another entity or another computer) and will retransmit the message if neces-sary and possible. Any secondary storage needed to store copies of the message is super-vised exclusively by the responsible entity. In EANFTP, control flow can be characterized by the following: A user issues a request for a file transfer through his FTUA. Once this request is properly formulated (this corresponds to step 1 in the Data Flow example above) the user is free to do other tasks. He is no longer responsible for the request. During step 2, responsibility is taken Chapter 3 34 A SAFFTS first by the MTA of the FTUA and then by the FTA on the source host. Similarly in step 3, responsibility passes from FTA of the source host to the MTA of the source host to the FTA of the destination host. The path of responsibility for the acknowledgement (steps 4 and 5) is the converse of steps 2 and 3. 3.4.4. Peer Protocol When properly configured, the peer protocol used by the UAs (Protocol Pc as shown in Figure 2.10) should be totally transparent to the MTL. Even though the MTL was ostensibly designed for personal mail, the transparency of the Pc protocol makes it possible for other applications, with their own peer protocols, to be placed on top of the MTL. So instead of the configuration shown in Figure 2.10, EANFTP will be configured as in Figure 3.3. "PF" is the FTA - FTA peer protocol. UAL MTL S3 S2 SI Figure 3.3 - Layered Description of. the File Transfer System Chapter 3 35 A SA F F T S The FTAs communicate in -what CCITT describes as Protocol Data Units (PDU). The CCITT has gone into considerable detail in specifying the complexion of these PDUs [CCIT409-83] q.v. EANFTP's PDUs are shown in Figure 3.4. A TransferRequest corresponds to what is sent as described in step 2 of the Data Flow description above. A SendRequest is what is sent in step 3. TransferReply and SendReply are the PDUs sent in steps 4 and 5 respectively. A FTPDU has structure. For example, a SendRequest contains, among other things, a SendHead and the File. The SendHead itself has structure. It has a sequence of AgentRecipient which in turn has structure. A PDU then, has a tree structure. A field corresponds to a leaf node of the PDU tree. The structure of an EANFTP SendRe-quest is shown in Figure 3.5 Transmission lines, can only transfer a stream of bytes (or octets). Before the PDU can be sent it has to be "flattened". When this "flattened" PDU is received it can then be built-up again. Each node of the PDU tree stores the number of octets needed to represent the node itself and its subtree when both it and its subtree are "flattened". These numbers are used by the receiving FTAs to rebuild the previously "flattened" trees. The CCITT in its PDU syntax specification stipulated that fields of a PDU are identified by code numbers rather than by included strings such as "TO" or "FROM" which are found in other protocols. This encoding scheme is suitable for applications which might be implemented on machines of countries with various different native languages. Each node of the PDU tree also stores the code number identifying what type of node it is. Each field is identified by a code number. Most of these code numbers are unique only to the subtree data unit. That is, the code numbers are con-text sensitive. This corresponds to the adjective IMPLICIT seen in Figure 3.4. Their Chapter 3 36 A SAFFTS MODULE FTAPDU ::= CHOICE { [0] IMPLICIT TransferRequest, [lj IMPLICIT SendRequest, TransferReply [2] IMPLICIT ReceiptNotiflcation, SendReply [3] IMPLICIT ReceiptNotiflcation } TransferRequest : — SET { [0] IMPLICIT SendHead, pageRanges [l] IMPLICIT SEQUENCE OF INTEGER OPTIONAL, mtaEventID [2) IMPLICIT STRING } SendRequest : — SET { [0] IMPLICIT SendHead, title [1] IMPLICIT STRING OPTIONAL, encrypted [2] IMPLICIT INTEGER { } OPTIONAL, File } SendHead ::= SET { TransferEventldentlfler, source [0] IMPLICIT ORName, - source FTA agentrecepients [l] IMPLICIT SEQUENCE OF AgentRedpient, - recipient FTAs Initiator^] IMPLICIT ORName, - initiator UA originatlngHost [3] IMPLICIT ORName, - initiator FTA NotiflcattonRequest [4| IMPLICIT INTEGER { report(0), noreport(l) } OPTIONAL, - defaults to noreport importanceOfContent [5] INTEGER { nonUrgent(0), normaI(l), urgent(2)} OPTIONAL, - defaults to normal sourceFOename [6] IMPLICIT Filename, sourceAcct |7] IMPLICIT STRING OPTIONAL, sourcePassword (8) IMPLICIT STRING OPTIONAL, ffleType [9] IMPLICIT INTEGER OPTIONAL { text(0), print(l), binary(2)} - defaults to text } AgentRecipient ::= SET { ftagent [0] IMPLICIT STRING, redpHst [l] IMPLICIT SEQUENCE OF Recipient, redpTlme [2] IMPLICIT STRING OPTIONAL} - time flies were delivered — to this ft agent Recipient : — SET { recipient [0] IMPLICIT ORName, dest Filename [1] IMPLICIT Filename, destAcct [2] IMPLICIT STRING OPTIONAL, destPassword [3] IMPLICIT STRING OPTIONAL, rejectReason [9] IMPLICIT INTEGER OPTIONAL { b*dSourceFile(l), badDestFUe(2), destFileNoAccess(3), destHostNoAccess(4), sourceHostNoAccess(6) } } Receipt Notification ::= SET { TransferEventldentlfler, destagent [ 0 ] IMPLICIT STRING, notiflcationTime [l] IMPLICIT STRING, notiflcationRecipient [2] ORName, deliveryLiflt [3] IMPLICIT SEQUENCE OF Recipient OPTIONAL, undeliverabfeList [4j IMPLICIT SEQUENCE OF Recipient OPTIONAL } — Application-defined TransferEventldentlfler ::= STRING - to be more structured later — current format: <hostname>:<time> Filename ::= STRING — to be more structured later File : — STRING - to be more structured later END Figure 3.4 - EANFTP FTA Protocol Data Units (CCITT X.409 syntax) Chapter 3 37 A SAFFTS Transfer -Request Sendtiead S end-Request ^ ~ FTAPDU SendHead Transfer Reply Fi l e Send Reply Z 3 Z I ' Receipt- \/ Receipt- \ .Notification ivNotificationj source r agent-vrecipients recipient dest-Filename i n i t i a t o r etc. destAcct dest-Passvord rej ect-1 Reason Figure 3 . 5 - Logical Structure of the EANFTP SendRequest PDU context sensitivitiy allow the codes to be represented with fewer bits than if the codes were application-wide unique. The combination of the PDU's tree structure and context sensitive encoding scheme make it easy to modify and upgrade protocols without the need of rewriting a lot of code. For example, if the modification of the protocol does not affect a sub-tree of the protocol then the code that accesses that sub-tree need not be Chapter 3 38. A SAFFTS r e w r i t t e n . CHAPTER 4 The Implementation This chapter -will describe some of the details of the implementation of EANFTP. The programs of EANFTP were written in the C programming language and were implemented under Berkeley UNIX version 4.2. There are two programs representing the two entities: the File Transfer User Agent Entity (FTUAE) and the File Transfer Agent Entity (FTAE). A person who wishes to request a file transfer explicitly runs the FTUAE program. Whereas the FTAE program is automatically activated whenever it receives any messages from the MTS. 4 . 1 . The User Interface This section will describe the implementation of the FTAUE program (FTAUEP). A user can do one of two things when running the FTAUEP: 1. Specify the transfer of a file: transfer request | | FTUA reply 4r" (transferid) Chapter 4 39 Implementation Chapter 4 40 Implementation 2. Inquire as to the status of a previously specified transfer request. status request (transferid) | FTUA I I status reply The information a user might supply to the FTUA in a transfer request include: 1. - the source file name which might include: 1.1 - the O/R address of computer system on which it resides 1.2 - an account on that computer system which might include: 1.2.1 - the password to that account 1.3 - the file's type (binary, text, print) 2. - a list of destination files each of which could include the O/R name, account, and password 3. - a request for notification upon successful delivery The syntax of these requests is shown in Figure 4.1. 4.2. Virtual Files This implementation of EANFTP has only a very rudimentary virtual file definition. The filename has no defined structure - it is just a printable character string. The semantic value of the string is determined by the computer system on which the entity that is interpreting the string resides. The users of EANFTP must be aware of the file naming syntax of the source and destination computer systems. Chapter 4 41 Implementation < option > <status> < request > < transfer > <destlist> <srcfilename> < typelessname > < hostid > <ident> <type> <optionlist> < transfer > | < status >. Transfer <srcfilename> {To} <destlist> { <optionlist> }. <typelessname >, <destlist> | < typelessname >. < typelessname > { [<type>] }. { < hostid> : } < filename >. < hostname > : { <ident> }. <userid> { / <password> }. Text | Binary | Print. - <option> <optionlist> | - < option >. Report J NOREport | High | NORMal | Low. Status <transferid> | Status All. (minimum abbreviations capitalized) ("{}" indicates an optional item) Figure 4.1 - Syntax of FTAUEP request For access rights, EANFTP uses the access rights associated with the user supplied account name. The account must reside on the same system as the file to which it refers. If an account is specified then a password must be supplied as well in order to validate the account. Only three basic file types are supported in this implementation: text - a regular sequential character (IA5, EBCDIC, etc.) file print - a sequential character file with printer carriage control binary - a sequential numeric file i.e. no collating sequence is used EANFTP transmits the octets of a binary file transparently, i.e. without interpreting their value. Because of this transparency, users who wish to transmit more complex or more structured files than the above three types can specify to EANFTP that the type of the files is binary. However it is the responsibility of the recipient to provide the proper semantic interpretation upon delivery. Chapter 4 42 Implementation The physical representation of the virtual files used for actual transmission in EANFTP are their UNIX 4.2 counterparts. For example, an EBCDIC text file would first have to be translated into an ASCII text file before it is transmitted. Similarly a print file with FORTRAN-like carriage control would have to have that carriage control transformed into its UNIX 4.2 equivalent. 4.3. Transfer Identifiers The transfer identifier (transferid) is what uniquely identifies the transfer request throughout the whole File Transfer System (FTS). In EANFTP, the transferid has two components to make it unique in the FTS. The major component is a serial number which is unique to the computer system on which the FTUAE resides. In this implemen-tation, this serial number is got from the system time clock. The other component of the transferid is the host name of the system and is appended to the front of the serial number. The host name, being an 0/R address (see Chapter 1, section 2.4.3), is unique to the Message Handling System (MHS) and to the FTS as well since EANFTP shares the naming space of the MHS on which it is based. In EANFTP, each FTUAE maintains a file which contains the last serial number the entity has forged. The serial number consists of the system clock time at the moment of the serial number's forging followed by a four digit sequence number. The sequence number is necessary as the system time's granularity is only to the second. When forging a new serial number, the FTUAE reads in from the file the last serial number forged and if the current system time is the same as that of the previously forged number the four digit sequence code is incremented by one and appended to the system time. If the system time differs then the sequence code is set to zeroes. As implemented, the FTUAE can forge at most 10,000 serial numbers per second. Because Chapter 4 43 Implementation each transfer request is assigned a transferid, the above rate is the theoretical limit to the number of transfer requests the FTUAE of a particular system can process. The syntax and an example of a transferid are shown in Figure 4.2. 4.4. Building of Protocol Data Units As mentioned in Chapter 3, Protocol Data Units (PDUs) are the units of communi-cation between agents in the FTS. The FTUAE sends a PDU to the File Transfer Agent Entity (FTAE) of the system with the source file in the transfer request. The FTAEs send each other PDUs for the ensuing communication necessary to accomplish the transfer. As outlined in Chapter 3, section 3.4.4, the PDU has a hierarchical tree structure. EANFTP's entities create these PDU trees with dynamically allocated storage before submitting them to their respective MTAs. The C Structure shown in Figure 4.3 corresponds to a field (i.e. a node) of the PDU. The id contains the protocol code for Syntax <transferid> ::= <hostname> : <serialno> <hostname> ::== O/R name <serialno> ::= <systemtime><sequenceno> Example ubc-ean:031812070001 where: ubc-ean is the <hostname > 03181207 is the <systemtime> at forging (day, hour, minute, second) 0001 is the <sequenceno> Figure 4.2 - Transfer Identifiers in EANFTP Chapter 4 44 Implementatio n typedef struct ENODE { long id; long length; unsigned char* primitive; struct ENODE* constructor; struct ENODE* next; } ENODE; Figure 4.3 - Definition in C of ENODE - the PDU node this node. The constructor points to the PDU sub-tree of this node. The length is the number of octets needed to represent the "flattened" subtree of the node plus the number of octets needed to represent the node itself. If this node is a leaf-node then primitive will point to the data contained in this leaf-node. The next variable points to the sibling of this node in the PDU tree. 4.5. Use of the UA - SDE Interface of EAN in EANFTP As shown in Figure 3.3, P3 is the Sending-Delivery Entity (SDE) - MTAE protocol by which the UAs of the CBMS communicate with their associated MTAs. The UA of the EAN CBMS has a subroutine interface (the P2 - P3 or UAL interface) that uses P3. A P2 (UAE - UAE) message submitted to the subroutine interface is transformed into a form acceptable for P3 which is then given to the MTA. Rather than using P3 directly, this implementation of EANFTP "packages" its PDUs in a P2 message and then uses the EAN UAL interface mentioned above. This strategy was chosen in order to take advantage of a previously written (and previously debugged) UA - SDE interface. The logical configuration is shown in Figure 4.4. Protocol PF is the FTA - FTA protocol. The UAL interface deals of course with P2-type messages not PF (FTP) file mes-sages. Consequently the FTUAE and FTAE must transform a PF PDU into a P2 PDU Chapter 4 45 Implementation UAL. MTL. FTUAE FTAE M S S UAE. MTAE PF P2. ^ 7 PL MTAE S3 S2 E 3 . "7 FTUAE FTAE MHS UAE" S D E S I Figure 4.4 - Layered Configuration of EANFTP before they can submit it to the UAL interface. Similarly a P2 PDU once received from the UAL interface must be transformed back into the "straight" PF PDU. As explained in Chapter 2, section 2.4.1, a P2 message consists of an envelope (or heading) and its content (or body). The content is transparent to the M T A and will, in EANFTP's case, be a PF PDU. The envelope will contain, among other fields, the reci-pient of the message - in this case the F T A of the source or destination system. Immediately prior to submitting an EANFTP PDU to the UAL interface a "regu-lar" (i.e. P2) message is created with the PF PDU as its content and with the recipient FTA's O/R address extracted from the PDU copied into the envelope. This message is in turn submitted to the M T A by the UAL interface. The reader is reminded that since E A N F T P shares the address space of EAN there is no discrepancy or ambiguity with the recipient O/R address which was extracted from the PF PDU and copied to the P2 mes-sage envelope. That is, there is no possibility of the CBMS confusing this O/R address with that of any other entity in the MTS. Immediately upon receipt from the UAL interface, the content of the P2 message, which is the PF PDU, is stripped from the P2 message and passed up to the F T A E . If the MTS is unsuccessful in delivering, a message to the specified recipient it will notify Chapter 4 46 Implementation the sending UA of this failure. This notification is passed from the UAL interface to the FTAE. A portion of the CCITT P2 UA PDU is shown in Figure 4.5. 4.5.1. UAL Interface Primitives There are six UAL interface primitives which the FTUA and FTA of EANFTP use: 1. UAL_init: which connects a UAE and confirms its identity with its associated MTAE. Confirmation involves the submission of a password. The argu-ments of the primitive are the O/R name and address of the agent and its password. The primitive returns with an indication of success or the reason why it failed. Once connected, the UA can make use of the services provided by the MTL, primarily the submission and reception of messages. - IP-message UAPDU IM-UADPU — heading Heading - body BODY S E Q U E N C E {Heading, Body} SET { IPMesageldd, originator [0| IMPLICIT ORDescriptor OPTIONAL, authorizingUsers [l] IMPLICIT S E Q U E N C E OF ORDescriptor OPTIONAL, — only if not the originator primaryRecipients [2] IMPLICIT S E Q U E N C E OF Recipient OPTIONAL, copyRecipients [3] IMPLICIT S E Q U E N C E OF Recipient OPTIONAL, blindCopyRecipients [4] IMPLICIT S E Q U E N C E OF Recipient OPTIONAL, inReplyTo [5 IMPLICIT IPMessageld OPTIONAL, omitted if not in reply to a previous message obsoletes crossReferences subject expiryDate replyBy replyToUsers 6 IMPLICIT S E Q U E N C E OF IPMessageld OPTIONAL, 7 IMPLICIT S E Q U E N C E OF IPMessageld OPTIONAL, 8 CHOICE {S61String} OPTIONAL, 9 IMPLICIT Time OPTIONAL, - if omitted, expiry date is never 10] IMPLICIT Time OPTIONAL, 11] IMPLICIT S E Q U E N C E OF ORDescriptor OPTIONAL, — each O/R descriptor must contain an O/R name [12] IMPLICIT INTEGER {low(0),normal(l),high(2)} D E F A U L T normal, [13] IMPLICIT INTEGER {personal(l),private(2),companyConfidentiai(3)} OPTIONAL, autoforwarded [14] IMPLICIT B O O L E A N D E F A U L T FALSE — indicates that the forwarded message body part(s) were autoforwarded - } ::= S E Q U E N C E OF BodyPart importance sensitivity Figure 4.5 - Portion of CCITT definition of IM - UAPDU [CCIT420-83] Chapter 4 47 Implementation 2. UAL_term: which disconnects a previously connected UA from its associated MTA. There are no arguments. 3. UAL_aend: which converts a P2 message into a P3 message and submits it to the MTAE to be delivered to the destination specified in its envelope. This is where the "flattening" of the PDU takes place. The message is an argument of the primitive. An indication of success or failure to submit the message is returned. A successful submission does not imply that it can be successfully delivered. Rather, success indicates that the message is well-formed. Upon successful submission, the MTA returns an identifier which uniquely identifies the P2 message with the MTA. This identifier is placed into the submission envelope of the P2 message. Alternatively if the message is submitted with an identifier already in the envelope then that identifier is used and returned by the MTA. identifier | | 4. UALjreceive: which receives from the MTAE the next P3 message in the queue of messages delivered to this MTAE and destined for this UA. The ture. The P3 message is then converted into its P2 counterpart. There are no arguments but the P2 message itself is returned. UAL send SDE | "flattened" message received from the MTAE is built up into its tree struc UAL receive SDE | P2 message Chapter 4 48 Implementation 5. UAL_itatu*: which receives from the MTAE the next status message in the queue of messages destined for this UA. A status message is an indication of the failure of the MTS to deliver successfully a P2 message previously sub-mitted by the UA. These submissions are identified by the identifiers returned by the MTA during the invocation of a UAL_tend. 6. UAL_confirm: which relieves the MTA of "responsibility" for a message received during the invocation of UAL_receive or UAL_atatus (see Chapter 3, Section 3.4.3). An unconfirmed message remains on the MTA's queue. Sub-sequent invocations of UAL_reccive or UAL_ttatiu would result in the unconfirmed message being redelivered. A State Transition Diagram relating the above primitives is shown in Figure 4.6. 4.6. A Walk-through a File Transfer For this section the reader is advised to refer to Figures 3.4 (EANFTP FTA Proto-col Data Units) and 4.5 (definition of IM - UAPDU). A person on Host A runs the FTUAE program and specifies that a file from Host B be sent to a file on Host C. The FTUAE builds a TranaferRequeat PDU (TRqPDU) (see Figure 3.4). A P2 message is created with the TRqPDU as the message PDU's Body field. The recipient of this message and its importance are copied from the PF PDU into the envelope of the P2 message. (The from and importanceOj'Content fields of the Send-Head field of the TRqPDU are copied into the primaryRecipient and importance fields respectively of the Heading field of the IM - UAPDU). The only allowable recipient of a PF PDU message is an FTA. EAN O/R addresses have the following format: mailboz@8ubdomain-liat Chapter 4 49 Implementation IDLE UAL receive UAL_status confirm nTOflNFr'RWETT MESSAGE" UAS.. conf irm UNCONFIRMED STAIUS MESSAGE. Figure 4.6 - UAL Primitives State Transition Diagram where aubdomain-liat is an unique O/R address (by definition) and mailbox is a name unique to the host identified in the aubdomain-Hat. Each EAN site has a special O/R name/address reserved for its FTA. The mailbox name is common throughout the domain of EAN sites. In the current implementation of EANFTP this mailbox name is "ftagent". Therefore the O/R address of any FTA is: ftagent® s ubdomain- list Chapter 4 50 Implementation This P2 message is then submitted to the UAL interface via UAL_send which in turn submits it to its associated MTA. The MTS delivers the message to the MTA asso-ciated with the FTA of Host B. The SDE has a facility by which a user specified program can be run when the SDE receives a message for that user. In EANFTP, the only allowable recipients of PF mes-sages are the FTAs. Whenever the SDE receives a message for an FTA, it runs the FTAE program. This mechanisms avoids having the FTAE run as a "daemon" pro-gram. The FTAE of Host B, once activated receives, via UAL_receive, the P2 message from the initiating FTAUE of Host A whose content is the TRqPDU (the Body field of the IM - UAPDU). The FTAE obtains a copy of the source file identified by the file name contained in the aourceFilename field of the SendHead field of the TRqPDU, source file. The FTAE then builds a SendRequest PDU (SRqPDU) into which it places the source file (the File field of the SRqPDU. Another P2 message is created where this time the content is the SRqPDU and the list of destination FTA O/R addresses is copied from the TRqPDU into the envelope of the P2 message. (A list of the ftagent fields -suitably transformed - of the AgentRecipient field of the Sendhead field of the SRqPDU is created and copied into the primaryRecipienta field of the Heading field of the IM -UAPDU). The P2 message is submitted to the MTAE once again via UAL_aend. The FTAE of Host C is activated upon receipt by its associated SDE of the P2 message. The message is received via UAL_reeeive and the SRqPDU is stripped from the P2 message. From the SRqPDU, the actual file is copied to the destination file specified in the PDU (the deatFilename field of the Recipient field of the recipliat field of the AgentRecipient field of the SendHead field of the SRqPDU). Chapter 4 51 Implementation The destination FTAE then constructs a Send Reply PDU (SRpPDU) with the appropriate indication of the FTAE's success in delivering the file (the deliveryLiat and undeliverableList fields of the SRpPDU). The SRpPDU is placed into the content of a P2 message which is sent via UAL_*eni to the FTA of Host B (the FTA who sent the SRqPDU). The FTAE of Host B is activated upon receipt of this message by its SDE and in turn receives the P2 message via UAL_receive. The SRpPDU is stripped from the P2 message. After the information in the SRpPDU is processed it is transformed into a TransferReply PDU (TRpPDU) - essentially only the id field is changed. A new P2 mes-sage is created with the TRpPDU as its content and the recipient being the FTA of Host A. The P2 message is sent via UAL_»end. The FTAE of Host A is activated upon receipt of this message by its SDE and in turn receives the P2 message via UAL_reeeive. The TRpPDU is stripped from the P2 message. If the user had so specified in his transfer request (the NotificationRequest field of the SendHead field of the TRqPDU), the FTAE sends a P2 message, via UAL_send, to the mailbox of the user. If the MTS were unsuccessful in delivering the TRqPDU to Host B, the FTAE of Host A would be activated where it could receive, via UAL_$tattu, an indication of this failure. The FTAE would then send an EAN mail message to the user notifying the per-son of this failure. Similarly if the MTS were unsuccessful in delivering the SRqPDU to Host C, the FTAE of Host B would be activated where it could receive, via UAL_statut, an indica-tion of this failure. The FTAE would then build a TRpPDU indicating the failure (in the undeliverableList field of the TRpPDU) and would place it in a P2 message which it Chapter 4 52 Implementation would then send, via UAL_send, to the FTA of Host A. The FTAE of Host A, upon receipt of the TRpPDU, would notify the user. If the FTAE of Host C were not able to access the specified destination file then an appropriate SRpPDU would be sent to the FTAE of Host B who would relay it (as a TRpPDU) to the FTAE of Host A. If the FTAE of Host B were not able to access the specified source file then an appropriate TRpPDU would be sent to the FTAE of Host A. In both cases, upon receipt of these negative TRpPDUs, the FTAE of Host A notifies, via a mail message, the person who requested the transfer. The FTAE of Host A will always notify the requester of unsuccessful file transfers. Figure 4.7 shows the relation-ship between the PDUs and the agents involved in a file transfer. 4.7. Keeping Track of Transfers in Progress A person is allowed to interrogate the FTUA for the status of his previously requested file transfers (the Status Request). Therefore the FTUA and the FTA must keep track of the transfers in progress. As mentioned above, a user can specify multiple destinations for a file transfer. The file transfer can be in one of three states for each destination file specified: delivered - where the file has already been delivered undeliverable - where the FTS, for some reason, was unable to deliver the file in progress - where it has yet to be determined whether the file was suc-cessfully delivered The FTA could receive a Status Report from the MTA stating the MTS's inability to deliver the TRqPDU to the source host (Host B in the above walk-through). This TRqPDU is identified by the mta identifier (the identifier returned by the UAL_send Chapter 4 5 3 Implementation Host A. Figure 4.7 - Relationship Between PFPDUs and Agents in EANFTP primitive). The FTA must be able to cross-reference the mta identifier with the TRqPDU to which it refers and thereby notify the person who had requested the transfer. One has the option of asking for notification of the successful transfer of a file. As described in the walk-through above, the FTAs send each other transfer confirmations. The FTA of Host A must know, when it receives a confirmation message from the FTA Chapter 4 54 Implementation of Host B, whether it should send a notification to the person who requested the transfer. This information could be passed in the reply PDUs (SRpPDU and TRpPDU). However, in the attempt to reduce network traffic, EANFTP's FTAs instead keep a record of the identity and location of the transfer's requester. These records are accessed whenever the corresponding reply PDU are received. The FTA Host B might be unsuccessful in delivering a SRqPDU to Host C . It must keep a record of the transaction indexed by the mta identifier that was returned by UAL_8cnd. The FTA and FTUA maintain a data base to keep track of these transfers. The logical structure of the data base is shown in Figure 4.8. A host could be the "Host A" of one transfer request and the "Host B" of another. That is, the host could be the system from which the transfer request was issued (in this case Host A) or it could be the host on which the source file of request is located (in this case Host B). A record in the data base consists of the PDU - either a TRqPDU or a SRqPDU (without the File field) depending on whether the FTA is a "Host A" or "Host B" FTA for the transaction stored. The "Host A" transaction records are structured two ways. In the first structure, all the records for "Host A" transactions, irrespective of the person who specifed them, are stored together. In the second, the transaction records for each person are stored together. This second structure allows a person to ask the FTUAE for the status of all his outstanding requests. The "Host B" transactions records are organized such that all the records for each initiating ( "Host A") host are stored together. As mentioned above, whenever a message is submitted to the MTA, an identifier is returned. The MTS uses these identifiers when reporting unsuccessful deliveries to the Chapter 4 55 Implementation Transfer^ .. H o s t A<. c r a n s a c t i o n s records iniciatinjg-Host 1 initxacin^ Host 2. individual records MIA records individual records initiating-Host a send, records "Hose B" transactions Figure 4.8 - Logical Structure of the Transaction Data Base submitting UAs. EANFTP store these identifiers in the data base. They are linked to the transaction records (both "Host A" and "Host B") to which they refer. For each destination file specified in the request, a record is kept of its state ( "delivered", "undeliverable", or "in progress"), then the data base record for a transac-tion is maintained until all the specified destination files are in the "delivered" or "undelivered" state. The record would then be deleted from the data base. Chapter 4 50 Implementation 4.8. Forwarding of Receipt Notifications There were two strategies which we could have used to implement the passing of FTA receipt notification: 1. The FTA of Host B collects all the outstanding receipt notifications from the FTAs of the destination files (or from the MTA indicating a failure to reach the FTA of a destination file) before forwarding the notifications to the FTA of Host A. Similarly, the FTA of Host A collects all the outstanding receipt notifications for a particular transfer request before notifying the person who requested the transfer. 2. The FTA of Host B relays any receipt notification to the FTA of Host A as soon as the FTA receives it. Similarly, the FTA of Host A notifies the requester for each notification that the FTA receives. This implementation of EANFTP chose the second approach. This prevents one reluc-tant FTA, which is either not operating or operating intermittently, from impeding the notification process. There is a nuisance factor to this strategy. If the request has several destination files and the person who had requested the transfer also specified that he be notified by the FTA of the transfer's deliveries then he would receive a separate message for each file. CHAPTER 5 Results and Evaluation EANFTP is currently running on three machines: - DEC VAX 780 (hostname ubc-vision) - DEC VAX 750 (hostname ubc-ean) - SUN Workstation (hostname ubc-andrew) All three machines run Berkeley UNIX version 4.2 and are interconnected by an Ether-net. 5.1. Results The following actual examples will demonstrate the capabilities of EANFTP. 5.1.1. Two Party Transfers All FTPs allow two party transfers (transfers that involve transfers between only two machines). The following shows examples of how EANFTP accomplishes these transfers. Pulling a File "Pulling" a file involves obtaining a local copy of a file from a remote machine. For example: A person with the account "sadowski" on the local host "ubc-ean" wanted to have a copy of the file "/lcv/ean/src/ftp/test" on host "ubc-vision" deposited in the local file "/user/sadowski/foo". He accomplished this by giving the following EANFTP com-mand: Chapter 5 57 Results & Evaluation Chapter 5 58 Results & Evaluation ftp ubc-vision:eanwork/ptp:/lcv/ean/src/ftp/test /user/sadowski/foo -report where: - "eanwork" is an account on host "ubc-vision" with password "pit;" which should have at least "read access" to the file "/lcv/ean/src/ftp/test". - "-report" indicates that the person issuing the request wanted to be notified by EANFTP upon the successful delivery of the file. Unsuccessful deliveries cause EANFTP to notify the requester whether or not he specified the "report" option. EANFTP gave the person the tramferid for this request with the following message: request assigned id: 211108410000 . where: "211108410000" is the serial number component of this particular transferid (see Chapter 4, section 4.3). The host name component (in this case "ubc-ean") is implicit. With this configuration (namely all three machines connected to an Ethernet) the transfer was done almost instantaneously (depending, of course, on the size of the file). Since the person had requested a notification upon successful delivery, the following EAN mail message was sent to him: Chapter 5 5 9 Results & Evaluation From: FTP agent <ftagent@ubc-ean> To: <sadowski@ubc-ean> Subject: Transfer Report FTP Transfer no: 241108410000 Successfully transferred u be- vis ion:e anw or k:/ lev/ ean/src /f tp /tes t to ::/user/sadowski/foo Pushing a File "Pushing" a file involves sending a copy of a local file to a remote machine. For example: A person with the account "sadowski" on "ubc-ean" wanted to have a copy of the local file "/user/ean/sre/ftp/test" deposited in the file "/usr/u/neufeld/bar" on "ubc-andrew". He accomplished this by giving the following EANFTP command: ftp /user/ean/src/ftp/test ubc-andrew:sadowski/pu;:/usr/u/neufeld/bar where: - "sadowski" is an account on host "ubc-andrew" with password "pw", that should have "write access" to the specified file if it already exists or be allowed to create a file by that specified name. - the "report" option was not specified so that the person would have only be notified if EANFTP were unsuccessful in delivering the file. After the tranaferid was reported as in the "pull" example above, the transfer was done almost instantaneously. Chapter 5 6 0 Results & Evaluation 5.1.2. Three Party Transfers The following demonstrates a major capability of EANFTP: "Three Party Transfers" - the ability to request from one host that a file from a second host be sent to a third. For example: A person with the account "sadowski" on host "ubc-ean" wanted a copy of the file "/usr/u/ean/src/ftp/test" on host "ubc-andrew" deposited to the file "/usr/ubc/majka/foo" on host "ubc-vision", he issued the following command: ftp ubc-andrew::/usr/u/ean/src/ftp/test ubc-vision:majka/pu>:/usr/ubc/majka/foo -report where: - "/usr/u/ean/src/ftp/test" should be "readable" by anybody as this person did not specify an account on "ubc-andrew". - "majka" is an account on host "ubc-vision" with password "pu>" that should have "write access" to the destination file if it already exists or be able to create a file with that name if the file does not exist. - the "report" option was requested in this case. In this example, after the transferid was reported as in the "pull" example above, the transfer was done almost instantaneously. The following message was sent to the person: Chapter 5 01 Results & Evaluation From: FTP agent <ftagent@ ubc-ean > To: <sadowski@ubc-ean> Subject: Transfer Report FTP Transfer no: 251211320001 Successfully transferred ubc- andrew:: / usr/u/ean/ src/ft p/test to ubc-vision:majka:/usr/ubc/majka/foo 5.1.3. Multiple Recipients The ability to transfer a copy of a file to several destinations is another major capa-bility of EANFTP. For example: A person with the account "sadowski" on host "ubc-ean" wanted to send copies of the file "/lcv/ean/src/ftp/status.c" on host "ubc-vision" to the local file "/user/ean/src/ftp/status.c" and to the file "/usr/u/ean/src/ftp/status.c" on host "ubc-andrew", he issued the following command: ftp ubc-vision:eanwork/pu>i:/lcv/ean/src/ftp/status.c :sadowski/pu/5:/user/ean / src/ftp/status.c, ubc-andrew :neufeld/pt0<9:/usr/u / ean/src /ftp/status.c -report where: - "eanwork" is an account on host "ubc-vision" with password upwl" that should have at least "read access" to the file "/lcv/ean/src/ftp/status.c". - "sadowski" and "neufeld" are accounts on hosts "ubc-ean" and "ubc-andrew" respectively, with passwords: upw2" and "pu;5" respectively, with the ability to "write" or "create" their Chapter 5 82 Results & Evaluation respective destination files. EANFTP gave the person the transferid for this transaction and once again this transfer was accomplished quickly. Two notification messages would have been sent - one for each destination file. The format of these messages is the same as those shown in previ-ous examples. 5.1.4. When Things Go Wrong The following examples show how EANFTP handles some of the error conditions that one is likely to encounter in transferring a file. If, in example, the file "/Icv/ean/src/ftp/test" on host "ubc-vision" did not exist or the account "ean-work" did not have access to it, the following EAN message would have been sent: From: FTP agent <ftagent@ubc-ean> To: <sadowski@ubc-ean> Subject: Transfer Report FTP Transfer no: 241108410000 Unable to transfer ubc-vision:eanwork:/lcv/ean/src/ftp/test to ::/user/sadowski/foo Reason: Invalid source filename, or no access to the source file If, on the other hand, the person with the local account "sadowski" did not have write access to the local file "/user/sadowski/foo", or was not able to create a file with that name, the following EAN message would have been sent: Chapter 5 63 Results & Evaluation From: FTP agent <ftagent@ubc-ean> To: <sadowski@ubc-ean> Subject: Transfer Report FTP Transfer no: 241108410000 Unable to transfer ubc-vision:eanwork:/lcv/ean/src/ftp/test to ::/user/sadowski/foo Reason: Unable to access destination file If, with the same example, the host "ubc-vision" were permanently dropped from the network, the following message would have been sent: From: FTP agent <ftagent@ubc-ean> To: <sadowski@ubc-ean> Subject: Transfer Report FTP Transfer no: 241108410000 Unable to transfer ubc-vision:eanwork:/lcv/ean/src/ftp/test to ::/user/sadowski/foo Reason: Unable to reach source host If, in example 5.1.2, host "ubc-vision" were permanently dropped from the network, the following message would have been sent: From: FTP agent <ftagent@ubc-ean> To: <sadowski@ ubc-ean > Subject: Transfer Report FTP Transfer no: 251211320001 Unable to transfer u be- andrew ::/usr/u/ean/src/ftp/test to ubc-vision:majka:/usr/ubc/majka/foo Reason: Unable to deliver file to destination host Chapter 5 04 Results & Evaluation 5.1.5. Status of Transfers in Progress When one is running the "ftp" program, one can inquire as to the status of a previ-ously requested transfer. For example: If, in example 5.1.3, the person with the local account "sadowski" were to inquire as to the status of the transfer request and when he did inquire, the file ":sadowski:/user/ean/src/ftp/status.c" had already been delivered and the file "ubc-andrew:neufeld:/usr/u/ean/src/ftp/status.c" had not, the following would be printed: FTP Transfer no: ubc-ean:251524310000 ubc-vision :eanwork:/lev / ean/src/ftp/status .c to :sadowski:/user/ean/src/ftp/status.c - delivered 84/03/25 15:27:02 ubc-andrew:neufeld:/usr/u/ean/src/ftp/status.c - in progress Indicating the fact that the file had been delivered to the first destination and when it had been. If, on the other hand, the file ":sadowski:/user/ean/src/ftp/status.c" had yet to be delivered but it had already been determined that, for some reason, EANFTP was unable to successfully deliver the file "ubc-andrew:neufeld:/usr/u/ean/src/ftp/status.c", the following status message would be printed: FTP Transfer no: ubc-ean:251524310000 ubc-vision:eanwork:/lcv/ean / sre/ftp / status .c to :sadowski:/user/ean/src/ftp/status.c - in progress ubc-andrew:neufeld:/usr/u/ean/src/ftp/status.c - undeliverable 84/03/25 15:26:11 5.2. Evaluations EANFTP is a Store-and-Forward FTS (SAFFTS). As mentioned in Chapters 2 and 3, that most FTSs are connection-baaed and as such will generally be faster than Chapter 5 65 Results Sc Evaluation EANFTP. However, EANFTP was not designed to compete with connection-based FTPs on that criterion. Instead these evaluations will concentrate on EANFTP's ability to achieve its design goals. As shown above, EANFTP is quite capable of handling a. third party transfer. Not being connection-based resulted in eliminating most of the problems inherent in coordi-nating such transfers. It may be the case, because of network topology, that, using the nomenclature of Chapter 4, section 4.6, a Host A cannot connect with a Host C, whereas a Host B, with which Host A is able to connect, can connect with that Host C. This is shown in Figure 5.1. If Host A wanted to send a file to Host C, a regular two-party connection based request f i l e be sent from Host B \ to Host C Figure 5.1 - Connection Asymmetry Chapter 5 86 Results & Evaluation FTP would fail. However, with EANFTP, Host A could send the file to Host B, and then request that Host B send it to Host C. This has been tried successfully with the machines running EANFTP. When the link between "ubc-andrew" and "ubc-vision" was temporarily inoperative, one was still able to send a file between the two machines by first transferring the file from one of the hosts to "ubc-ean" and then, in a separate request, transfer the file from "ubc-ean" to the other host. EANFTP demonstrated a capability virtually non-existent in other FTPs: the abil-ity to send a file to multiple recipients. This was a serendipitous benefit of mapping EANFTP onto the MTA of a CBMS, where this capability is an integral function. Assuming that the MTA is itself reliable, then EANFTP, which places all responsi-bility of retransmission and error-checking on the MTA, is as reliable as the MTA. Since EANFTP is an SAFFTS, when and how these retransmissions are done are transparent to the user. In choosing an SAFFTS, the user is not that concerned with speed (within a reasonable limit). The FTUAE, when running, was approximately 100K bytes large. The FTAE, when running, was approximately 140K bytes. Both entities were co-resident with their MTAs and as such were linked to the MTAE. The MTAE code component is approxi-mately 40K. EANFTP's UA components were hastily written and could be made sub-stantially more space and time efficient. In comparison "ftp", the Berkeley UNIX FTP, occupies approximately 86K bytes. However, the EANFTP's entities do not need to be co-resident with their associated MTAs and in these cases they would occupy less space than their co-resident counterparts. Chapter 5 67 Results & Evaluation This implementation of EANFTP has no theoretical upper-bound to the size of files it will try to transfer. This is not to say that it can transfer any size file. If an inter-mediate MTA is not able to relay a file a status message would be returned to the requester. CHAPTER 6 Conclusions This thesis set out to investigate the efficacy of Store-and-Forward File Transfer. What has been shown is that a SAFFTS can be realized and relatively easily if the underlying medium of communication is the Message Transfer Layer of a Computer Based Message System based on the current CCITT proposal for such systems. By designing and implementing an application other than that originally intended in the CCITT CBMS design, we have shown that the model is both general and powerful enough in its layering to allow such diverse customized User Agents. We have shown how a SAFFTS makes genuine third party transfers relatively easy to implement, how chosing the MTL of a CCITT CBMS has eliminated the concerns of check-pointing and its error-recovery as those functions^ are assumed by the MTL. 6.1. Further Improvements The following are areas requiring further work and research. 6.1.1. Management Functions In the current design and implementation of EANFTP, there is no facility for "management" functions such as getting directory listings from remote hosts, interrogat-ing the virtual file attributes of remote files, creating or destroying remote files. As many FTPs have this capability, it is one that should be included in EANFTP. The CCITT Presentation Transfer Syntax (the format of the PDUs used by EANFTP) makes this inclusion straight forward. Chapter 0 08 Conclusions Chapter 8 69 Conclusions A ManagementRequest PDU (MRqPDU) and a ManagementReply PDU (MRpPDU) could be defined at the same level of the PF PDU tree as the TRqPDU, SRqPDU, TRpPDU, and SRpPDU. As explained in chapter 3, the inclusion of a MRqPDU and a MRpPDU should not affect the code already written to handle the other PDUs of the same level. However before these Management Requests can be properly defined, more work should be done on the specification of a virtual filestore. 8.1.2. The Virtual Filestore Currently, EANFTP can handle only three types of files: text, print, and binary. Conspicuously absent is any provision for a hierarchical file. The OSI Virtual File Ser-vice document [OSI-83] defines a virtual filestore. Ideally, EANFTP should adopt this standard, which addresses both File and Access attributes (see Chapter 2, section 2.1.1). OSI document provides for hierarchically structured files. It treats them as oriented trees (see Figure 2.4). The CCITT Presentation Syntax is well suited to representing tree structures. The OSI document does not deal with network-structured files (struc-tured files with nodes of in-degree greater than one). With a full implementation of the OSI definition, EANFTP could dispense with the need for specifying an account and password for a file. The OSI definition has a pass-word attribute which is dissociated from the account of the owner the file. Instead of supplying an account and associated password that has access to the file (the access being system dependent), one need only supply the password particular to the file requested. The access (which is another file attribute of the OSI definition) one would have would be standardized throughout the FTS. Chapter 6 70 Conclusions 6.1.3. Automatically Initiated Jobs Upon the delivery of a file, would it be desirable and feasible to initiate a user specified program with the file as input? This facility could be used, for example, to automatically print a file upon its receipt. The delivery of a file could initiate a data base update program. The syntax and semantics of this facility have yet to be sketched. 6.1.4. Cancellation of In-progress Transfers There is no facility in EANFTP to cancel transfers once they have been requested. In a CCITT based CBMS, the MTA has the capability of cancelling messages previously submitted to it by its associated UAs. However the MTA can only cancel those mes-sages that have yet to be forwarded to another MTA. EANFTP could make use of this feature in providing a cancellation service. It would be considerably more complex to handle the cancellation of transfer requests'that have already been forwarded to another MTA. 6.1.5. Bundling of Source Files EANFTP can only transfer one source file per request (even though one could specify an unlimited number of destination files). It would be desirable to be able to "bundle" a group of source files, which, upon delivery, could be automatically "un-bundled" to their respective destination files for that particular host. 6.1.6. Encryption The OSI definition specifies a File Encoding attribute whereby one can access and transfer encrypted files. EANFTP should make provision for such transfers. A field, the encrypted field, has already been defined in the SRqPDU but is not used by EANFTP, as of yet. Ostensibly, the field would identify the encryption algorithm used by the Chapter 6 71 Conclusions delivered file. 8.1.7. Partial Transfers There is no facility in EANFTP to transfer a sub-range of a file to another file. There is, however, a "hook" already defined in the TRqPDU (the pageRangea field). 6.2. An Interesting Application An application that could make use of EANFTP in its current state is a "Docu-ment Retrieval System". This system would allow one to acquire copies of documents located on various systems throughout the network to which one is connected. A data base (either centralized or distributed) would be kept of the available documents and their locations in the network. A person wanting a particular document would run the document retrieval program which would access the appropriate information from the document data base and then the program would issue an EANFTP transfer request, transferring the document to the location specified by the person. The speed with which these documents are delivered is usually not that important and as such would be ideally suited for a Store-and-Forward transfer. This application would be ideally suited for the dissemination of academic papers of various universities to interested parties. References [BBN-80]: Forsdick H. Autodin II File Transfer Protocol. Bolt, Beranek and New-man Inc. Report no. 4246, February 1980. [CCIT400-83]: Comite Consultatif Internationale de Telegraphique et Telephonique. Recommendation X.400, Message Handling Systems: Systems Model-Service Elements, December 1983. [CCIT409-83]: Comite Consultatif Internationale de Telegraphique et Telephonique. Recommendation X.409, Message Handling Systems: Presen-tation Transfer Syntax and Notation, December 1983. [CCIT411-83]: Comite Consultatif Internationale de Telegraphique et Telephonique. Recommendation X.411, Message Handling Systems: Message Transfer Layer, December 1983. [CCIT420-83]: Comite Consultatif Internationale de Telegraphique et Telephonique. Recommendation X.420, Message Handling Systems: Interper-sonal Messaging User Agent Layer, December 1983. [ECMA-80]: European Computer Manufacturers Association. Standard ECMA File Transfer Protocol. Fifth Draft. ECMA/TC23/80/158. 1980. [ISO-80]: International Standards Organization. Data Processing - Open Systems Interconnection - Basic Reference Model. ISO/DP7498, December 1980. [ISO-82]: International Standards Organization. Working Draft on File Transfer, Access and Management. ISO/TC 97/SC 16: N 1190, N 1222, N 1223, N 1124, June 1982. [NBS-82]: National Bureau of Standards, Institute for Computer Sciences and Tech-nology. Features of a Message Transfer Protocol. N 50. January 1982. [NEIG-73]: Neigus H. File Transfer Protocol, NIC 17759, ARPAA RFC, July 73. [NEUF-83]: Neufeld, G., Sample R., Hilpert B., Deering S., and Demco J. The EAN Distributed Message System. University of British Columbia, Department of Computer Science Technical Manual, November 1983. [NPL-81]: Guy M. (ed.) A Network Independent File Transfer Protocol, As revised by the File Transfer Protocol Implementors' Group of the UK Department of Industry Data Communications Protocol Unit. FTP-B(80), February 1981. [TELE-73]: Telnet Protocol Specification: ARPA Doc. NIC 18639, August 1973. Appendix I F T P User Manual Introduction FTP is a system of programs that allows one to transmit a computer file to one or more destination computers over a data network. The system makes use of the EAN Distributed Message System1 (q.v.) and as such, FTP is restricted, at present, to the domain of those sites that use EAN. The system allows one: to retrieve files from a remote computer to be delivered to a local destination ( "pull"); to send files from a local computer to a remote destination ( "push"); to send a file from one remote computer to another remote destination ( "third party transfer"); to send copies of a file to any number of destinations ( "broadcast"). FTP makes use of EAN, a computer-based message system, as its underlying means of transmission. As such, it is a "Store-and-Forward" file transfer system. The actual transmission of a file might not immediately follow the initiation of a transfer request. The actual transmission of a file (as opposed to the request that it be transferred) is accomplished in the background - i.e., transparent to the initiator of that request. As with most message systems, the transmission might be done immediately or as late as overnight. Thus FTP is suitable for those transfers that immediacy of transmission is not a paramount concern. ^he E A N Distributed Message System, G. Neufeld, et al, Dept. of Computer Science, UBC, Nov. 1983. Appendix I 74 FTP User Manual FTP is especially suitable for: specifying file transfers from machines with no real-time access to the specified files the distribution of files to multiple destinations the transmission of very large files over slow, unreliable media - thus FTP is useful for those people who do not want a terminal monopolized by such a transmission third party transfers (where both the source and destination files are on remote computer systems) transfer of files where transmission costs are a major concern broadcasting - e.g. the distribution of new releases of a document or a pro-gram to several subscribing destinations. There are three distinct phases when transferring files using FTP: 1. The specification phase, where one indicates to FTP which files one wants transmit-ted and to where. 2. The transmission phase, where the files are actually transmitted in the background. 3. The notification phase (optional), where the initiator of the transmission is notified by FTP of the transmission's success or failure. This document describes the specification and notification phases. Appendix I 75 FTP User Manual Transfer Specification In initiating a transfer, one specifies the source filename (i.e. the name of the file one wishes to be transmitted) and one or more destination filenames (i.e. the files to which you want the source file sent). For each source and destination file one can optionally specify its content's type. A file can be one of three types: 1. Text - a file of characters (ASCII or EBCDIC for example). 2. Print - a file of print-ready characters. 3. Binary - a file of numeric data. A file is specified using the following grammar: <typelessname> { [<type>] }. { <hostid> : } <filename>. <hostname> : { <ident> }. <userid> { / < password > }. Text | Binary | Print. the name of the computer system on which the file resides. the name of the file as known in the computer system on which it resides, a computer account on the same system as the file. The account should have appropriate access rights to the file, the <userid>'s password allowing access to the computer system on which the file resides. (minimum abbreviations capitalized) <srcfilename> <typelessname> <hostid> <ident> <type> <hostname> < filename > <userid> <password> Appendix I 76 FTP User Manual Example* Some examples of valid source filenames: 1.1 foobar 1.2 /usr/ignatz/src/hello.c 1.3 ubc-medgen::foo 1.4 / src/letter [print] 1.5 sf u-c m pt:: /sr c /dat a [binary] 1.6 :acct/violet.ignatz 1.7 :acct:ignatz 1.8 — the filename of a file that resides on the same computer system as that of the initiator of this request. — another filename of a local file as in 1.1 r- specifies the file named "foo" on the computer system whose network-unique hostname is "ubc-medgen" — specifies the print-ready local file: "/src/letter". — specifies the numeric file: "/src/data" on the computer system whose network-unique hostname is "sfu-cmpt". — specifies the local file: "ignatz" which should be accessible by the account: "acct" whose password is "violet". — specifies the same file as in 1.6, however the password is not specified (and will be prompted for later on by FTP). ubc-vision:bult/money: / ubc/usr/bult / file[text] — specifies the text-file residing on the system whose hostname is Appendix I 77 FTP User Manual "ubc-vision" and the file should be accessible by the account: "bult" whose password is: "money". Destination files are specified like source files except one cannot specify their type. Hence they are referred to as "typeless" files. The type is inferred from the source file. With the above format for specifying a filename in mind, one can specify a transfer command using the following format: Format of a transfer command: <transfer> ::= Transfer <ftpfilename> {To} <destlist> { <optionlist> }. <destlist> ::= <typelessname>, <destlist> | <typelessname>. <optionlist> ::= - <option> <optionlist> | - < option >. <option> ::= Report | NOREport | High | NORMal | Low. (minimum abbreviations capitalized) Examples Some examples of valid transfer commands: 2.1 transfer bar[print] to ubc-medgen::foo Requests FTP to transfer a copy of the local print-ready file "bar" to the file "foo" at ubc-medgen (the file specified in example 1.3). 2.2 tr ubc-vision:bult/money:/ubc/usr/bult/file[text] ubc-medgen::foo Requests FTP to transfer a copy of the file as specified in example 1.8 to the file specified in example 1.3. 2.3 tr ubc-medgen::foo :acct:ignatz,foobar Appendix I 78 FTP User Requests FTP to transfer a copy of the file as specified in example 1.3 to. the files specified in examples 1.7 and 1.1 respectively. (FTP would prompt the initiator of this request for the password associated with the account: "acct". 2.4 tr ubc-medgen::foo :acct:ignatz,foobar - report This is essentially the same request as 2.3 except the initiator requests that he be notified of the success or failure of the transmission. Appendix I 79 FTP User Manual One can initiate a transfer by one of two ways: 1. By issuing the following command: ftp A prompt, "ftp> " will appear and then one can specify a transfer such as those given in example 2. 2. By specifying the source and destination files on the same line as the run command. e.g. entering: ftp ubc-medgen::foo to :acct:ignatz,foobar - report will specifies the same transfer request as given in example 2.4. In either case FTP will display what it thinks has been requested and will then ask for confirmation. For example: Given the following request: ftp ubc-medgen::foo to :acct/pw:ignatz,foobar -rep FTP will respond with: Transfer Request: from: ubc-medgen::foo[text] to :acct:ignatz[text] ::foobar[text] report requested, priority: normal OK?: Appendix I 80 FTP User Manual If this is indeed the proper request one should answer with "yes" or "ok". If it isn't then one should respond with "no". A negative reply will be acknowledged with the message: Request cancelled An affirmative reply will be acknowledged with the following message: request assigned id: xxxxxxxxxxxx where "xxxxxxxxxxxx" is the unique identifier for the request. For example: request assigned id: 281125080001 This identifier will be used by FTP when notifying the requester of the status of his request (if he requested the report option), and can be to obtain the status of a particu-lar file transfer. Appendix I 81 FTP User Manual Notification and Status Regardless of the report option specified in a transfer request, FTP will notify the requester of an unsuccessful transfer. The notification is in the form of a computer mail message. The message one would receive would have as its subject field: Transfer Report The content of the message would have the following format: FTP Transfer no: xxxxxxxxxxxx Unable to transfer <sourcefile> to <destinationfile> Reason: rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr. where: u. xxxxxxxxxxxx rrrrr...etc. is the transfer identifier as explained above, is the reason why the transfer was unsuccessful. For example: FTP Transfer no: 281125080001 Unable to transfer sfu-cmpt:bobo:ledger to :alfo:ledger.b Reason: Unable to access source file. If the "report" option is specified, a successful transfer will cause FTP to send a message to the requester similar to the message given above except, of course, there would be no "Reason:" line. Appendix I 82 FTP User Manual For Example: FTP Transfer no: 281125080001 Successfully transferred sfu-cmpt:bobo:ledger to :alfo:ledger.b One can also explicitly query FTP for the status of a previously submitted transfer request. After issuing the ftp command and when prompted by "ftp> ", one can enter the following command: status xxxxxxxxxxxx where: "xxxxxxxxxxxx" is the transfer id. FTP first checks to see if one is indeed the requester of the specified transfer and then prints out the source filename and the list of successfully delivered, unsuccessfully delivered, and in-progress files. FTP can only give a status report of those outstanding transfer requests that still have some "in-progress" deliveries (i.e. files that have yet to be successfully or unsuccessfully delivered). A status report has the following format: FTP Transfer no: xxxxxxxxxxxx < sourcefilename > to <filel> - <status> <file2> - < status > etc. Where "<status>" is one of the following: Appendix I 83 FTP User Manual in progress delivered <time> undeliverable <time> where "<time>" is the (un)delivery time. If one is not the requester of the transfer with the id, "xxxxxxxxxxxx", FTP would reply with: Access to xxxxxxxxxxxx not allowed. As well, one could specify "status all", which would result in FTP issuing the status of all outstanding transfer requests associated with oneself. Appendix I 84 FTP User Manual Association with EAN Since the underlying communication system for FTP is the EAN Distributed Mes-sage System, one should familiarize oneself of the basic principles of that system before using FTP. In particular, one should study EAN's concept of network addressing. FTP uses the same host names that are used by EAN. One does not have to be a registered user of EAN to use FTP. However if one is a registered user, FTP makes use of the EAN profile, to get the alias list (if any) contained in the profile. The alias list can be used to help in specifying the hostname in a transfer request. When indicating to which host one wants to send a file, one can specify the host associated with a particular EAN recipient. Normally the host associated with a partic-ular file is indicated as follows: hostname:acct/passwd:filename where the host's name is a simple string. Alternatively, instead of explicitly specifying the host's name, one could take advantage of the EAN profile's alias list to implicitly specify the host. For example: If "gerry" were defined in one's EAN profile as an alias representing: <ignatz@sfu-cmpt> Specifying the destination file as follows: alias==gerry :acct / passwd:filename Would be equivalent to specifying: sfu-cmpt:acct/passwd:filename Appendix I 85 FTP User Manual One can also specify source files in the same manner. a=gerry:acct/passwd:filename FTP would assume that the source file "filename" is located at the host computer associated with the recipient associated with the EAN profile alias "gerry". No messages would be sent to the recipient associated with "gerry" regarding the status of the transfer. Appendix I 88 FTP User Manual Some points to ponder. As FTP is implemented right now, it is necessary to concern oneself with access rights. This is the purpose behind specifying the local user computer account and corresponding password of the source and destination files. The computer accounts specified should have the appropriate access to the associated files. That is, the com-puter account associated with a specified source file should be able to at least "read" that source file. The computer account associated with a specified destination file should be able to at least "write" to that destination file if it already exists. If the destination file does not exist, then the computer account should be able to create the specified file at the specified host computer system. If FTP is unsuccessful in accessing files, it will notify the requester appropriately. When "pulling" files (i.e. specifying transfers of files from remote hosts to your own system), one should be aware of the access rights to the destination files. Unless the destination files are universally accessible by anybody the destination files, when specified, should include a computer account/password pair that has appropriate access to the file. This includes files associated with the computer account of the requester. 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items