@prefix vivo: . @prefix edm: . @prefix ns0: . @prefix dcterms: . @prefix skos: . vivo:departmentOrSchool "Science, Faculty of"@en, "Computer Science, Department of"@en ; edm:dataProvider "DSpace"@en ; ns0:degreeCampus "UBCV"@en ; dcterms:creator "Sadowski, Edward Richard"@en ; dcterms:issued "2010-05-16T05:01:37Z"@en, "1984"@en ; vivo:relatedDegree "Master of Science - MSc"@en ; ns0:degreeGrantor "University of British Columbia"@en ; dcterms:description """Most existing File Transfer Systems are connection-based, where the actual transfer of a file immediately follows the transfer's specification. Few systems allow Three-Party Transfers, where a person on one computer can request the transfer of a file residing on a second computer to a third computer. A Store-and-Forward File Transfer System separates the specification and transmission phases of a file transfer. A Store-and-Forward File Transfer System has been designed and implemented using as its medium of transmission a Computer Based Message System based on the current CCITT proposal. This File Transfer System allows genuine Three-Party Transfers. Results indicate that such a system is both feasible and practical."""@en ; edm:aggregatedCHO "https://circle.library.ubc.ca/rest/handle/2429/24762?expand=metadata"@en ; skos:note "C l T H E EFFICACY OF A STORE-AND-FORWARD FILE TRANSFER SYSTEM by EDWARD RICHARD SADOWSKI B.Sc. The University of British Columbia, 1 9 8 1 . A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in T H E F A C U L T Y OF GRADUATE STUDIES DEPARTMENT OF COMPUTER SCIENCE We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA April, 1 9 8 4 ©Edward R. Sadowski, 1 9 8 4 In presenting t h i s thesis i n p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree at the University of B r i t i s h Columbia, I agree that the Library s h a l l make i t f r e e l y a v a i l a b l e for reference and study. I further agree that permission f o r extensive copying of t h i s thesis fo r s c h o l a r l y purposes may be granted by the head of my department or by h i s or her representatives. I t i s understood that copying or pub l i c a t i o n of t h i s t h e s i s fo r f i n a n c i a l gain s h a l l not be allowed without my written permission. Department of Cft^^ X -p r -^C \\ t?iA C^, The University of B r i t i s h Columbia 1956 Main Mall Vancouver, Canada V6T 1Y3 Date ^~Tyf'\\ Abstract Most existing File Transfer Systems are connection-based, where the actual transfer of a file immediately follows the transfer's specification. Few systems allow Three-Party Transfers, where a person on one computer can request the transfer of a file residing on a second computer to a third computer. A Store-and-Forward File Transfer System separates the specification and transmis-sion phases of a file transfer. A Store-and-Forward File Transfer System has been designed and implemented using as its medium of transmission a Computer Based Mes-sage System based on the current CCITT proposal. This File Transfer System allows genuine Three-Party Transfers. Results indicate that such a system is both feasible and practical. ii Table of Contents Chapter 1 - Introduction . 1 Chapter 2 - A Survey . 4 2.1 Network Architecture 4 2.2 FTP Architectures 4 2.2.1 The Virtual Filestore 7 2.3 Actions 11 2.3.1 Checkpointing and Error-Recovery 16 2.4 Implementation Architecture 17 2.5 The CCITT/ISO Message Handling System Model 18 2.5.1 Overview 18 2.5.2 Physical Mapping 20 2.5.3 Naming and Addressing 20 2.5.4 Layers Representing the MHS Model 22 2.5.5 The Message Transfer Layer 23 2.8 Store-and-Forward File Transfer 25 Chapter 3 - A Store-and-Forward File Transfer System 26 3.1 Motivation 26 3.2 Design Issues 27 3.3 Design and Implementation Outline 27 3.3.1 EAN and The CCITT CBMS Model 27 3.3.2 Unique Design Features 28 3.3.2.1 Three Party Transfers 28 3.3.2.2 Multiple Recipients 29 3.3.2.3 Reduction in Code Size 29 3.3.2.4 Elimination of Some Concerns 29 3.4 General Design 30 3.4.1 Functional Organization 30 3.4.2 Data Flow 32 3.4.3 Control Flow 33 3.4.4 Peer Protocol 34 Chapter 4 - The Implementation 39 111 iv 4.1 The User Interface 39 4.2 Virtual Files 40 4.3 Transfer Identifiers 42 4.4 Building of Protocol Data Units 43 4.5 Use of the UA - SDE Interface of EAN in EANFTP 44 4.5.1 UAL Interface Primitives 46 4.6 A Walk-through a File Transfer 48 4.7 Keeping Track of Transfers in Progress .'. 52 4.8 Forwarding of Receipt Notifications 58 Chapter 5 - Results and Evaluation 57 5.1 Results 57 5.1.1 Two Party Transfers 57 5.1.1.1 Pulling a File 57 5.1.1.2 Pushing a File 59 5.1.2 Three Party Transfers 60 5.1.3 Multiple Recipients 61 5.1.4 When Things Go Wrong 62 5.1.5 Status of Transfers in Progress 64 5.2 Evaluations 64 Chapter 6 - Conclusions 68 6.1 Further Improvements 68 6.1.1 Management Functions 68 6.1.2 The Virtual Filestore 69 6.1.3 Automatically Initiated Jobs 70 6.1.4 Cancellation of In-progress Transfers 70 6.1.5 Bundling of Source Files 70 6.1.6 Encryption 70 6.1.7 PartiaJ Transfers 71 6.2 An Interesting Application 71 References 72 Appendix I - FTP User Manual 73 List of Figures 2.1 - ISO OSI Network Architecture Layers 5 2.2 - File Transfer Entitles 6 2.3 - Two Host Configurations 7 2.4 - Virtual Filestore Representation of a Hierarchical File 11 2.5 - Interaction of Service Primitives 13 2.8 - Sender Active File Transfer 14 2.7 - Receiver Active File Transfer 15 2.8 - Functional View of the MHS Model 19 2.9 - Combinations of Co-resident and Stand-alone UAs and MTAs 21 2.10 - Layered Description of the Message Handling System 23 3.1 - Functional Organization of EANFTP 31 3.2 - Data Flow In EANFTP 32 3.3 - Layered Description of the File Transfer System 34 3.4 - EANFTP F T A Protocol Data Units (CCITT X.409 syntax) 36 3.5 - Logical Structure of the EANFTP SendRequest PDU 37 4.1 - Syntax of FTAUEP request 41 4.2 - Transfer Identifiers In EANFTP 43 4.3 - Definition In C of ENODE - the PDU node 44 4.4 - Layered Configuration of EANFTP 45 4.5 - Portion of CCITT definition of EM - UAPDU 46 4.6 - UAL Primitives State Transition Diagram 49 4.7 - Relationship Between PFPDUs and Agents in EANFTP 53 4.8 - Logical Structure of the Transaction Data Base 55 5.1 - Connection Asymmetry 65 v CHAPTER 1 Introduction Lars Posrsena of Clusium By the Nine Gods he swore That the great house of Tarquin Should suffer wrong no more. By the Nine Gods he swore it, And named a trysting day, And bade his messengers ride forth East and west and south and north, To summon his array. Lays of Ancient Rome Thomas Babington, Lord Macaulay The ability to transfer data files from one computer system to another is one of the most obvious applications of computer networking. This ability is so desirous that it had virtually spurred the inception of computer networks. In the infancy of computer communication there were many incompatible networks and each network had its own file transfer protocol which in most cases was incompati-ble with the protocol of any other network. From the mid-1970s to present an effort has been undertaken to standardize both the networks and their application programs (such as file transfer) in an attempt to achieve universal accessibility or Open Systems Inter-connect (OSI). The International Standards Organization (ISO) has sponsored such a standardiza-tion. Its current proposed standard for file transfer stipulates that it should be connection-based. That is, even before a file transfer can be specified, the file transfer program must set up a network connection between the source and destination comput-ers, whereupon the transfer will take place immediately. The ISO standard makes no provision for a Store-and-Forward (SAF) file transfer. That is, a transfer that takes Chapter 1 1 Introduction Chapter 1 2 Introduction place some time after its specification. Potential advantages of SAF file transfer are: 1. The potential better use of network resources by, for instance, the possible bundling of requests before a network connection is established between the source and destination computers thereby minimizing the overhead of estab-lishing such connections. Savings could be realized by establishing the con-nections during favourable rate periods or during periods of less network congestion. 2. Its \"off-line\" nature makes it easier to address such concerns as synchronization and error-recovery. 3. Common elements of other SAF applications such as Computer Based Message Systems (CBMS) can be used in the design and implementation. ISO has been in the process of standardizing CBMSs and it would be desirable for a SAF file transfer program to take advantage of aspects of the design of the ISO CBMS. This thesis will deal with the design, implementation, and evaluation of a Store-and-Forward File Transfer Program. This thesis is organized in the following manner: Chapter 2 is a survey of concepts and previous work in File Transfer Sys-tems. It also deals with an overview of the Comit'e Conaultatif Internationale de Telegraphique et T'elephonique (CCITT) Computer Based Message Sys-tem model on which the SAF File Transfer System, implemented as part of this thesis, is based. Chapter 3 presents the objectives in designing and implementing a SAF File Transfer System (SAFFTS). It also presents the general design of EANFTP, Chapter 1 3 Introduction the implementation of a SAFFTS. Chapter 4 describes the implementation of EANFTP, paying close attention to the information passed between entities in the system. Chapter 5 gives some examples of some actual file transfers accomplished on EANFTP. The chapter also assesses EANFTP's success in achieving the goals stated in Chapter 3. Chapter 6 gives general conclusions of the above enterprise as well as present-ing areas of needed further research and work. Appendix A is the User Manual for EANFTP. CHAPTER 2 A Survey 2.1. Network Architecture Before the advent of networking, the most reliable way of transferring a file was to copy the file to magnetic tape or some other physical medium and physically transport it to the destination system. Even this most rudimentary service required the parties of a transfer to agree on matters of format. They had to agree, or at least be aware of, the density of the recording, the character set used, the blocking factor, etc. of the tape. One can imagine the number of issues that must be addressed in order to realize this transfer on a computer network. In part, to make this task (along with other network-ing applications) more tractable, the International Standards Organization (ISO) devised a layered architecture model. In this model, services and their related protocols common to all applications, characterized by a common function, are grouped together in a hierarchical manner. They are organized as in Figure 2.1. Readers unfamiliar with the above model should see [ISO-80]. Since transferring a file over a network is an application, one might intuitively guess that the majority of a File Transfer Protocol (FTP) would reside in the applica-tion layer. However there are aspects that are usually associated with the presentation and session layers. These aspects will be discussed later in this chapter. 2.2. FTP Architectures A seminal approach to handling network file transfers was through the medium of an emulated interactive terminal. This is known as the Virtual Terminal Approach Chapter 2 4 Survey Chapter 2 5 Survey application presentation session transport network data link physical Figure 2.1 - ISO OSI Network Architecture Layers (VTA) (see [TELE-73]). A universally understood model of a terminal function is chosen. The receiver would have the identical characteristics of physical terminal and the sender of the file would \"login\" and \"enter\" the file through this terminal. Draw-backs in this approach include: - the inability to modify a transfer already in progress - the need to \"login\" to the remote system and thus requiring operational knowledge of the remote computer system, accounts, and passwords - inability to specify partial transfers - inability to specify services particular to or singularly helpful for a file transfer The shortcomings of VTP prompted the design and implementation of a separate service expressly designed for file transfer. Chapter 2 6 Survey An FTP can be split into two parts: the Control Service and its corresponding Pro-tocol (CSP) and the Transfer Service and its corresponding Protocol (TSP). The CSP is responsible for the initiation, specification, and supervision of a particular file transfer. The TSP is responsible for the actual transfer of a file. In its most general configuration a file transfer involves at least three parties (or hosts): the controlling or initiating host -who specifies and coordinates the transfer; the sending host - the host where the source file resides; one or more receiving hosts - the host(s) where the file is to be sent. The relationships between the hosts and the protocols are shown in Figure 2.2. To make the coordination and control of the transfer easier it is assumed in many FTPs that the controlling host and one of the sending or receiving hosts are one and the same. With this configuration the host containing the CSP is known as the active host and the other known as the passive host. Configurations are shown in Figure 2.3. These configurations are restricted to either \"pulling\" a file (a file from a remote source is transferred to a local system) or \"pushing\" a file (a file from the local system is transferred to a remote system). Figure 2.2 - File Transfer Entities Chapter 2 7 Survey active host passive host passive host active host control control ® data or data Figure 2.3 - Two Host Configurations In most FTPs, the CSPs and TSPs utilize the same connection between the sending and receiving systems. The CSP initiates a connection between the two hosts and nego-tiates the file transfer. Once this is accomplished the TSP uses the same connection to transfer the file. The two phases can overlap: the CSP synchronizing and interrogating an in-progress transmission. This approach is known as a connection-oriented FTP. The ARPA.FTP [NEIG-73] utilizes two connections at the same time: one for the CSP and the other for the TSP. Alternatively, the CSP and the TSP can be treated as two distinct phases each requiring its own connection. This method does not require the actual transfer to take place immediately. Instead the request is queued for later delivery thus allowing the person who had specified the request to do other tasks. This approach is known as a transaction-oriented or Store-and-Forward File Transfer (SAFFT). 2.2.1. The Virtual Filestore The architecture and organization of files on various systems are legion. Byte-size, character sets, handedness, etc. all contribute to a veritable Babel of file types. Similar to a system wide definition of a virtual terminal protocol, so too do most FTPs stipulate a virtual terminal filestore. A filestore is a logical collection of physical files. A virtual filestore is a universally known and understandable abstraction of any particular Chapter 2 8 Survey physical filestore. The mapping of the actual file to its virtual file representation - a presentation layer task - is a local matter and is not a concern in the definition of the FTP. What is important is that once this mapping is done, it appears as a properly formed virtual file to the FTP. To describe a virtual file, a number of attributes are associated with it. In the ISO proposal [ISO-82], these attributes can be classified as either: file attributes - a set of which describes and identifies a particular virtual file or activity attributes - a set of which describes and identifies a particular file transfer This and any other concrete FTP example in this section, unless otherwise stated, are from the ISO FTP model. The file attributes include: Filename: This is essential to any FTP. The \"filename attribute is usually the physical filename prefaced with the host's Service Access Point (SAP) address. A SAP is a universally known or univer-sally obtainable address. It is unique to the domain of the partic-ular network. All requests for a particular service are channeled through its SAP. The ISO FTP uses a session layer SAP. The combination of the SAP and the local filename is enough to uniquely identify a file throughout a network. The FTPs do not specify any syntax for mapping a local filename into its virtual filename attribute other than stipulating that it be prefaces by its Chapter 2 9 Survey SAP. Therefore it is up to the parties involved in a particular transfer to agree on its representation. The filename attribute may or may not reflect the same structure as its local operating system filename syntax. This is an area of further research. Access Control Lists & Passwords: One of the disadvantages of the virtual terminal approach (at least in some of its manifestations) is that it requires one of the transfer parties to \"log-in\" to the other's computer. This necessi-tates the knowledge of passwords. By keeping an explicit access list, the FTP can ascertain, without the need of log-ins, whether the particular parties are allowed their desired transfer. For secu-rity purposes, the access lists could contain passwords associated with the virtual file rather than with an account. Account: Indicates against whom the filestore charges should be charged. This could include the cost of transferring this file anywhere. Date and Time of Creation, Last Modification, Last Access, Identity of Creator, Last Modifier, Last Accesser: Various biographical data of the file. File Availability: Indicates whether any delay could be expected before the file can be opened. This attribute can be used to avoid deadlock - the attribute could be interrogated and if a delay is indicated the transfer could be cancelled. A delay could also warrant the use of a SAFFTP. Chapter 2 10 Survey Device Type: The frequency of checkpoints (whose function will be described below) could be influenced by the volatility of the device of the virtual file. For example, a file to be transferred residing on disk would require fewer checkpoints than a file to be read in from a card reader as the disk is less prone to malfunction. Data Types: A file can contain mixed data: some character, some binary, different character sets. In the Autodin II [BBN-80] and ECMA [ECMA-80] FTPs, mixed data types are not allowed, limiting the utility of those FTPs. Encryption Algorithm Name: The algorithm used to encrypt the file. Current and Future File Sizes: If the file is very large it might be incapable of being transferred as a unit. Before the file could be transferred it would have to be disassembled into components small enough to be transferred. These sub-files would then be transmitted and collected at the destination, where the components would be reassembled. Structure Type: Files may be organized in many ways; they could be \"unstruc-tured\", \"flat\", \"hierarchical\", or some other structure. This diversity must be represented in some standard way. The ISO FTP provides a syntax for describing hierarchical files (such as Indexed Sequential Access Method (ISAM) files). The file is Chapter 2 11 Survey represented as an oriented tree. A node of such a tree can be given an unique (to the file) identifier. The subtree rooted by this node can be uniquely identified and accessed. The root of the entire file will, of course, be labelled with the virtual filename attribute. The structure is shown in Figure 2.4. No FTP has yet been designed that has a similar abstraction for multi-keyed files or files of non-hierarchical structure. Activity attributes include: T y p e of access: \"Read\", \"Write\", \"Management\". Management access allows the filestore attributes to be changed. 2.3. Actions By means of a virtual filestore an FTP should be able to perform a number of operations on a file such as: select (to locate, interrogate, but not open), open or close, read, write, append, update, create, and destroy. root node (entire file) block node block node record node record node ... record node Figure 2.4 - Virtual Filestore Representation of a Hierarchical File Chapter 2 12 Survey As part of the selection and specifications of a file transfer, the FTP should have the ability to read, negotiate, and modify file and activity attributes. This is known as management access. ISO has not yet designed a mechanism for negotiating the parame-ters of an attribute. The parties can either agree or disagree with the proposed change of an attribute. NIFTP [NPL-81] however has a more elaborate negotiating scheme. In this scheme, each party has a capability and a requirement for each attribute. A capa-bility is the upper bound, the requirement the lower bound. Updating - the ability to replace a part of a file - can be relatively easily realized through ISO's (or similar) hierarchical tree representation outlined above. A connection-based file transfer follows this characteristic sequence: 1) application connection: the source and destination FTPs are connected via their respective SAPs. 2) file selection: the particular files are specified and located. 3) file opening: 4) data transfer: actual transfer of file data. 5) file closing: the connection to the file is still kept but any read/write locks are released. 6) file deselect: lose knowledge of the file. 7) application disconnection: the connection is closed. Between steps 1 and 2 the FTP can give management information such as file directories of a particular filestore. Between steps 2 and 3 the FTP can access the attri-butes of a particular file. Any interrogation of a file transfer in progress takes place dur-ing step 4. Chapter 2 13 Survey The two parties privy to transfer communicate through the intermediary of the File Transfer Service Agent (FTSA). The FTSA is charged with the routing, establish-ment, and forwarding of communication of the transfer parties. Most such communica-tion between FTSAs follow the pattern shown in Figure 2.5. The actual transfer of file data is itself communication - a typical transfer might follow that shown in Figures 2.6 and 2.7. Notice how, in Figures 2.6 and 2.7, the actual file data communication departs from the usual Request - Indication - Response -Confirmation (RIRC) dialogue shown in Figure 2.5. Only requests and indications are used. This is an efficiency concession. Notice the slight asymmetry (*) in the flow between when the sender or the receiver is the active component (Figure 2.6 vs. Figure 2.7). The file data might have to be \"massaged\" before transmission. That is, the file might have to be encrypted or compressed. These desirable facilities would normally be placed in the presentation layer. The ISO layer model maps out a logical demarcation of functions. The functions, services, and applications particular to a layer do not neces-sarily have to physically reflect this characterization. In an FTP, for example, the code for the desirable presentation services could well be with the code for the application ser-Figure 2.5 - Interaction of Service Primitives Chapter 2 14 Survey W i t h t h e s e n d e r a s t h e a c t i v e c o m p o n e n t : s e n d e r ( a c t i v e ) c o n n e c t r e q u e s t -c o n n e c t c o n f i r m s e l e c t r e q . — . o p e n c o n f . < < -w r i t e r e q . . w r i t e c o n f . < £ -d a t a r e q -d a t a r e q . d a t a r e q . e n d - f i l e r e q . e n d - f i l e c o n f . c l o s e r e q . c l o s e c o n f . d e s e l e c t r e q . d e s e l e c t c o n f . ^ — d i s c o n n e c t r e q . . d i s c o n n e c t c o n f . F T S s e l e c t c o n f . < ^ • o p e n r e q . ; j s , 4r r e c e i v e r ( p a s s i v e ) - ^ - c o n n e c t i n d i c a t i o n c o n n e c t r e s p o n s e — 7 - s e l e c t i n d . ^ - s e l e c t r e s p . o p e n i n d . o p e n r e s p . ^ w r i t e i n d . / w r i t e r e s p . — > d a t a i n d . r > d a t a i n d . -9> d a t a i n d . - > e n d - f i l e i n d . / — e n d - f i l e r e s p . c l o s e i n d . — c l o s e r e s p . d e s e l e c t i n d . d e s e l e c t r e s p . ( t i m e ) \" ^ d i s c o n n e c t i n d . d i s c o n n e c t r e s p . F i g u r e 2.6 - S e n d e r A c t i v e F i l e T r a n s f e r Chapter 2 15 Survey With the receiver as the active component: receiver (active) connect request FTS connect confirm <^— select req. select conf. open req. open conf. read req. read conf.. <~~ data ind. ^ data ind. <£.—-data ind.*^ end-file ind.^~ end-file resp end-read ind.-close req. - i s , close conf.^— deselect req. _ deselect conf disconnect req. ^ disconnect conf.^-sender (passive) —^ > connect indication connect response select ind. • select resp. ^> open ind. open resp. read ind. — read resp. - data req. . data req. data req. - end-file req. V (time) -—> end-file conf. — end-read req. (*) ^ close ind. 11 UAE II SDE 1*: 7**rRys:-e; * S2 SI Figure 2.10 - Layered Description of the Message Handling System [CCIT400-83] ing model. The Submission and Delivery Entity (SDE) is the means by which a UAE in a Sl-type system can access the services of the MTL. The SDE acts as the interface t between the UAE and its corresponding MTAE. Px are the various peer protocols. PI is the MTAE-MTAE protocol. P3 is the SDE-MTAE protocol. Pc is the UAE-UAE protocol and will reflect the functionality of the UA. 2.5.5. The Message Transfer Layer The MTL provides the UAs the necessary services by which they can send mes-sages. A criterion in the design of the MTL is that it be both general and functional enough to support a variety of customized UAs. The definition allows UAs that might not even be concerned with the handling of personal mail messages per se, as will be Chapter 2 24 Survey shown in this thesis. The services the MTL provide are characterized by a number of primitives. These primitives can be grouped into the following categories: 1. Those primitives dealing with the initiation and termination of a dialogue between a UA and the MTL. 2. Those primitives dealing with the parameters maintained by the MTL that deal with that particular UA. 3. Those primitives dealing with the temporary specification of the nature of the types of messages that the UA will accept from the MTL. 4. Those primitives dealing with the submission of messages from a UA to its MTA. 5. Those primitives dealing with the determination of the delivery of a message. 6. Those primitives dealing with the actual delivery of the message from the recipient's MTA to his UA. <» 7. Those primitives dealing with the notification of the inability to deliver a mes-sage. 8. Those primitives dealing with the explicitly requested notification of delivery of messages. 9. Those primitives dealing with the cancellation of messages that had been queued for deferred delivery. It is upon these set of service primitives that a UA maps its functionality. Chapter 2 . 2 5 Survey 2.6. Store-and-Forward File Transfer As mentioned above, it may not be necessary to have the transfer of a file take place immediately. In a Store-and-Forward File Transfer Service (SAFFTS), the initia-tor of a transfer need only specify the transfer, leaving him free to do other tasks since the actual transmission of the file would take place on another connection at a (though not necessarily) different time. This is analogous to a CBMS2. Would it be possible to use a CBMS to realize an SAFFTS? In this case the \"message\" would be a file (of potentially enormous size). This thesis will address the design, implementation, and cri-ticism of an SAFFTS. Realizing the similarity between a CBMS and an SAFFTS, we will try to determine the common elements of both applications. ^ o r an example of an implementation of a C C I T T CBMS see [NEUF-83]). CHAPTER 3 A Store-and-Forward File Transfer System As was mentioned in Chapter 2, there has been little work done in the area of Store-and-Forward File Transfer Systems (SAFFTS). This thesis will attempt to design and implement such an FTP (herein to be referred to as EANFTP) and will address the efficacy of the undertaking. 3.1. Motivation Why have a Store-and-Forward File Transfer System? One might suppose, at first glance, that it is a step backwards. After all, since a SAFFTS is not connection-oriented, transfers do not take place in \"real time\". Historically, most applications have striven to provide real-time service. However a SAFFTS is especially suitable for the fol-lowing: specifying file transfers from machines with no instantaneous access to the specified files the distribution of files to multiple destinations the transmission of very large files over slow, unreliable media - thus FTP is useful for those people who do not want a terminal monopolized by such a transmission third party transfers (where both the source and destination files are on remote computer systems) transfer of files where transmission costs are a major concern broadcasting - e.g. the distribution of new releases of a document or a pro-gram to several subscribing destinations. Chapter 3 26 A SAFFTS Chapter 3 27 A SAFFTS 3.2. Design Issues A designer and implementor of an SAFFTS must address some major issues: 1. The functionality of the system: What capabilities will the SAFFTS have? Will they differ substantially from those of a connection-oriented FTS (COFTSf. By its very nature, an SAFFTS will not necessarily be able to access immedi-ately a remote computer system. This must influence the capability of the FTS. In particular how will this inability affect the management requests such as remote directory listings commonly allowed in a COFTS? 2. The underlying medium of transmission: Will the SAFFTS have its own session and presentation layers? Alternatively, the session, presentation, and even some of the application layers of other networking applications might be util-ized. 3. Application overlap: If the underlying medium of transmission is going to be from part of another networking application then from which applications should it be? How can the SAFFTS enhance its functionality by utilizing the capabilities of the applications from which it is derived. 3.3. Design and Implementation Outline 3.3.1. E A N and The C C I T T CBMS Model This thesis will describe the approach we have chosen. EANFTP will use as its medium of transmission a Computer Baaed Message System (CBMS). In particular, EANFTP will be using the EAN CBMS [NEUF-83]. EAN is an implementation, developed at the University of British Columbia, of the CCITT CBMS model At this Chapter 3 28 A S A F F T S stage it is capable of operating on machines supporting Berkeley UNIX1 version 4.2 as well as on DEC VAXs under the VMS operating system. Because the desirability of Computer Based Messaging is manifest, most computer systems already have or will have in the near future a CBMS. Since EAN adheres to the CCITT CBMS model (in the model's current incarnation) it should be compatible with systems supporting the ISO-OSI networking model. Ideally the support for the ISO model will/in the future, become almost universal. It is to be hoped then, that CCITT CBMSs will become pervasive. Those systems that do support a CCITT CBMS will then be capable of implementing EANFTP. This makes the choice of a CCITT CBMS as a transmission medium both sound and attractive. EANFTP will make the most use out of the Message Transfer Layer (MTL) of EAN. (Refer to the description of the CCITT model in Chapter 2). The MTL encom-passes the session and presentation layers of the ISO-OSI model. 3.3.2. Unique Design Features How will the utilization of the MTL of a CBMS influence the design of the FTP? The following are some features made possible and easier. 3.3.2.1. Three Party Transfers As outlined in Chapter 2 (Figure 2.2), in its most general form, a file transfer involves three parties: the controlling or initiating host the sending or source host the receiving or destination host(s) 1 UNIX is a Trademark of Bell Laboratories. Chapter 3 29 A SAFFTS It was also mentioned that the synchronization of these three parties in an FTP presented a non-trivial task. So much so that many FTPs require a Two Host Configuration (See Figure 2.3). This thesis will investigate the ability of EANFTP to synchronize a generalized third party transfer. The problem of synchronization in a COFTS is in maintaining three concurrent network connections. In an SAFFTS, not being connection-oriented, these connections do not have to be concurrent, thus eliminating much of the synchronization problem. 3.3.2.2. Multiple Recipients One of the requirements of a CBMS is that it be able to deliver a message to a list of recipients. It will be shown in this thesis how this requirement will be utilized by EANFTP. The ability to specify multiple recipients in a COFTS is either rare or unk-nown. Any synchronization problems posed by a Three Party Transfer is only com-pounded by multiple recipients. The MTS will automatically \"bundle\" some messages. That is, if a source file is destined for two different FTAs that share the same MTA then the MTS sends only one copy of the source file to that destination MTA. 3.3.2.3. Reduction in Code Size The MTAE need not be co-resident with its associated UAEs (as shown in Figure 2.9). It is possible to configure EANFTP correspondingly, resulting in programs of smaller size which might be important for small machines. 3.3.2.4. Elimination of Some Concerns By taking advantage of the MTL, EANFTP will not have to concern itself with issues such as: 1. Checkpointing: as this is handled in the session layer of the MTL Chapter 3 30 A SAFFTS 2. Accounting: files will essentially be sent as messages and therefore the charges that should accrue can be handled by the accounting system (if any) of the MTL 3. Managing its own address space: EANFTP will use the name space of the CBMS on which it is based. The MTA notifies its UAs of any invalid addresses 4. Sequencing of requests: the MTA delivers messages to the UA one at time and therefore acts as a sequencer for the UA 5. Connection failures: the MTL is responsible for the reestablishment of lost con-nections and for the retransmission of effected messages. 6. Priorities of Service: High, Low, and Normal priorities are supported and managed by the MTL. 3.4. General Design 3.4.1. Functional Organization EANFTP is organized as shown in Figure 3.1. The CCITT CMBS Model has two major entities: the Message Transfer Agent Entity (MTAE) and the User Agent Entity (UAE). EANFTP will make use of its own specialized UAEs that deal almost exclusively with files. To these UAEs, \"messages\" will be properly couched files. There are two types of UAEs: the entity corresponding to the File Transfer User Agent (FTUA) which is the agent with whom the user interacts. That is, the FTUA Entity (FTUAE) is the entity responsible for the proper formulation of a file transfer request which is then relayed via its MTAE to the File Transfer Agent Entity (FTAE). The FTA is responsible for the supervision of the file's transmission via the MTS to the Chapter 3 31 A S A F F T S Figure 3.1 - Functional Organization of EANFTP file's destination(s). This supervision entails the sending and receiving of the file as well as the handling of delivery confirmation which is received either from other FTAs or, as in the case of an invalid host name, from the FTA's associated MTA. Similar to the configurations shown in Figure 2.9, EANFTP's file transfer agents can either be co-resident with its associated MTA or stand-alone (where an MTA can. be associated with file transfer agents from other systems). Chapter 3 32 A SAFFTS 3.4.2. Data Flow A transfer of a file in EANFTP follows the steps as shown in Figure 3.2. 1. A user, on some initiating host machine, specifies, through an FTUA, that a file be sent from some particular source host machine to some other host machine. (The user is allowed to specify more than one recipient machine for the file). Figure 3.2 - Data Flow in EANFTP Chapter 3 33 A S A F F T S 2. This request, once properly formulated by the FTUA, is sent via the associated MTA and the MTS to the FTA at the source host machine. 3. The FTA sends a copy of the file via FTA's associated MTA and the MTS to the FTA of the recipient host machine. 4. This FTA deposits the file in the requested location and then sends a reply via the MTS to the FTA of the source host. 5. The FTA of the source host relays this acknowledgement back to the FTA of the initiating host machine. 6. If the user had requested delivery notification then the FTA of the initiating machine would send a computer mail message, through the FTA's associated MTA and the local UA, to the user. 3.4.3. Control Flow There are explicit boundaries of \"responsibility\" in the FTS made necessary by its Store-and-Forward nature. As the file goes from one layer of the system to the next and from one machine to the next, responsibility for the message (in this case a file) has to be taken by the entity who possesses it. An entity is \"responsible\" for a message when it takes \"possession\" of the message and will ensure that it will get delivered to its destina-tion (be it another entity or another computer) and will retransmit the message if neces-sary and possible. Any secondary storage needed to store copies of the message is super-vised exclusively by the responsible entity. In EANFTP, control flow can be characterized by the following: A user issues a request for a file transfer through his FTUA. Once this request is properly formulated (this corresponds to step 1 in the Data Flow example above) the user is free to do other tasks. He is no longer responsible for the request. During step 2, responsibility is taken Chapter 3 34 A SAFFTS first by the MTA of the FTUA and then by the FTA on the source host. Similarly in step 3, responsibility passes from FTA of the source host to the MTA of the source host to the FTA of the destination host. The path of responsibility for the acknowledgement (steps 4 and 5) is the converse of steps 2 and 3. 3.4.4. Peer Protocol When properly configured, the peer protocol used by the UAs (Protocol Pc as shown in Figure 2.10) should be totally transparent to the MTL. Even though the MTL was ostensibly designed for personal mail, the transparency of the Pc protocol makes it possible for other applications, with their own peer protocols, to be placed on top of the MTL. So instead of the configuration shown in Figure 2.10, EANFTP will be configured as in Figure 3.3. \"PF\" is the FTA - FTA peer protocol. UAL MTL S3 S2 SI Figure 3.3 - Layered Description of. the File Transfer System Chapter 3 35 A SA F F T S The FTAs communicate in -what CCITT describes as Protocol Data Units (PDU). The CCITT has gone into considerable detail in specifying the complexion of these PDUs [CCIT409-83] q.v. EANFTP's PDUs are shown in Figure 3.4. A TransferRequest corresponds to what is sent as described in step 2 of the Data Flow description above. A SendRequest is what is sent in step 3. TransferReply and SendReply are the PDUs sent in steps 4 and 5 respectively. A FTPDU has structure. For example, a SendRequest contains, among other things, a SendHead and the File. The SendHead itself has structure. It has a sequence of AgentRecipient which in turn has structure. A PDU then, has a tree structure. A field corresponds to a leaf node of the PDU tree. The structure of an EANFTP SendRe-quest is shown in Figure 3.5 Transmission lines, can only transfer a stream of bytes (or octets). Before the PDU can be sent it has to be \"flattened\". When this \"flattened\" PDU is received it can then be built-up again. Each node of the PDU tree stores the number of octets needed to represent the node itself and its subtree when both it and its subtree are \"flattened\". These numbers are used by the receiving FTAs to rebuild the previously \"flattened\" trees. The CCITT in its PDU syntax specification stipulated that fields of a PDU are identified by code numbers rather than by included strings such as \"TO\" or \"FROM\" which are found in other protocols. This encoding scheme is suitable for applications which might be implemented on machines of countries with various different native languages. Each node of the PDU tree also stores the code number identifying what type of node it is. Each field is identified by a code number. Most of these code numbers are unique only to the subtree data unit. That is, the code numbers are con-text sensitive. This corresponds to the adjective IMPLICIT seen in Figure 3.4. Their Chapter 3 36 A SAFFTS MODULE FTAPDU ::= CHOICE { [0] IMPLICIT TransferRequest, [lj IMPLICIT SendRequest, TransferReply [2] IMPLICIT ReceiptNotiflcation, SendReply [3] IMPLICIT ReceiptNotiflcation } TransferRequest : — SET { [0] IMPLICIT SendHead, pageRanges [l] IMPLICIT SEQUENCE OF INTEGER OPTIONAL, mtaEventID [2) IMPLICIT STRING } SendRequest : — SET { [0] IMPLICIT SendHead, title [1] IMPLICIT STRING OPTIONAL, encrypted [2] IMPLICIT INTEGER { } OPTIONAL, File } SendHead ::= SET { TransferEventldentlfler, source [0] IMPLICIT ORName, - source FTA agentrecepients [l] IMPLICIT SEQUENCE OF AgentRedpient, - recipient FTAs Initiator^] IMPLICIT ORName, - initiator UA originatlngHost [3] IMPLICIT ORName, - initiator FTA NotiflcattonRequest [4| IMPLICIT INTEGER { report(0), noreport(l) } OPTIONAL, - defaults to noreport importanceOfContent [5] INTEGER { nonUrgent(0), normaI(l), urgent(2)} OPTIONAL, - defaults to normal sourceFOename [6] IMPLICIT Filename, sourceAcct |7] IMPLICIT STRING OPTIONAL, sourcePassword (8) IMPLICIT STRING OPTIONAL, ffleType [9] IMPLICIT INTEGER OPTIONAL { text(0), print(l), binary(2)} - defaults to text } AgentRecipient ::= SET { ftagent [0] IMPLICIT STRING, redpHst [l] IMPLICIT SEQUENCE OF Recipient, redpTlme [2] IMPLICIT STRING OPTIONAL} - time flies were delivered — to this ft agent Recipient : — SET { recipient [0] IMPLICIT ORName, dest Filename [1] IMPLICIT Filename, destAcct [2] IMPLICIT STRING OPTIONAL, destPassword [3] IMPLICIT STRING OPTIONAL, rejectReason [9] IMPLICIT INTEGER OPTIONAL { b*dSourceFile(l), badDestFUe(2), destFileNoAccess(3), destHostNoAccess(4), sourceHostNoAccess(6) } } Receipt Notification ::= SET { TransferEventldentlfler, destagent [ 0 ] IMPLICIT STRING, notiflcationTime [l] IMPLICIT STRING, notiflcationRecipient [2] ORName, deliveryLiflt [3] IMPLICIT SEQUENCE OF Recipient OPTIONAL, undeliverabfeList [4j IMPLICIT SEQUENCE OF Recipient OPTIONAL } — Application-defined TransferEventldentlfler ::= STRING - to be more structured later — current format: :