Print

FTP-based EDI vs. Web Service

Abstract

Web services technologies are being developed as the foundation of a new generation of B2B applications.  One of the direct impacts is its replacement of the traditional FTP-based EDI systems in the retail industry.  While the FTP-based system supplies an adequate functionality base for the development of the new Web Services system, the Web services approach intends to provide a better solution for data exchange between businesses.   This paper illustrates the design and functionalities of two real-world systems using these two approaches and discusses the differences between them in terms of order information storage, data transmission, system extensibility, and security.

Introduction

A growing number of companies are creating B2B applications to streamline their supply chain operations [Gilber, 2003, Kane, 2004].  One of the major retailers in the U.S. (hereafter called eMart) has also required its suppliers to develop software to exchange daily flow of purchase orders, invoices, and shipping notices.  Currently, eMart is using an FTP-based EDI approach for the data exchange process but is moving toward a better solution known as Web services.  The introduction of XML Web services to existing data intensive applications allows for an API interface between unknown clients and the server to send end receive required data. The transparent nature of the XML files serialized though Web services transmissions allows for any platform on any operating system to accept and parse the requested data. 

This paper first illustrates the design and functionalities of eMart systems using the two approaches.  Security implementations for the Web services approach are further explained.  At the end, the paper discusses the differences between the two approaches and the important issues related to order information storage, data transmission, system extensibility, and security.

EDI-based FTP Approach

The current order tracking and processing system, utilized by eMart for its major vendors, is comprised of an FTP-based EDI system using XML formatted documents to relay specific orders and associated consumer information to a desired vendor.

Using a console shown in Figure 1, the system is engaged during an order from the eMart e-commerce Website which formats the consumers user information, order number, etc. along with each items SKU number, description, quantity, etc. from the sale into a parent-child XML arrangement.  The entire consumer information content comprises the first parent section with each additional section specific to each item being purchased.  Once order payment has been completed the generated XML file is saved using the eMart order number, similar in format as 123456.XML, and transferred, via FTP, to the vendors specific folder on the eMart FTP server.
At this segment, the system moves onward to the point of vendor interaction.  In most cases, each vendor has an established backend business system to manage inventory and process specific orders as they are acquired.  Integrating the eMart order with the vendors backend occurs with the use of a custom consol application built specifically for the eMart vendors.  With the application, each vendor is able to login to the eMart FTP server and retrieve all orders placed within the vendors specific folder during a specified time span defined by the last time stamp of the final order processed into the vendors database.  As the XML files are retrieved, child fields are parsed into their respective columns and inserted as a new order into the vendors system.  Immediately following the parse of the order file, a confirmation file in XML format is sent from the XML application to the eMart FTP server to allow eMart to update the order status and notify the customer the order has been successfully processed.

Each vendor then processes the order and ships the items to the respective eMart customers.  Once items have been shipped, the vendor must once again use the custom application and send a second confirmation file in XML format denoting that the item is successfully shipped, which carrier was used and the tracking information for the package. 

f1
Figure 1. The FTP-based EDI System Console

f2
Figure 2. The FTP-based EDI System

This information is placed on the eMart Website for the consumer to view. To ensure the eMart server has received this information, a confirmation XML receipt is returned to the vendor application.

Additional functionality to the vendor application allows for each vendor to update the current online catalog by submitting a properly formatted XML document to the eMart FTP server.  Subsequent to parsing the catalog file and updating the item index, an XML confirmation file is returned to the vendors application.  The complete system is illustrated in Figure 2.

Web Service Approach

Microsofts .NET technology was employed to create eMarts XML Web services. Using the services, customer orders stored in an SQL database server can be retrieved through SQL query requests and converted into files in XML format.  Potential user programmers can then review the service description and the message structure defined in WDSL and use the specifications to create a proxy to request the services. 

Vendor are required to build a custom client application to bridge the gap between eMarts file formats and the ability to consume the data into some types of backend system.  For those with high end, recently built systems, integrating the Web Services result sets will require less development time and ultimately result in lower cost to facilitate the synchronization.  For other vendors whose systems could not consume XML formats, smaller packages, such as Microsoft Excel, are able to connect to Web Services and manipulate the data directly.  Keeping each vendor on the same page allows eMart to reduce subsequent costs in conforming to multiple standards.   

As illustrated in Figure 3, eMarts ordering system begins with a customer order being placed from eMarts Website.  Once order payment process is concluded, the order information is appended into eMarts order database via a Web services request. During the pulling process from the vendors client application, an additional Web service is triggered to carry the date and time stamp from the last entry in the vendors database to the eMart customer order database.  Once a request from a vendor has been initiated (see the system console in Figure 4), a Web method is called through a client proxy, transferring with it the latest date, time and the vendors identification number as parameters to the eMarts Web services server.  Once the eMart server receives the call and the parameters, the Web service processes the requested data from the attached order database using in-line SQL statements and returns a .NET structured dataset containing all unprocessed orders for that specific vendor.  The data is then serialized and sent back to the vendor via the SOAP protocol over the HTTP.  Once all XML data has been delivered, the Vendors client application de-serializes the data back into the dataset format and parses it into the vendors backend system.  After the vendor commences the process of filling the orders and shipping the purchased items to the consumer, the client proxy is called again to update the order status with shipment information and confirmation to eMart.
At this point in the process, the vendor commences the process of filling the orders and shipping the purchased items to the consumer.  Immediately after the shipment process has been completed, the vendor completes the cycle employing the application one last time to trigger another Web Service updating the order status with shipment information and receiving a confirmation in the Web Services resulting XML.


f3
Figure 3. The Web Services System

f4
Figure 4. The Web Services System Console

As it relates to updating vendor catalogs with new items or removing outdated or out of stock items, the Web services system also include the same functionalities.  The vendors client application provides a user-friendly front end to allow vendors to browse their current online catalog, update item information, add new items or remove items completely. These Web services methods also allow for seamless integration with eMart backend operations without compromising access to the internal file system or other vendors information.

The inherited problem with XML Web services is the exposure of sensitive data to unauthorized users on the networks, especially those with malicious intent.  The resolution to secure Web services can be achieved through various methods with various forms of additional programming and costs associated.  Two of the most commonly used methods are the basic SSL authentication and the custom SOAP Header authentication.

Basic SSL Authentication

The least cost scenario that provides some encryption support to protect the XML Web services data is the basic authentication over SSL. SSL certificates can be purchased for as low as $30 per year, depending on the level of warranty protection and verification information required, and are relatively simple to install.

To begin the process (see Figure 5), the vendors client application makes a request through the proxy.  The request is encrypted using a SSL encryption key to prevent users with malicious intent from reading and duplicating the request and obtaining any included user/windows credentials.  Prior to the verification process completed by eMarts Web services server, an eMart firewall validates the requesting IP address. By allowing only specific IP addresses through the firewall, invalid requests by unauthorized users are rejected and only users that have been established as order processors may gain access.  Once the data is ready for transmission, depending on the desired result data type, the dataset is serialized into an XML formatted file and re-encrypted using 128-bit SSL encryption from the servers certificate authority, and sent back through the firewall to the authorized requesting vendor. The final process, completed by the vendors client application, is to decrypt the incoming xml document, de-serialize the XML format back into the desired data type and display the data is any method that is required.

Using this method, an order transmitted from a processing location to the order database is only secure via the secure socket layer protocol while its being transmitted between the two locations. With the firewall blocking unauthorized inbound requests, only those from legitimate locations are processed. Any transfer of data once behind the Web services server is, most likely, unencrypted and capable of being compromised by internal users. Encryption levels on the serialized data can be anywhere between 40 to 128 bits depending on the certificate purchased and the users operating system. At the lowest level of encryption, a hacker would be able to compromise the transferring order information in a few minutes. Likewise, at the 128 bit encryption level, which is about 300 septillion times stronger than the minimal 40 bit, it would require over one trillion years to compromise the secure communication.

f5
Figure 5. The Basic SSL Authentication

Custom SOAP Authentication

One of the most sophisticated security methods for Web services requires the use of custom SOAP headers passed by the client and authenticated by the Web services server. In doing so, custom authentication methods can be created and used outside of the standard HTTP request. Unlike the Body element of the XML Web services, the custom header, located in the Header element of the Web services, is processed outside of the normal Web services requests via custom authentication methods.  Although additional development is also required on both the client applications and the Web services server, the complexity of the authentication along with the added SSL for ensured protection is an unbeatable combination for data that can not be compromised under any circumstance.

f6
Figure 6. The Custom SOAP Authentication

As shown in Figure 6, the process begins with the vendor client encrypting the users credentials, which include a username and password combination, into a base64 encoded string using either AES or TripleDES found within the .net cryptographic classes. This type of encryption uses a shared key and initialization vector that must be identical on both the client and the server to be encrypted and decrypted properly. Doing so creates another level of security on the credentials to be sent. Once encrypted the vendor clients credentials are applied to the Web request in the SOAP headers element of the XML Web request for order data and sent through the Internet to the eMarts Web services server over SSL. As mentioned previously with basic authentication over SSL, eMarts firewall is established to allow only specific IP address through which keeps unauthorized requests from reaching the Web services server. Once through the firewall the Web request is received by the eMarts Web services server where the SOAP headers are immediately processed prior to the actual Web request. Since the username and password credentials are encoded, the Web services server begins the process of decrypting, using the shared key and vector, on the base64 encoded string back to comparable strings for authentication. A Boolean value is generated based on the authentication and sent back to the Web services server. If a true value was obtained, the Web services server continues to process the Web request, execute any SQL commands, package the data from the data source, and return the order data in a dataset to the Web services server. The dataset is then the serialized into an XML document, encrypted using the SSL certificate authority and returned to the vendor client for processing and display. Should the authorization value been false, eMarts Web service server will deny the request and return an error report to the vendor client as a notification.

Utilizing this method of encryption within this custom authentication method not only ensures an authorized user from an authorized vendor client is initiating a request, but that all credentials and information passed between vendors and eMart are secure, or at least extremely unlikely to be compromised by malicious intervention. The most beneficial asset of custom authentication is the separation of header credentials from the body of the Web request. While keeping the requesting and transmitted data under SSL, the Web services headers can house any form of encrypted user/password credentials, regardless of the type of encryption. Those headers can be processed completely separate from the Web request, securing the contents of the request and most importantly the data that would have been sent back, should the request been fraudulent.

Discussions

The current FTP-based system supplies an adequate functionality base for eMart and its vendors to develop the new Web Services system.  However, there are some important differences between the two approaches. The following section discusses the differences and other important issues.

Order information storage

Order information storage, as it is currently warehoused, utilizes the operating systems file structure to create file folders for each vendor capable of being accessed via FTP. Order information is uploaded and downloaded using username and password authentication from either eMart or a vendor. In comparison, the Web Services system employs a Web server running IIS and an attached database for a much more organized compilation of customers, their orders and vendor statuses for each order. By moving away from file system organization, all orders, vendor statuses and shipping information can be contained in one location rather than multiple files, and with the use of SQL, complex reporting and detailed order history can be accessed on demand instead of browsing through multiple order documents. 

Data Transmission

When analyzing the two systems, data transmission method stands as the most prominent difference between them. The current FTP-based system simply copies order information from one computer to another, creating excessive redundancy and ultimately requiring greater system resources. On the contrary, Web Services creates API methods to retrieve order information from the eEMart database. These methods create XML formatted documents that can be displayed or consumed into each vendors own backend system. Order information can then be updated and maintained just as fast by initiating other methods to transmit content back to eMart.

System Extensibility

System extensibility, for many large companies, is a major requirement of new technology that is to be implemented to replace a functional system. With the current FTP-based system, updating data format or warehousing requires a substantial modification of each vendors custom application as well as changes to the XML file format schema. Since the Web Services methods create the XML formatted files on demand, schema changes can be made on the eMart Web server and the changes are immediately available to each vendor.  Also, if a Web Service method is modified to add or remove parameters or result sets, only minor changes to the vendor application must be made.

Security

Almost every corporation that engages in Internet sales and data transmission keeps the security of their data as a high priority and under close supervision. The current system, other than plain text authorization to the FTP server, does not prohibit unauthorized access to the order information during transfer from server to server.  This is, by specification, all the FTP protocol offers in terms of security.  For the most part, the Web Services specification does not offer much more benefit. However, security can be added to Web Services HTTP layer for data transfer and the application layer for internal kernel protection. For all e-commerce systems, the use of SSL technology is a requirement for consumers to even ponder purchasing from the Web. Likewise, SSL offers 128 bit encrypted data transfer from eMart to vendors and vice versa. Additionally, since new Web Services systems can be developed through the collaboration between eMart and its vendors, agreed encryption algorithms can be integrated allowing supplementary security on top of SSL.

Conclusion

While the current FTP-based system supplies an adequate functionality base for the development of the new Web Services system, the Web services approach intends to provide a better solution for data exchange between businesses.  Through illustration, this paper found that traditional FTP-based EDI approach is in disadvantage when issues such as order information storage, data transmission, system extensibility, and security are essential considerations to system design and development.  However, the systems being studied are running on Microsoft's .NET platform.  Further study is needed to compare the two approaches in different computer platforms.

The overall security of data passed through the Internet will remain a constant battle between those who wish to continue the theft of data not intended for their use and the organizations that wish to keep the data private.  However, the security concern has not deterred the industry from developing Web services applications.  eMart has taken the its step to take advantages of the Web services technologies.  They will certainly have to rigorously continue searching for better security methods to keep their data exchange on the Internet safer.

References

A. Gilbert (2003). Wal-Mart Project Boon for Software Makers, CNET News.com, August 14,

M. Kane (2002). Amazon, Google Lead New Path to Web Services, CNET News.com, November 20.