Knowledge Base

What is EDI

EDI stands for ELECTRONIC DATA INTERCHANGE, and in its most general sense is the direct transfer of information from one computer to another. This is a category that can be stretched to encompass so much that we should start by saying what we mean when we talk about EDI, namely the exchange of structured trade, financial or administrative information between the computer applications of participating partners without direct human intervention. This process covers everything from the physical and electrical connections, the protocols required to handle networks, the methods for transferring information to and from application programs, up to the structuring of the information such that the application programmes can make use of it (this 'hierarchy of compatibility' is often defined in terms of the OSI Model which defines seven separate layers from physical connections up to application processing level). We shall be discussing the highest, application, level

EDI differs from other forms of Electronic Commerce, such as documents sent by email or product ordering via a Web page, because it is intended for application to application data transfer. There is no human involvement.

Why use EDI

Everyday business transactions generate a flood of paper documents; orders, order responses, delivery notes, invoices etc. Although most firms are now computerised and work internally with application systems for accounting, stock handling and process control, their communications with their business partners involve laboriously extracting information from computer databases to produce printed documents and physical transfer of these documents to their partners; who then employ staff to read the documents and key the information into their computer systems! This is an expensive, time-consuming and notoriously error-prone process. The object of EDI is to bypass the printing, posting and re-keying stages by allowing the application systems of business partners to transfer information directly between themselves. This seems self-evident but until fairly recently the wide variety of computer systems and application programmes used meant that such transfers had to be designed from scratch for each business partner and were not cost-effective for anything other than very large transaction volumes. Some progress had been made in standardising the low-level transfer of information, the 'nuts & bolts' of communication, but the central problem of making sense of information transferred from an external system had not been addressed.

Enter EDIFACT

In the mid eighties this problem had become so acute that the United Nations developed a standard which in 1987 was formalised as ISO 9735, EDIFACT (Electronic Data Interchange for Finance, Accountancy, Commerce and Trade). This defined the syntax which could be used for electronic documents and was accompanied by the definition of key documents in terms of their required and allowable information content and sequence, the format and possible content of individual items of information, and extensive codelists which could be used for unambiguous definition of the data items. Together these were known as UN/EDIFACT. Although drawing from earlier standards such as GTDI (used principally in the UK) and ANSI X.12 (prevalent in the USA) this was the first international standard and has gained wide acceptance

Conventional Document

Before we plunge into a description of EDIFACT it might be a good idea to consider what a business document normally looks like:-

 Invoice

MYCO Ltd

Invoice No: A12345

123 High Road

Anyton

Invoice Date: 12 Nov 1993

WESSEX WG7 5DT

Purchaser:

Invoicee (if other than purchaser)

A. Buyer

Vaktrix Ltd

17 Honeydew Close

45 Stonepark Rd

Middleton

Middleton

WESSEX WG8 1DE

WESSEX WG8 5HN

 

Order No:

Sales Contact

Order Date

Terms

ON-17462

Chas Meredith. tel: 0123-443 678

93-11-11

28 days

 

Article No.

Description

Number

Unit Price

Cost £

1935243

Transistor, pnp, ZTX502, 30volt, 2 watts

6

0.27

1.62

1276097

Transistor, npn, BC107 equivalent

5

0.36

1.80

4563251

Capacitor, electr, 500 microfarad

8

0.84

6.72

2309812

Capacitor pack, assorted

1

3.50

3.50

1549875

Resistor pack, preferred values, 0.5 watt

2

1.75

3.50

TOTAL

17.14

 A typical invoice consists of a pre-printed form with three major sections in it; a header which contains information pertinent to the whole invoice, names, addresses, contacts, dates, references etc.,. an invoice body which contains a number of repeating sections describing the items being charged, article number, description, quantity, price, references, comments etc., and finally a trailer which often summarises and totals information from the invoice body.

The form is often printed with 'boxes' on it which contain associated information, such as name and address, since this makes it easier to fill in and to read. Each 'box' will often contain several lines of information, typically with a pre-printed legend to identify what appears on the line. Some lines may be further subdivided; a box for name and address might have a line for Name which was divided into Christian Name and Surname. A box will also have a title which indicates what contents it is supposed to have.

Some information items have to be qualified. If you write down the weight of an article you may need to specify what sort of weight it is (gross, net etc.) and also the units you are using (tonnes, grammes, ounces etc.)

You very often do not need to fill in everything in a form. If you are a normal wage-earner who is making a tax declaration then certain boxes, such as that for your own name and address, have to be filled in but much of the possible information which could be supplied is not relevant and you leave the boxes blank or strike a line through them. Some parts of the form are obligatory (mandatory) but most can be ignored if not relevant (conditional). Sometimes the mandatory / conditional nature of information is more subtle. If you order something and state your own name and address, but leave the box for delivery address blank, then there is no problem; the goods will be delivered to your address. But if you enter just a postcode in the delivery address box then you pose a problem for the seller. By putting something in the delivery address box you have indicated that your own address is not the delivery address, but you haven't specified enough to let him know to which address they should be sent. The information in the form has dependencies; if some lines are filled in then others must be also

Edifact Messages

An EDIFACT message is conceptually very similar to a business document as we have outlined it above. Its overall structure, header - detail - trailer, is the same. It has the equivalent of 'boxes' which are called segments, of lines within a box which are called data elements, and of lines with a number of separate but closely linked items of information on them which are called composite data elements. Many of the segments in a message are conditional, as are many of the data elements within each segment. You get the same sort of dependencies as we discussed above; a conditional segment can be omitted completely but if you give any information at all in it then some data elements (lines) must often be present. The same can hold true for a composite data element. A composite data element for weight might consist of a numeric weight and a qualifier to give the units of weight. You may be able to leave out the weight entirely, specifying '100 kgm' is also acceptable, but specifying just 'kgm' would be an error. Specifying 'ABC kgm' would also be an error; EDIFACT specifies the maximum size and content of data elements. The qualifier will also be subject to some restrictions. EDIFACT will probably specify a codelist which contains the possible values for it. 'kgm' may be the only allowable representation; 'kilos' and 'kg' may not be.

Syntax

People seem to be preoccupied with the EDIFACT syntax. It is obviously necessary to have a way of structuring the information and identifying the structural parts, but unless you are writing an EDI converter yourself it is not really something that you need to spend much time learning when you are planning to use EDI, in just the same way as the electrical signal levels on the telephone wires you use can be taken for granted. Your problems will tend to be on a higher level! Nevertheless we shall give a brief description of the syntax.

EDIFACT information is sent as a stream of characters. There are special dedicated characters to delimit segments, data elements, and component data elements. If these characters appear in the data which you are sending then they are preceded by a release character to say that they should be regarded as data. Segments are identified by a 3 character tag at their start. Within a segment the order of data elements is rigidly defined, as is the order of components within a composite data element, and if a data element or component is absent then its separator must still be present so that the EDI conversion software can 'count' its way through the segment to identify which data element is which. There are rules which allow you to omit the separators when trailing components in a composite, or data elements in a segment, are wholly absent (this is useful since the segments and composites are normally defined on a 'most important first´ basis). Data is sent as ASCII characters, and redundant information is normally truncated (trailing blanks in alphanumeric fields, leading zeroes and trailing decimal zeroes in numeric fields).

The start of an EDIFACT message is defined by a special UNH header segment, and its end is signalled by a UNT trailer segment. These correspond to what you would have in a paper document printed out on continuous stationary; an indication of where each document starts and stops and a title to identify what the document is. There is an EDIFACT equivalent to the envelopes in which paper documents are sent as well: a number of messages to the same trading partner can be sent as an interchange, whose start is signalled by a UNB segment containing address and routing information, and whose end is indicated by a UNZ segment. There is also defined within EDIFACT the notion of a Functional Group - a number of messages of the same type enclosed in the special segments UNG and UNE (Group and End) which acts as a sort of paperclip to hold bunches of documents together. It is scarcely ever used in practice and some EDI converters do not even bother to include code to handle it.

The specimen invoice we looked at earlier would look something like this if it were processed to EDIFACT form through a converter:-

UNH+MSGRF1+INVOIC:90:1:ZZ:EDMTRX'
BGM+380+A12345'
NAD+SE+++MYCO LTD+123 HIGH RD+ANYTON+WESSEX+WG7 5DT'
CTA+SR+:CHAS. MEREDITH+0123-443 678:TE'
NAD+BY+++A. BUYER+17 HONEYDEW CLOSE+MIDDLETON+WESSEX+WG8 1DE'
NAD+IV+++VAKTRIX LTD+45 STONEPARK RD+MIDDLETON+WESSEX+WG8 5HN'
PAT+09++931222:014'
UNS+D'
LIN+1++1935423:SA++:6:PCS+0.27'
IMD+ZZZ+++TRANSISTOR,PNP,ZTX502,30VOLT,:2 WATTS'
LIN+2++1276097:SA++:5:PCS+0.36'
IMD+F+++TRANSISTOR,NPN,BC107 EQUIVALENT'
LIN+3++4563251:SA++:8:PCS+0.84'
IMD+F+++CAPACITOR,ELECTROLYTIC,500 MICRO:FARAD,25VOLT'
LIN+4++2309812:SA++:1:PCS+3.50'
IMD+F+++CAPACITOR PACK,ASSORTED'
LIN+5++1549875:SA++:2:PCS+1.75'
IMD+F+++RESISTOR PACK,PREFERRED VALUES,:0.5WATT'
UNS+S'
TMA+17.14+++17.14'
UNT+21+MSGRF1'

A real invoice, both as printed form and EDIFACT message, would be more complex since our example has completely omitted any VAT related information.

Application - EDIFACT Coupling

EDI is about direct application to application data communication. EDIFACT provides us with the structure for such communication, but you are still left with the problem of how to transform between EDIFACT messages and data in your application, which will typically be contained in a database. You also need to consider how the files containing interchanges (logical envelopes containing EDIFACT messages) will be transmitted and received. The process can be represented like this:-

                                       

  The converter which transforms to and from EDIFACT lies at the centre, and transfers data as files to and from interfaces to your application system databases, and to the network communication software.

Converters

An EDIFACT converter is a program which can transform information between EDIFACT format and an in-house file format more suitable for handling by an application. Because of the wide variety of EDIFACT messages and in-house file formats which may have to be dealt with such converters are normally table-driven; the formats of the incoming and outgoing files are defined in a file which is read by the converter and controls its subsequent processing logic. The creation of such tables is normally a specialised job and users are often supplied with the tables they need as part of their converter purchase.

Since EDIFACT messages are intended for direct transfer of information between applications an important part of a converters function is to identify and react to errors. This is a complex task since many errors are heavily dependent on the context of other information in a message, and is one of the major factors which distinguishes a good converter from an indifferent one.

Other factors which can influence the choice of converter (apart from the obvious one of availability for your hardware platform!) are the ability to handle standards other than EDIFACT, the in-house file formats supported, their logging facilities, and their ability to operate in a complex EDI environment without additional supporting software (see Gateways below). If you intend to make considerable use of EDI and wish to develop and maintain the messages used yourself then you should investigate the table maintenance tools available.

Application/ EDI Interface

The application interface is the technique by which your application can create or read in-house files.The most common intermediate (in-house) file format is called a tagged flat file and consists of records identified by fixed length tags with defined field positions on each record for data items. In such a file the information is presented in roughly the same sequence as in the EDIFACT message, which in its turn has the same sort of sequence as an equivalent paper document. Exporting information to such files is normally fairly easy since most applications contain a report generator, and the tagged flat file is an example of such a report.

Another common intermediate file format is in fact a series of related files, which correspond to the tables of a database. The information in business documents, and EDIFACT interchanges, is in the form of nested repeatable groups of information and is readily transformable to a number of files, one for every repeatable group, with a record for each repetition. Where the number of repetitions possible is limited it is common to 'stretch' the data into the record for the nesting level below it in order to reduce the number of files created and the consequent burden of handling them. The records in each file are linked by key fields to corresponding records in other files. Both CSV (Comma Separated) and SDF (fixed field positions within a record) formats are possible for the individual records.

If your applications supports the import / export of CSV or SDF files then this is the best solution for you, otherwise the tagged flat file is probably marginally easier to deal with. If your application does not support file import directly then it normally takes a programmer conversant with your application 2-5 days to produce an import program. The ideal EDI converter would be one which could access your database directly. Unfortunately because of the wide variety of databases used such converters are rare and, when they do exist, are expensive. The wisdom of allowing direct modification of your database by data from an external application is also questionable

Network Interface

You can conduct EDI by transferring interchanges between machines on magnetic media, but most users opt for transfer-by-wire in some form for speed and convenience. This can be done by point to point connection (dial-up, leased line, packet switching) or via a store and forward network service. In the latter case the connection to the VAN (Value Added Network) is similar to that which you might use for point to point communication but each partner can choose the times when it is convenient for them to access the network, and value-added services in the form of security, logging, integrity checking, archiving etc. are often available.

The choice of communication method is normally made on the basis of the existing communications links which your partners have. Most networks now offer interworking so that it is possible to communicate from one network with partners on another, but this can add to the costs and may, in a few cases, still suffer from teething troubles, although by now (1997) this is unlikely. Whichever communication method you choose you will almost certainly need a software interface to transfer files to and from the communications link. An EDI aware interface will be capable of examining files which you submit to it to find UNB interchange header segments, splitting out interchanges and sending them to the relevant partner / mailbox / link based on the addressing information it finds there. It will similarly be able to analyse incoming files and perform a data-splitting operation on a partner and possibly message basis to ease the logistics of subsequent conversion. For a more conventional interface you will need to make sure that you run your application data extraction and EDI conversion in such a way that interchanges for a particular partner are created in one or more discrete and identifiable files. You will also have to arrange your receipt of incoming traffic to ensure that EDI and other communications are kept separate, and that different EDI messages are not bundled together in a way which is beyond the capabilities of your conversion and application import software.

Gateways

The combination of application interface, converter, and network interface works well when you are only making limited use of EDI. As you start to use many different messages, communicate with many partners, and connect to several networks there are demands placed on your system which require a more sophisticated approach. You begin to need scheduling, management and maintenance functions, logging and reporting, error recovery, verifications of receipt and delivery, and archiving. A system of this sort is commonly referred to as an EDI Gateway. For an EDI pilot project it could be overkill, the complication of learning and managing the system could outweigh its benefits, but for extensive EDI use, particularly if many messages and networks were involved, you would end up either building it or buying it.

Interchange Agreements

In a golden future the use of EDI may become so widespread, and the transactions conducted through it so standardised, that you could send and receive EDI messages in the same way that you use a fax today. Until then you will need to agree with your trading partners the what, when, and how of EDI transfers. You do this by making an Interchange Agreement with them. Standard forms of interchange agreement are available from trade associations and the UK Electronic Commerce Association. They cover the following topics:-

  •  Code of Conduct agreement
  •  Data standards and messages to be exchanged
  •  Data element values and interpretation
  •  Processing cycles
  •  Security and data storage requirements
  •  Fallback arrangements
  •  Terms and Conditions of business (using EDI).
  •  Contact functions and personnel
  •  Legal requirements
Message Handbooks

Items 2 and 3 from the list above are normally dealt with in a technical addendum to the interchange agreement, and take the form of a Message Handbook for each EDI message to be exchanged. This defines the layout of the EDI message, the segments and elements which are mandatory, required, recommended, allowed or unused, and the explanation of how the contents of each element is to be used and interpreted. This part of a Message Handbook is called a Message Implementation Guide (MIG, in the USA often referred to as an IC or IG). Each partner to an interchange agreement normally complements the MIG with an explanation at data element level of how each field will appear in their in-house file(s) and how their applications should handle it.

The Message Handbook is a significant document for the technical implementation of EDI. EDIMatrix's rule of thumb, based on a great deal of practical experience, is that without one we quote five times the man-hours which would otherwise be required! In many cases newcomers to EDI can gain access to an existing Message Handbook for the EDI messages to be exchanged, either from a trading partner who is experienced with EDI or through a trade association. This is to be strongly recommended.

You need a high level of detail in the handbook. Seemingly trivial things like the anticipated number of decimal places can cause trouble if not carefully defined there. There can be inconsistencies between the applications of trading partners which cause trouble - a partner may send you a 14 character reference which should be quoted in subsequent transactions and your database may only have room for 12! Such things are best dealt with at an early stage instead of by impromptu and undocumented 'fixes' as problems occur in production, and the steps taken to deal with them should be recorded in the handbook.

Is EDI something for Us?

Two of the most common reasons for the introduction of an EDI capability have little to do with the conventional justifications given to motivate it. You may either wish to please some significant customers, or you may be dependent on one who has placed the commercial equivalent of a gun to your head (which has the dubious benefit of simplifying the decision making process considerably!). Apart from the obvious savings in time and effort when the print / post / re-key cycle is removed the major benefits of EDI are greater reliability and faster response. Most of the other cited benefits of EDI stem from these. With greater reliability the considerable amount of time normally spent by staff in identifying and correcting erroneous and incomplete information is markedly reduced. This gives the possibility to redeploy staff to more productive activities or to reduce overall staffing levels. Faster response gives competitive advantages, the possibility for just-in-time working with consequent reduction in stock levels (and the capital tied up in them), and a significantly higher service level which many customers consider more important than the immediate cost of products

Cost for EDI

These are dependent on your ambition level and your hardware platform. A simple PC order receipt station connected to one of the cheaper networks will cost about £300 for software and a year's network subscription. Such a station is normally used to receive from a trading partner who has already defined the messages which will be used so that the development costs are minimal.

At the other end of the scale a mainframe Gateway for which you develop many messages yourself and make considerable use of EDI for your trading communication will cost tens of thousands of pounds. A PC based equivalent will cost about one-fifth of the price, and is increasingly popular in a mainframe environment where PC's are often used as terminals in any case.

Pilot Project

Since the full-scale introduction of EDI is a strategic decision with significant consequences for the internal operations of a company, most choose to test the technology by running a pilot project with one or two willing trading partners. This is a low-cost, low-risk way of gaining experience. It is strongly recommended that one person is made responsible for the project, and that those who will be involved in it receive some formal training in EDI as soon as possible. Experience has shown that pilot projects run more smoothly if the significant personnel involved have some knowledge of EDI. An effective alternative is to use knowledgeable consultants but when they then leave their experience goes with them!

Use of existing messages and documentation is desirable. The project will take less time and stand a better chance of becoming operational without significant problems if you do not have to re-invent the wheel. You should start with a single message and partner if possible. Although all the topics listed under 'Interchange Agreements' above need to be considered , the project will be of limited scope and the legal and fallback requirements may not be too exacting.

You should aim to begin EDI transmissions in parallel with your existing paper document transfers. Initially the application should still be using the conventional methods. When you are satisfied that the EDI transmissions are functioning satisfactorily the application can be switched to use them. After a further period of dual working without problems the conventional document can be phased out.

EDI and the Internet

There is a lot of discussion about using the Internet to send and receive EDI. The advantages are low cost and excellent connectivity. The disadvantages tend to concern security and reliability. If you are considering this option then you might like to bear the following points in mind.

If your proposed Trading Partners currently use VAN-based EDI you will be under pressure to follow the same route. Although most major VANs can interwork successfully, gating to and from the Internet is not currently common. A VAN which has X400 connectivity will probably be technically capable of Internet gating, but finding someone at the VAN who knows how to set it up may not be easy.

Are security and reliability really issues for you? Given that most non-EDI business transactions involving getting your most junior employee to put documents in thin paper containers (envelopes) and drop them through a slot into a metal box on the street, the sudden concern about security when the transactions become electronic can seem a little strange. If you genuinely expect fast response to your EDI transactions, you need confirmation of delivery/receipt, or the information is sensitive then communication via a VAN would be better. Remember that most of the time the Internet works very well.

FTP is used increasingly to transport EDI, both on a point-to-point basis and as an access mechanism to a conventional VAN. Secure FTP (TLS or SSH) is also implemented occasionally. The EDI-INT initiative has resulted in three standards for Internet Transfer; AS1 (email), AS2 (web) and AS3 (FTP) which address the perceived problems with security and accountability. AS2 is the most common since it was adopted by Walmart in the USA, but in the UK none of them are widely used.. The prevalence of permanent Internet connection via broadband has tended to increase the use of direct transfer between Trading Partners.

XML

In the last decade there has been considerable development effort devoted to the use of XML as a substitute for the conventional EDI syntaxes such as EDIFACT. Initiatives such as ebXML and RosettaNet are the best known examples. Whilst XML has advantages for hybrid EDI, and is a more general technology than conventional EDI, it does not appear to have been adopted with any great enthusiasm. The majority of new trading relationships still use the 'traditional' standards.