This article sets out to show the reasons for, problems related to, and solutions for successful data interchange.
Data interchange can be described as moving information between one computer and another. Depending on types of computer, size of data, confidentially of data, and location of each computer system there are many possible methods of implementing a data interchange process. There are also three basic layers of interchange that have to be considered; media, file system, and data structure.
The labels are used to define the tape and also the files. They also have a built in mechanism for handling multi-volume tapes.
There are many reasons why data is moved from one system to another but the end result is so that information that starts on one computer can be accessed, processed or just viewed on another system, anywhere. A very simple example of data interchange can be walking across the office with a floppy disk containing a word processing file. In this example the data size will be small, and most likely the computer at each end will be a PC running the same word processing package. The same result could be achieved in most offices by use of the internal network.
Many data interchange scenarios are much more complex where very large volumes of data on different types of computer and operating systems reside in different locations. These large volumes of media, file systems and data structures all have to handled in the most efficient manner.
Media Conversion
Media conversion is handling different magnetic or opto-magnetic media. With high-speed communications, it is now possible to send large files via Internet, and many people have seen this as the death of tape. But interestingly as communications get faster so do file sizes that need to be transferred, and with magnetic tape now able to store 100GB of data, it is still in many cases the best way to transfer large volumes of data. However, there are many types of tape drives, from 4mm DATs to DLT and IBM LTOs. To interchange data, it is essential to have a compatible drive that must read the relevant variation of tape that may be provided by the supplier of data. For instance on the IBM 3590 there 128 track and 256 track versions.
Format Conversion
Format conversion is required if each computer has a different operating system. A Unix system often uses different file systems to a DOS system or an IBM mainframe. The way that files are stored on tapes can vary enormously due to both operating system and application. Even on a Windows based PC, tapes could be written with ArcServe, NT Backup, Legato and many more. As a rule, each tape writing method is incompatible with others.
There are a few IBM interchange formats, but these are largely unlabelled tapes, and labelled tapes are not recognized by most PCs and mid range systems. All other nominal interchange formats are defacto standards, such as the NT backup (MTF) and others such as Unix CPIO and TAR. For Vax systems, the VAX VMS Backup would count as a standard. All other formats can be considered as unique, and to make matters worse, updates on a format are not always backward compatible for more than one or two versions. In order to interchange tapes, it is essential to have both a compatible tape drive and a way of reading a tape structure that may not be supported on the receiving platform. At this stage, attention has to be paid to file naming and subdirectory structures. Unix is different from DOS and for a few files the issue can be overcome by hand, but for a large database structure with many hundreds of files, care has to be taken possibly on both sides of the interchange path.
Once you overcome the media and format interchange problems, then files can be moved from one system to another, but…..
Data Conversion
Different operating systems have different application software, and each application normally has its own internal way of saving data. There are some standards such as CSV files for databases and RTF files for word processing text, however, these are few and far between and often only save the basic information rather than the full structure. Other significant areas of incompatibility come from mainframe type applications that use EBCDIC and packed numbers to keep their data. Data conversion problems can arise with any interchange system, and although networks and the Internet can mask many media and format interchange problems, data incompatibility can still remain.
The Solutions from eMag
Fortunately, for each of the above functions, eMag has the right solution for your company. If you are looking for a media, format or data conversion service then check out our US or European Conversion Service Centers.
Or if you are looking for software to handle your conversion needs then we offer MediaMerge/PC. MM/PC will interface to all standard SCSI tape drives and many optical drives so that a single application can interface to the relevant hardware. Apart from being able to duplicate one media onto a different type of media, the main function of MM/PC is to handle all tape format conversions. MM/PC has a growing library of over 200 different logical tape formats, and each time it will read a tape and produce real files that can be used on the final system in the same way as native files. However, the contents of the files are not changed except for optionally between ASCII and EBCDIC. MM/PC will therefore solve many media and format interchange issues. Visit the MediaMerge/PC product page today to learn more.
For data conversion, the main tool that eMag has is the Record Reformatter. This will process structured files and enable different outputs to be created. The most common application is to convert packed numbers, typical on mainframe type applications. There is also a range of data conversions and several numeric conversion tools built in.
Over the past 19 years eMag’s InterMedia division has built up a large range of other data conversion tools that are used within the eMag Service Centers. For relevant applications, these tools can be packaged into a complete solution for solving a very large range of problems. The final result is the ability to take information from any computer system, to any other system, as easily as possible. Whether it is 1MB or multiple TBs, eMag is happy to advise and assist.
Recent Comments