File systems on UNIX, Windows and z/OS

This post is an extremely long answer to the question “Why do I have to say LINE SEQUENTIAL in my Cobol SELECT statements?”.

The target audience are programmers who are used to the Windows and UNIX file systems, but does not know anything about the filesystems on IBM’s mainframe operating system z/OS.

The UNIX and Windows file systems

All files are byte-oriented. The smallest unit that can be read or written is one byte. There is no additional structure to files; a program can read or write any number of bytes at any offset in the file.

The OS or an application may impose a structure on a file, but the file system does not know about it. And what it does not know, it can not enforce.

The native z/OS file systems

The OS that runs on IBM mainframes is called z/OS, a rebranding of earlier OS/390. It has its own file system that is very different from what UNIX and Windows have. It does not seem to have a name, but I will call it z/FS.

Do not confuse z/FS with ZFS. ZFS is an IBM implementation of the UNIX file system. It is available on several platforms, including Linux.

On z/FS (N.B. not z/OS) almost all files, including text files, are record-oriented. Files either has a fixed record size, or a variable record size. The smallest unit that can be read or written is one record. It is not possible to read or write a part of a record.

In this text, all z/FS files are assumed to be record-structured.

z/OS also provides a UNIX file system, called HFS (Hierarchical File System). HFS behaves just as any other UNIX file system.

What is a text file?

The phrase text file is not a well defined technical term; it is more of a convention. By text file we usually mean:

  • A file that contains mainly printable characters.
  • A file that is divided into lines.
  • A file that can be edited in a text editor.

If a character is printable or not depends on the character set used. On UNIX and Windows the character set is either the ASCII character set, or some character set derived from ASCII. On z/OS it is some variant of the EBCDIC character set.

On UNIX and Windows file systems, text files are stored as sequences of bytes. There is a line terminator after each line. The line terminator is one or two bytes. The byte value of the line terminator depends on both the file system type and the host character set.

On z/FS, the native z/OS file system, text files are stored as records. Each records is one line of text. There are no line terminators.

OS File system Char. set Line term. Comment
Windows FAT, NTFS ASCII 0x0D 0x0A Carriage return fllowed by line feed.
UNIX UNIX ASCII 0x0A Line feed, ASCII encoding.
z/OS z/FS EBCDIC none Assuming record-oriented files.
z/OS UNIX EBCDIC 0x15 Line feed, EBCDIC encoding.

As programmers we must be aware of the target file system. On UNIX file systems we should provide and expect LF at the end of lines. On Windows file systems we should provide and expect CR/LF at the end of lines. And on z/FS we should neither provide nor expect a line terminator.

How to do it in Cobol

This is where we use SELECT ... LINE SEQUENTIAL. It tells Cobol that the file is on a byte-oriented file system with line terminators. The Cobol runtime will then do the right thing.

Without LINE SEQUENTIAL line terminators are neither expected nor provided. This works well with record-oriented files on z/FS. But on byte-oriented files output will be one long line. If the Cobol record is 77 bytes, then exactly 77 bytes will be written, and exactly 77 bytes will be read.

Here is a summary of writing in different cases. Reading works in a similar way.

UNIX Text mode write. The whole records is written, including trailing spaces. A line feed character is appended. The byte value of the newline character depends of the characterset, see table above. Binary mode write. The whole record is written, but no line feed is appended.
z/OS + HFS
z/OS + z/FS This is not allowed, and will result in a runtime error. Record oriented binary write. One record is written.

It is unfortunate that z/FS does not simply ignore LINE SEQUENTIAL.

Final notes.

Saying that a record-oriented z/FS file can only be read and written record-by-record is not strictly true. It is possible to trick z/OS into reading such a file as a byte stream. One way is to write a C program that a) opens the file in binary mode, and b) runs on the UNIX subsystem.

Yes, there is a complete UNIX subsystem in z/OS that can compile and run many UNIX program. It is possible to ssh to the UNIX subsystem for shell access. Even Emacs and other open source software is available, but not installed by default.

The UNIX subsystem is a set of z/OS routines that implements UNIX system calls. With the system calls in place, libraries can be ported, as well as programs. This is similar to Cygwin, the UNIX emulation layer for Windows, except that here it is part of the OS.

You can reach me by email at “lars dash 7 dot sdu dot se” or by telephone +46 705 189090

View source for the content of this page.