Release Notes RSX-11M[+]/IAS Software Tools VOS Fall 1983 DECUS Distribution Joseph Sventek Computer Science & Mathematics Department Lawrence Berkeley Laboratory Berkeley, CA 94720 This document describes the RSX/IAS implementation of the Software Tools Virtual Operating System. For those new to this game, the basic principles behind the software are described in the article "A Virtual Operating System" which appeared in the September 1980 issue of the Communications of the ACM. The contents of this release supercede all previous releases. NOTICE This software is provided on an as-is basis. No guarantee of performance or support is stated or implied. Any errors or omissions in the code or documentation are regrettable, but not unusual considering the man-power and mode of distribution. Written notification of bugs WITH fixes are appreciated and will be incorporated into the next release, if possible. IAS Notice The extra source files needed to bring this distribution up on IAS (version 3.1) are included in [307,37]. Consult the file [307,37]iasreadme.1st for more details. -1- Release Notes Currently available tools Acat - concatenate nested archive entries on standard output Admin - administer TCS file. Alist - generate paginated listing of source archive Ar - archive file maintainer Args - use standard input as arguments for command Asam - generate index for archive file Asplit - salvage garbaged archive files Axref - cross reference symbols in archive files Banner - generate large banner lines BarGraph - draw a 0-100% bargraph of integer data Box - draw boxes around block structure of RatFor or C programs Cat - concatenate and print text files Ccnt - character count Cd - change working directory Ch - make changes in text files Chd - change working disk and directory from MCR Cmp - compare two files Comm - print lines common to two files Cpress - compress input files Crt - copy files to terminal a screen at a time Crypt - crypt and decrypt standard input Date - print the date Dc - desk calculator Delta - make an TCS delta Detab - convert tabs to spaces Diff - isolate differences between files Echo - echo command line arguments Ed - line-oriented text editor Entab - convert spaces to tabs and spaces Expand - uncompress input files Fb - search blocks of lines for text patterns Fc - fortran compiler Fd - fast directory list in sort order Fell - fell a process tree Field - manipulate fields of data Find - search a file for text patterns Form - produce form letter by prompting user for information Format - format (roff) text Get - get generation from TCS file Grep - search file[s] for a pattern Hsh - shell with history and editing functions Incl - expand included files Intro - list on-line documentation Isam - generate index for pseudo-indexed-sequential access Kwic - make keyword in context index Lam - laminate files Lcnt - line count -2- Release Notes Ld - loader Ll - print line lengths Ls - list contents of directory Lsd - list alpha-numeric directory names Macro - process macro definitions Man - run off section of users manual Mcol - multicolumn formatting Mcrbck - spawn MCR commands in the background Mkd - make alpha-numeric directory for ODS-1 Mubld - build a multi-user task for RSX-11M Mv - move (or rename) a file Number - number lines Objfix - fixup object modules for building multi-user tasks Os - convert backspaces into multiple lines for "printers" Pack - pack words into columns Pl - print specified lines/pages in a file Pr - paginate files to standard output Prd - print current working directory in alpha-numeric mode Printf - justify fields of data in fixed-width fields Prlabl - format labels for printing Pwd - print working directory name on standard output Rar - rearrange archive Ratfor - RatFor preprocessor Rc - RatFor compiler Rev - reverse lines Rm - remove files Ruler - display ruler on terminal screen Sched - a way to repetitively invoke a command Sedit - stream editor Sepfor - Split FORTRAN programs into multiple files Sh - shell (command line interpreter) Sleep - cause process to suspend itself for a period of time Sort - sort and/or merge text files Spell - find spelling errors Split - split a file into pieces Tail - print last lines of a file Tee - copy input to standard output and named files Tr - transliterate characters Tsort - topologically sort symbols Ttt - 3-dimensional tic tac toe Txtrpl - perform generalized text replacement Ul - convert backspaces into multiple lines for "terminals" Uniq - strip adjacent repeated lines from a file Unrot - unrotate lines rotated by kwic Wc - count lines, words, and characters in files Wcnt - (character) word count Whereis - search for file matching regular expression Xch - extended change utility Xfind - entended find utility Xref - make a cross reference of symbols -3- Release Notes Previously released mail utilities Mail - utility for sending mail to local users Msg - utility for manipulating message files Msplit - utility for salvaging message files Postmn - report the presence of mail Resolve - resolve mail system user names Sndmsg - utility for sending mail to other users Users - list valid mail users SIG Tape UIC's with relevance to VOS [307,30] Toolgen.cmd, the command file for building the tools system. Toolgen.doc contains a complete TOOLGEN dialogue, with user responses in boldface. [307,31] All of the files necessary to build this release of the Tools on RSX-11M[+]. [307,32] The source files for RSX-specific tools and primitive functions. [307,33] The source for the portable VOS utilities. [307,34] The files necessary to build the variable-length send/receive driver for RSX-11M. It may be used for argument passing between parent and child processes. There are also correction files and a document describing home directory management for RSX-11M[+]. [307,35] Archives containing the format input files for the manual entries in sections 2-4 of the manual. [307,36] The initial distribution of the Software Tools Distributed Mail System. [307,37] The sources for the previous release of the mail system (for IAS and M+ sites), as well as the additional source files needed to build the system on IAS. -4- Release Notes On Disk Structure of the Tools VOS The tools system uses six (6) consecutive members of one UIC group divided over two logical disks. ST0: is used for the three read-only directories, while ST1: is used for the three writable directories. The system uses group 105 as distributed, but this can be modified during the installation. Assuming that the default UIC's are used, the current assignments are: st0:[105,1] main tools directory, ~bin. All system task images, script files and other known files reside here. st0:[105,2] location of site-dependent images, script files and other known files, ~usr. Any modified tools, or site-dependent utilities desired to be in the shell's search path should be placed here. This directory is searched by the shell before ~bin. st1:[105,3] scratch files created by the tools go here, ~tmp. st1:[105,4] files formatted by the `lpr' tool go here, ~lpr (not implemented). st1:[105,5] mail directory, ~msg. The mail database and msg help file are kept here. st0:[105,6] man directory, ~man. The manual entries for the tools reside here. In order to save space on the disk, one may refrain from building all of the tools. This can be achieved by copying the files in [307,31] to a disk and deleting the files with `.f' extensions which are not wanted. Note: do NOT delete `sh.f', as it is needed during toolgen. The tool summary above may be consulted for one-liners on each tool to determine candidates for deletion. The tools perform their own I/O with FCS calls, with the only reliance upon the Fortran OTS being the use of the $FCHNL entry point to retrieve the FDB address for each line. (Note: if you are running F4P v.3.0 with RMS support, the definition for D.FDB in RDATA.MAC may have to be changed before performing a toolgen.) The runtime system provides for 7 concurrent open files, using luns 1-7. Lun 8 is a scratch lun used by the system. Each tool attaches TI: to enable typeahead and ^O. -5- Release Notes FCS resident library It is strongly recommended that an FCS resident library be used, not just for the tools, but for all of your user-level code. This may be the single most important step you can take to reduce checkpointing and disk utilization in your system. A note of caution: no matter how one builds FCSRES, all of the FCS entry points cannot fit into one 8192 byte segment. If code written with references to one of the non-included entry points attempts to link to FCSRES, TKB will be unable to resolve the reference, since it is unable to extend the FCS psect, $$RESL. A way to eliminate this problem is to patch lb:[1,1]fcsres.stb to change the psect name of the routines in FCSRES. The recommended patch, using ZAP in absolute mode, is to patch location 26 (octal), which contains ESL in rad50, to ESK in rad50. Runtime requirements Two global logical device assignments must exist, ST0: and ST1:. For ease of use, it is suggested that one of the shells (sh.tsk or hsh.tsk) be installed as "...SHL". The default shell installed in tools.cmd is hsh.tsk. ~bin/felltree must be installed as "...FEL". FELL permits the user to kill all subtasks in the process tree emanating from the shell invoked from MCR. This can be performed as follows: 1. Type ^C and wait for the MCR> prompt. 2. Type FELL and wait for the termination messages from the subtasks. If your terminal is currently operating in half-duplex, and the bottom task in the tree is reading your terminal, you may have to type a carriage return in order to prime the fall of the tree. You should see a shell prompt (%) when all is done. If you do not wish to see an extraneous MCR prompt (>), then terminate the FELL line with an ESCAPE. Due to the large task images of the tools, and the way that shell scripts are performed (a copy of the shell is spawned reading the script file as its input file), the checkpoint file[s] is heavily used. A reasonably safe assumption is 500 blocks per SIMULTANEOUSLY active tools user. The average number of background process trees active must be included in the count of active users. -6- Release Notes Memory Limitations (and how to overcome them) Several of the more heavily used tools from the toolkit are right at the hairy edge of exceeding their address space limitations. This problem is most acute when linking to the PLAS-overlaid version of FCSRES. Fortunately, all of the tools will fit when not linked to FCSRES. Two courses of action can be taken in the face of these problems, one of them preventive in nature, the other corrective. BEFORE starting your toolgen, if you are using the F4P or F77 compilers, it behooves you to be sure that the "XXXNER.OBJ" module provided on your compiler distribution kit can be found in LB:[1,1] under the following name: if compiler == F77, then LB:[1,1]F77NER.OBJ if compiler == F4P, then LB:[1,1]F4PNER.OBJ Toolgen, if it finds such a file in LB:[1,1], will automatically include its specification in the task build command file used to build each tool. If one or more of the taskbuilds aborts with an "illegal memory limits" message, the following procedure can be followed to set things right: 1. Move to a convenient working directory. 2. Fetch the appropriate source files from [307,31] (tool.f). 3. Install the shell as INSTALL ST0:[105,1]SH/TASK=...SHX This is necessary since the history shell, which is normally installed as "...SHL" is usually one of the tools which overflows its bounds. 4. Invoke shx; rebuild the overgrown tasks with command lines of the following form noresbld tool.f 5. Test out the new utilities to make sure that they work correctly; then copy the task images to the ~bin/ directory. If hsh was one of the culprits, be sure to remove "...SHL" and re-install. The reason that this procedure works is that most tools which have to use a known file (such as `ld', which uses tools.tkb) look for the file along the standard search path (current, home, ~usr, ~bin). The script `noresbld' creates a local version of tools.tkb, omitting the reference to FSCRES; it then invokes `fc', which invokes `ld', which finds and uses the local version of tools.tkb. Finally, the local version of tools.tkb is removed. -7- Release Notes Source File Structure The source code for `tool' is contained in a file [307,33]tool.tcs (if the tool is portable across operating systems) or [307,032]tool.tcs (if it is an RSX-specific tool). This TCS source file contains an edit history of all changes made to the source. The output of the `get' utility operating on a `.tcs' file results in a file (tool.w) which is all of the environment necessary to rebuild the tool, provided that the VOS is operational. The tool.w file is an archive containing: 1. All of the files "included" by the ratfor source code. 2. The ratfor source file, tool.r. 3. The format input for the manual entry, tool.fmt. 4. And optionally, any extra definition files needed to build alternate versions of the tool (eg. sh => hsh). As an example, suppose that you wish to change the subroutine "module" in "tool". The suggested scenario is as follows: >shl % get /307,33/tool.tcs tool.w % ar xv tool.w % ar xv tool.r module % ed module (make changes and write file) % ar uv tool.r module % rc -v tool.r % (test out new tool. repeat last three steps until satisfied.) % ed tool.fmt (modify writeup to reflect changes) % ar uv tool.w tool.r tool.fmt % pip st0:[105,2]tool.tsk=tool.tsk % delta tool.w /307,33/tool.tcs (Identify in the comments the reason for the changes, and which modules changed.) % format tool.fmt >tool % ar uv ~man/s1 tool % asam <~man/s1 | sort >~man/i1 Placing tool.tsk in ~usr causes the shell to find your modified version of "tool" rather than the distributed one. If the utility being changed has a large source file, you may have to use `pdelta' (paged delta) instead of `delta' to successfully update the TCS source archive. The last two commands above cause the manual entry for `tool' to correctly correspond to the utility itself. -8- Release Notes Source for Primitive and Library Functions The source archive for the primitive and library functions may be found on [307,32]rlib.w. This archive consists of six (6) modules: 1. minprim.mar - an archive of macro files which constitute the routines written in assembler that are RSX-specific and always included in each tool. 2. minlib.mar - an archive of macro files which are assembler versions of portable library routines (usually rewritten for speed) and are always included in each tool. 3. prim.m - an archive of macro files which are written in assembler and used by one or more tools. 4. lib.m - assembler versions of portable ratfor routines which are used by one or more tools. 5. prim.r - archive of ratfor source routines which are RSX-specific and used by one or more tools. 6. lib.r - an archive of ratfor archives of portable library routines. To generate an assembler input file from the .mar files above, simply execute one of the following commands % ar p minprim.mar >minprim.mac % ar p minlib.mar >minlib.mac To assemble any of the modules in prim.m or lib.m, it is necessary to extract the module(s) and assemble them individually % ar xv prim.m f11sub.mac When assembling any of the .mac files, it is necessary to also include the site-dependent prefix file generated by toolgen, as in %mac f11sub=st0:[105,1]asm.pre,sy0:''f11sub To modify one of the routines in prim.r, simply extract it using the archiver, edit it up, update the archive, and recompile via % rc -cv prim.r To modify one of the routines in lib.r, it is necessary to perform two extractions and two updates, as in % ar xv lib.r arsubs.r -9- Release Notes % ar xv arsubs.r aopen % ed aopen % ar uv arsubs.r aopen % rc -cv arsubs.r % ar uv lib.r arsubs.r Of course, after generating new object modules for modified routines, it is necessary to make a system-specific version of [105,1]rlib.olb in [105,2], and to replace the object module in [105,2]rlib.olb. It is also a good idea to avoid replacing the modified modules in the archives until you are sure that they work. Writeups on all primitive routines which are to be visible to programmers may be found using the `man' command on section 2 of the manual. Writeups for library routines are in section 3. Manual entry structure In order to simplify the generation of manual entries for utilities and functions, a set of `format' macros are defined in the file `~bin/manhdr'. For the correct working of the `intro' utility, it is necessary that the first three lines of any site-dependent writeups consist of .so ~bin/manhdr .hd (
) () one line description of the tool or function where is replaced by the name of the function or tool,
is the section of the manual this entry is for and is the date the document was created. The .hd macro guarantees that the margins are correct, the header line on the manual pages is consistent with the software tools standard, and that NAME name - one line description of the tool or function appears in the writeup. This particular landmark is used by the intro utility to list the one-liners for the known entries in each section. The best method is to peruse the macros in ~bin/manhdr and to look at some of the writeups supplied with the system. -10- Release Notes Source Differences for F4P vs. FOR The tools are designed to work with either compiler. Two utilities are dependent upon which compiler is used, with the compiler dependency handled through conditional preprocessing of the source code. During TOOLGEN, three symbol definitions are appended to ~bin/symbols to reflect the answers given to questions concerning the compiler. They are: define(FORT_COMP,FOR/F4P) define(COMP_NAME,"FOR"/"F4P"/"OTHER") define(FLOAT_PT,YES/NO) The compiler name, if not "FOR" or "F4P", is whatever string was answered to TOOLGEN question 7A. If the compiler is F4P, FLOAT_PT is defined as YES, otherwise NO, unless the answer to TOOLGEN question 9 was yes. Given these definitions, `fc' and `ld' will conditionally pre-process correctly. Running tools from MCR Each tool is built to retrieve its arguments from MCR if it was installed with a "...XXX" name. To make "tool" available as "...TOL" from MCR, simply type the following MCR command: >INS ST0:[105,1]TOOL/TASK=...TOL -11- Release Notes Legal file specifications for the tools Besides the legal RSX file specs (ddn:[g,m]file.typ;ver), several shorthand file specs are understood by the tools i/o system, and thereby, understood by all of the software tools. They are [g,m]file.typ;ver ddn:[g,m]file.typ;ver /g,m/file.typ;ver /ddn/g,m/file.typ;ver [alpha-dir]file.typ;ver ddn:[alpha-dir]file.typ;ver /alpha-dir/file.typ;ver /ddn/alpha-dir/file.typ;ver ~name/file.typ;ver ~/file.typ;ver The two specs which include `/g,m/' simply replace the varied ODS-1 delimiters by slashes, which is typically lower-case on all terminals and is normally easy to strike using the right pinky. The four specs which include `alpha-dir' are exploiting the alpha-numeric directory capability which is provided by the utility `mkd'. `mkd' permits one to associate a name (such as sventek) with a UIC and to be able to refer to the directory as /sventek instead of /g,m. The ~name capability is only available for the six known directories of the tools system, ~bin, ~usr, ~tmp, ~lpr, ~msg and ~man. They permit one to write portable scripts for utilities across different operating systems. The ~/ is shorthand for the user's home directory. Consult the document [307,34]home.doc for information on how to assure that a user's home directory can always be found. In utilities which manipulate directories, all of the above formats are valid when the file.typ;ver trailer is removed. -12- Release Notes Changes for the Spring 1982 Release Modified Utilities * `dspc' has been removed. The functionality is now provided by `cat' using the "-v" flag. * `roff' is no longer distributed, having been superceded by `format'. * `crt' now reads the terminal in rare mode. * `ed' accepts several new commands. See the manual entry for details. * `fb', `find' and `grep' understand the flag "-i" which causes pattern matches to be case insensitive. * `man' has been rewritten to support sections of the manual and to page the output to the user's terminal. * `ratfor' understands character constants of the form 'c', as in the C programming language. As such, the apostrophe (') can no longer be used to delimit hollerith strings and strings in `string' declarations. Escaped characters may be expressed as '@c' (eg. '@n' == NEWLINE). The manifest constant definitions in `~bin/symbols' for the characters, NEWLINE and TAB are no longer automatically loaded when invoking the pre-processor. The version of ratfor distributed in the Spring 1981 release is provided as `oldratfor' to permit conversion of the source code at the programmer's convenience. To aid the conversion process, an utility `txtrpl' is provided such that txtrpl new_ratfor_file ~bin/pf will convert all the missing manifest constants to character constants. Of course, if `old_ratfor_file' was an archive, you will need to salvage `new_ratfor_file' before using it. * `rm' understands a flag "-i" which causes the file removal process to be interactive. -13- Release Notes Changes for the Spring 1982 Release Modified primitives * remove - remove a file old: call remove(file) new: stat = remove(file) * note - note current address of file pointer old: call markl(fd, addr) new: stat = note(addr, fd) * homdir - determine user's home directory old: call homdir(buf) new: call homdir(buf, dtype) * mailid - return the user's mail identification string Comment strings inside of parentheses () now are returned by mailid. If your code does not expect any blanks in the buffer returned, you will have to add the following code call mailid(user) i = index(user, ' ') if (i > 0) user(i) = EOS -14- Release Notes Changes for the Fall 1982 Release Modified Utilities * Bugs fixed in `fb', `format', `crt', `msg', `oldratfor', `os', `xch'. No change in functionality. * `rc', `fc' and `ld' modified to understand the "-m" flag, which indicates that a load map is to be generated. In addition, specifying a "-d" causes the generated fortran source files to be retained. * `man' will now report all known manual sections when invoked with a bare "-s" option. * `ratfor' has had several bugs fixed, most notably the integer overflow problem when gobbling large numeric strings. Strings quoted with apostrophes (') will now be handled as before, except for those of the form 'c' and '@c'. A "-n" flag will cause `ratfor' to forego the loading of the symbols file. * `sh' now understands several new internal commands, including aliases and parameters. Consult the manual page for more information. Additionally, commented commands in scripts are handled differently. Formerly, if a '#' was the first character of a line, the entire line was treated as a comment. Now, if the verb of a command is a '#', then that command is a comment. For example: echo this; # this is a comment; echo that is valid and will result in the output this that * `form' now can be informed that it is to read a multi-line answer to a prompt without having to escape the end of each intermediate line. If the prompt is of the form <-string> then `form' will prompt with "string" and read the response until it receives a line consisting of a bare period '.'. * `tail' has had its internal buffer increased to handle larger screen sizes. * `txtrpl' has been taught to NOT perform string replacements inside -15- Release Notes of ratfor quoted strings and comments. An additional pattern file is available in ~bin to permit `vgrind'-like pretty printing of ratfor source files. New Utilities * `printf' permits users to display fields of input data justified in fixed-width output fields. See the manual page for more information. Modified Library Routines * `stake' had a typographical error fixed. New Library Routines * A function `rmdef' has been added to permit a (name,defn) pair to be removed from a symbol table. Consult the manual page for more information. -16-