The WOMBAT EXAMINER July 1979 Vol. 1 No. 1 DTR V2 will Support Variables 1 Welcome to the first issue of the DATATRIEVE SIG Newsletter. If the cover does not make sense, you need to invoke the following DATATRIEVE commands: HELP WOMBAT HELP ADVANCED WOMBAT A search is on for a cover photo for the next issue. Member contributions should be sent to the editor. The cover need not be computer-generated; an actual photo of one or more Wombats in action would be appropriate. TABLE OF CONTENTS MESSAGE FROM CHAIRMAN 3 VERSION 1.0 3 VERSION 1.1 MULTIPLE FILE USAGE 4 NOTES FROM USERS 10 SPR SUMMARY 17 UNDOCUMENTED FEATURES 17 VERSION 2.0 PREVIEW 17 WISH LIST 19 AN APPROACH TO DATATRIEVE MANAGEMENT 23 QUESTIONS - ANSWERS 28 SAN DIEGO 28 2 INSTRUCTIONS FOR CONTRIBUTIONS Contributions should be sent to: Editor, DATATRIEVE Newsletter c/o DECUS, One Iron Way MR2-3/E55 Marlboro, MA 17562 Letters and articles for publications are requested from members of the SIG. They may include helpful hints, inquiries to other users, reports in SIG business, summaries of SPR's submitted to Digital or other information for members of the DATATRIEVE SIG. Camera ready copy is appreciated, but almost anything legible will be considered. However, this newsletter is not a forum for job and/or head hunting, nor is commercialism appropriate. Names of Contacts: The following SIG members have volunteered to receive phone calls regarding DATATRIEVE. Please remember that your phone call is an interruption of a full-time obligation to their employer. Prepare your questions beforehand and limit the call to 15-20 minutes. Additional volunteers are welcome. SIG CHAIRMAN: NEWSLETTER EDITOR: Chuck Watson Kathy Tamer Pacific Northwest Laboratories MATSCO Battelle Northwest 1050 Bay Area Blvd. Richland, Wash. 99352 Houston, TX 77058 509/942-3251 713/483-6121 FTS 444-3251 FTS 525-6121 OTHER CONTACTS: Dave Nordby Jim McIlvaine G.D. Searle Bridgeport-Trextron P.O. Box 5110 200 Precision Rd. Chicago, Ill. 60680 Horsham, PA. 19044 312/982-4631 215/674-2700 3 MESSAGE FROM THE CHAIRMAN This is the first official copy of the Wombat Examiner, the DATATRIEVE SIG newsletter. It contains some material published informally in January and some material distributed at the DECUS meeting in New Orleans. Our membership list is currently approaching 200, and we percieve that membership is motivated by a need to share DATATRIEVE techniques which transcend the User's Guide. We plan that the newsletter will be the primary activity of the SIG. The success of this approach depends on written contributions from members. Because we are just getting started, I wrote most of the material in this issue. I coerced Jim McIntyre to write up his portion (after all, he works with me). Dave Nordby and Joan Hilton are the first (and only) SIG members to voluntarily contribute written material. If you find that sharing of DTR techniques, gripes, and suggestions through this newsletter has been helpful, take a few minutes and send us your written material. On the other hand, if you think the focus of this inqugural newsletter is wrong - let us know. Chuck Watson VERSION 1.0 There is no news worth publishing about Version 1.0 except this: If you are still running DATATRIEVE V 1.0 your software salesman is not on the ball. Version 1.1 became available in the fall of 1978. 4 Version 1.1 MULTIPLE FILE USAGE Using Two Files: It is quite easy to use two files in DATATRIEVE. The easiest way to illustrate this is with the YACHTS file and another file - OWNERS. Since the OWNERS file wa not distributed with DATATRIEVE, you must create it as follows: DOMAIN OWNERS USING OWNER-REC ON OWNER.DAT; RECORD OWNER-REC 01 OWNER. 03 TYPE. 06 BUILDER PIC X(10). 06 MODEL PIC X(10). 03 NAME PIC X (10). ; Populate the file as follows: BUILDER MODEL NAME ALBERG 35 SHERM ALBIN VEGA STEVE ISLANDER BAHAMA JIM ISLANDER BAHAMA ANN ISLANDER BAHAMA STEVE PEARSON 26 DICK This gives us an example of one model with 3 owners (either a co-op or three separate yachts) and one owner with two yachts (obviously STEVE has too much excess capital!). Now we have two files which we can use for testing the following and future suggestions. Hang onto YACHTS and OWNERS, they will be used in all future Newsletter Question and Answer articles. If you have questions, frame them in terms of these files. When you read an answer, you can test it using these files. 5 WHICH YACHTS ARE OWNED? The following query will print a list of those YACHTS which have entries in the OWNERS file: DTR. FOR OWNERS PRINT YACHTS WITH TYPE = OWNER.TYPE LENGTH OVER MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE ALBIN VEGA SLOOP 27 5,070 08 $18,600 ISLANDER BAHAMA SLOOP 24 4,200 08 $6,500 ISLANDER BAHAMA SLOOP 24 4,200 08 $6,500 ISLANDER BAHAMA SLOOP 24 4,200 08 $6,500 PEARSON 26 SLOOP 26 5,400 08 Note: This gives 5 lines, one per owned YACHT. WHO OWNS WHICH YACHT? This requires a similar type of intersection between the two files. We need to print the names of the owners as well as the description of the YACHT. DTR> FOR OWNERS FOR YACHTS IF OWNER.TYPE = BOAT.TYPE PRINT OWNER.NAME,BOAT LENGTH OVER NAME MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE STEVE ALBIN VEGA SLOOP 27 5,070 08 $18,600 ANN ISLANDER BAHAMA SLOOP 24 4,200 08 $6,500 JIM ISLANDER BAHAMA SLOOP 24 4,200 08 $6,500 STEVE ISLANDER BAHAMA SLOOP 24 4,200 08 $6,500 DICK PEARSON 26 SLOOP 26 5,400 08 Note: This still gives one line per owner. 6 MAKING A COLLECTION OF OWNED YACHTS It is often used to make a collection from one file based on attributes in another. One might expect the following to work: DTR> FOR OWNERS FIND ALL YACHTS WITH TYPE = OWNER.TYPE [0 records found] [1 Record found] [1 Record found] [1 Record found] [1 Record found] [1 Record found] DTR> SHOW COLLECTIONS Collections: CURRENT DTR> SHOW CURRENT Collection CURRENT Domain: YACHTS Number of records: 1 No selected record DTR> PRINT ALL OF CURRENT LENGTH OVER MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE PEARSON 26 SLOOP 26 5,400 08 Obviously the FOR LOOP approach will not work. SOLUTION: Version 1.1 supports an undocumented method for doing this which depends on the word ANY. DTR> FIND YACHTS WITH ANY OWNERS WITH TYPE = BOAT.TYPE [3 records found] DTR> SHOW CURRENT Domain: YACHTS Number of records: 3 No selected record DTR> PRINT ALL OF CURRENT LENGTH OVER MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE ALBIN VEGA SLOOP 27 5,070 08 $18,600 ISLANDER BAHAMA SLOOP 24 4,200 08 $6,500 PEARSON 26 SLOOP 26 5,400 08 7 USING THE COLLECTION OF OWNED YACHTS Here is a simple report of owned yachts and their owners. It depends on the previous FIND. DTR> PRINT ALL BOAT, ALL NAME OF OWNERS WITH TYPE = BOAT.TYPE LENGTH OVER MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE NAME ALBIN VEGA SLOOP 27 5,070 08 $18,600 STEVE ISLANDER BAHAMA SLOOP 24 4,200 08 $6,500 ANN JIM STEVE PEARSON 26 SLOOP 26 5,400 08 DICK Note: This is equivalent to: DTR> DTR> FOR CURRENT PRINT BOAT, ALL NAME OF OWNERS WITH TYPE = BOAT.TYPE LENGTH OVER MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE NAME ALBIN VEGA SLOOP 27 5,070 08 $18,600 STEVE ISLANDER BAHAMA SLOOP 24 4,200 08 $6,500 ANN JIM STEVE PEARSON 26 SLOOP 26 5,400 08 DICK CONCLUSION We now have two files YACHTS and OWNERS on our systems. I have shown several ways of using them in DATATRIEVE. Please try my examples and let me know of problems or other interesting constructs. 8 EXAMPLE OF USING TWO FILES TO CREATE A THIRD DTR> SHOW YACHT RECORD YACHT USING 01 BOAT. 03 TYPE. 06 MANUFACTURER PIC X(10) QUERY-NAME IS BUILDER. 06 MODEL PIC X(10). 03 SPECIFICATIONS QUERY-NAME SPECS. 06 RIG PIC X(6). 06 LENGTH-OVER-ALL PIC XXX QUERY-NAME IS LOA. 06 DISPLACEMENT PIC 99999 QUERY-HEADER IS "WEIGHT" EDIT-STRING IS ZZ,ZZ9 QUERY-NAME IS DISP. 06 BEAN PIC 99. 06 PRICE PIC 99999 EDIT-STRING IS $$$,$$$. ; DTR> DTR> DTR> SHOW OWNER-REC RECORD OWNER-REC USING 01 OWNER. 03 TYPE. 06 BUILDER PIC X(10). 06 MODEL PIC X(10). 03 NAME PIC X(10). ; DTR> DTR> DTR> SHOW DEBTOR RECORD DEBTOR USING 01 DEBTOR-REC 02 NAME PIC X(10). 02 DEBT PIC 9(5) EDIT-STRING IS $$$,$$$. 02 REMARK PIC X(30). ; DTR> DTR> READY YACHTS DTR> DTR> READY OWNERS DTR> DTR> READY DEBTORS WRITE DTR> DTR> FOR YACHTS FOR OWNERS DTR> IF OWNER.TYPE = BOAT.TYPE AND BOAT.PRICE NE 0 DTR> STORE DEBTORS USING DTR> NAME = OWNER.NAME THEN DTR> DEBT = BOAT.PRICE THEN DTR> REMARK = "SOLD BEFORE 1979" 9 DTR> DTR> PRINT ALL DEBTORS NAME DEBT REMARK ANN $6,500 SOLD BEFORE 1979 JIM $6,500 SOLD BEFORE 1979 STEVE $6,500 SOLD BEFORE 1979 STEVE $18,600 SOLD BEFORE 1979 DTR> FOR YACHTS FOR OWNERS DTR> IF OWNER.TYPE = BOAT.TYPE AND BOAT.PRICE NE 0 DTR> STORE DEBTORS USING DTR> NAME = OWNER.NAME THEN DTR> REMARK = "SALES PRICE UNKNOWN" DTR> PRINT ALL DEBTORS NAME DEBT REMARK STEVE $18,600 SOLD BEFORE 1979 JIM $6,500 SOLD BEFORE 1979 ANN $6,500 SOLD BEFORE 1979 STEVE $6,500 SOLD BEFORE 1979 DICK SALES PLRICE UNKNOWN DTR> 10 VERSION 1.1 NOTES FROM USERS From: Dave Nordby and Joan Hilton April 16, 1979 DTR NOTES Working with More than One Domain and/or Collection EXAMPLE 1: The following procedure searches field 1 of domain 1, picks up value in field 2 of domain 1, and uses this value to search field n of domain 1. In other words, it establishes a collection form one domain based on a collection from another domain and prints it. DTR> READY YACHTS DTR> READY OWNERS DTR> FIND OWNERS WITH BUILDER = "PEARSON" [1 record found] DTR> FOR RACHTS FOR CURRENT IF BOAT.TYPE.MODEL = TYPE.MODEL PRINT BOAT.TYPE MANUFACTURER MODEL AMERICAN 26 GRAMPIAN 26 PEARSON 26 RANGER 26 SAN JUAN 26 TANZER 26 WAYS THAT DIDN'T WORK DTR> FOR YACHTS FOR CURRENT IF BOAT.TYPE.MODEL = CURRENT.TYPE.MODEL PRINT No context has been established for field name "CURRENT.TYPE.MODEL" DTR> FOR YACHTS FOR CURRENT IF BOAT.TYPE.MODEL = TYPE.MODEL PRINT BUILDER MODEL NAME PEARSON 26 DICK PEARSON 26 DICK PEARSON 26 DICK PEARSON 26 DICK PEARSON 26 DICK PEARSON 26 DICK 11 EXAMPLE 2: The following procedure retrieves the same records as Example 1, but does so using the FIND command rather than the FOR...PRINT commands: DTR> FIND OWNERS WITH BUILDER = "PEARSON" [1 record found] DTR FIND YACHTS WITH ANY CURRENT WITH MODEL = BOAT.TYPE.MODEL [6 records found] DTR> PRINT No record selected, printing whole collection LENGTH OVER MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE AMERICAN 26 SLOOP 26 4,000 08 9,895 GRAMPIAN 26 SLOOP 26 5,600 08 11,495 PEARSON 26 SLOOP 26 5,400 08 RANGER 26 SLOOP 26 5,860 09 SAN JUAN 26 SLOOP 26 4,400 08 TANZER 26 SLOOP 26 4,350 09 11,750 WAYS THAT DIDN'T WORK: DTR> FIND COLLECT FOR YACHTS FOR CURRENT IF BOAT.TYPE.MODEL = TYPE .MODEL Expected end of statement, encountered "FOR" DTR> FOR YACHTS FOR CURRENT FIND COLLECTION WITH BOAT.TYPE.MODEL = TYPE.MODEL Collection is neither a collection nor a readied domain DTR> FIND ALL COLLECTION FOR/IN YACHTS FOR/IN CURRENT WITH BOAT.TYPE.MODEL = TYPE.MODEL Expected end of statement, encountered "IN/FOR" DTR> FOR YACHTS FOR CURRENT FIND COLLECTION WITH BOAT.TYPE.MODEL = TYPE.MODEL Collection is neither a collection nor a readied domain DTR> FOR YACHTS FOR OWNERS FIND YACHTS WITH BOAT.TYPE.MODEL = TYPE.MODEL (found 113 records, i.e., all of yachts, over and over again till terminated) DTR> FOR YACHTS FOR OWNERS FIND YACHTS WITH BOAT.TYPE.MODEL = OWNERS.TYPE.MODEL (found 5,1,1,1,1,6,5,1,1,1,1,6....records till terminated) DTR> FOR YACHTS FOR CURRENT IF BOAT.TYPE.MODEL = TYPE.MODEL SELECT BOAT Non-digit string in American 26 . . . ignoring characters Value too large Value too large Record number out of range for collection 12 EXAMPLE 3: The following procedure restricts itself to one domain, but uses more than one collection. It searches field 1 of domain, picks up the value in field 2 of the selected records, and uses this and a specified value for field 3 to retrieve records. Specifically, it searches the yachts file for all rigs = "ketch", stores them in collection "a", and for those builders, prints out their rigs = "sloops". DTR> FOR A IN YACHTS WITH RIG = 'KETCH' [Looking for statement] DTR> FOR YACHTS WITH BUILDER = A.BUILDER AND RIG = 'SLOOP' PRINT LENGTH OVER MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE CHALLENGER 32 SLOOP 32 12,800 11 $31,835 CHALLENGER 35 SLOOP 35 14,800 12 39,215 GRAMPIAN 26 SLOOP 26 5,600 08 11,495 GRAMPIAN 28 SLOOP 28 6,900 10 14,475 GRAMPIAN 30 SLOOP 30 8,600 09 17,775 GRAMPIAN 2-34 SLOOP 34 11,800 10 29,675 IRWIN 30 SLOOP 30 10,000 10 19,950 IRWIN HALF TON SLOOP 30 7,300 10 IRWIN 25 SLOOP 25 5,400 12 10,950 ISLANDER 28 SLOOP 28 5,994 10 15,908 ISLANDER 30 SLOOP 30 8,600 10 20,990 ISLANDER 36 SLOOP 36 13,450 11 31,730 ISLANDER BAHAMA SLOOP 24 4,200 08 6,500 NORTHERN 29 SLOOP 29 7,250 09 20,975 PEARSON 26 SLOOP 26 5,400 08 PEARSON 26W SLOOP 26 5,200 09 PEARSON 28 SLOOP 28 7,850 09 PEARSON 30 SLOOP 30 8,320 09 PEARSON 10M SLOOP 33 12,441 11 PEARSON 35 SLOOP 35 13,000 10 PEARSON 36 SLOOP 37 13,500 11 PEARSON 39 SLOOP 39 17,000 12 PEARSON 26 SLOOP 26 5,400 08 PEARSON 26W SLOOP 26 5,300 09 PEARSON 28 SLOOP 28 7,850 09 PEARSON 30 SLOOP 30 8,320 09 PEARSON 10M SLOOP 33 12,441 11 PEARSON 35 SLOOP 35 13,000 10 PEARSON 36 SLOOP 37 13,500 11 PEARSON 39 SLOOP 39 17,000 12 13 Unfortunately, I was unable to repeat Example 3 using the FIND command. Some of my attempts follow. WAYS THAT DIDN'T WORK: DTR> READY YACHTS DTR> FIND A IN YACHTS WITH RIG = "KETCH" [13 records found] DTR> FIND YACHTS WITH ANY BUILDER = A.BUILDER AND RIG = "SLOOP" No context has been established for field name "A.BUILDER" DTR> FIND YACHTS WITH ANY A WITH BUILDER = A.BUILDER AND RIG = "SLOOP" No context has been established for field name "A.BUILDER" DTR> FIND YACHTS WITH ANY A AND WITH RIG = "SLOOP" Expected end of statement, encountered "RIG" [Syntax error--flushing input] DTR> FIND YACHTS WITH ANY A WITH RIG = "SLOOP" [10 records found] DTR> FIND A IN YACHTS WITH RIG = "KETCH" [13 records found] DTR> FIND YACHTS WITH ANY BUILDER = A.BUILDER AND RIG = "SLOOP" Expected end of statement, encountered "=" [Syntax error--flushing input] DTR> FIND YACHTS WITH BUILDER = A.BUILDER AND RIG = "SLOOP" No context has been established for field name "A.BUILDER" DTR> SHOW COLLECTIONS Collections: A (also CURRENT) DTR> SHOW CURRENT Domain: YACHTS Number of records: 13 No selected record DTR> FIND YACHTS WITH ANY BUILDER = A.BUILDER Expected end of statement, encountered "=" [Syntax error--flushing input] DTR> FIND YACHTS WITH ANY BUILDER WITH BUILDER = A.BUILDER AND RIG = "SLOOP" "BUILDER" is neither a collection for a readied domain DTR> FIND YACHTS WITH ANY CURRENT WITH BUILDER = A.BUILDER AND RIG = "SLOOP" No context has been established for field name "A.BUILDER" 14 DTR> FOR YACHTS WITH BUILDER = A.BUILDER PRINT No context has been established for field name "A.BUILDER" DTR> FOR A IN YACHTS WITH RIG = "KETCH" [Looking for statement] [1 record found] [3 records found] [2 records found] [5 records found] [1 record found] [1 record found] [4 records found] [5 records found] [2 records found] [1 record found] [10 records found] [10 records found] DTR> FOR A IN YACHTS WITH RIG = "KETCH" [Looking for statement] DTR> FIND YACHTS WITH BUILDER = A.BUILDER AND RIG = "SLOOP" [0 records found] [2 records found] [0 records found] [0 records found] [4 records found] [0 records found] [0 records found] [3 records found] [4 records found] [1 record found] [0 records found] [8 records found] [8 records found] DTR> FIND A IN YACHTS WITH RIG = "KETCH" [13 records found] DTR> FIND YACHTS WITH RIG = "SLOOP" AND BUILDER = A.BUILDER No context has been established for field name "A.BUILDER" 15 FROM OUR ILLUSTRIOUS CHAIRMAN, CHUCK WATSON, The following is one way of dealing with variable length records: DTR> DEFINE DOMAIN YACHT-TEXTS USING YACTXT-REC ON YACTXT.DAT; DTR< DEFINE RECORD YACTXT-REC USING DFN>@YACTXT.REC 01 YACHT-TEXT-REC. 02 YACHT-TEXT-KEY. 03 TYPE. 04 BUILDER PIC X(10). 04 MODEL PIC X(10). 03 SEQ-NO PIC 99. 02 REMARK PIC X(20). DFN> ; [Record YACTXT-REC is 42 bytes long] DTR> DEFINE FILE FOR YACHT-TEXTS KEY=YACHT-TEXT-KEY For the purpose of demonstration, populate your file as follows (please note: For those of you that picked up DTR notes in New Orleans, the BUILDER entries were spelled wrong!!!!!!) SEQ BUILDER MODEL NO REMARK ALBIN VEGA 01 THE VEGA IS NICE BUT ALBIN VEGA 02 TENDS TO CAPSIZE AFTER ALBIN VEGA 03 THE OWNER HAS TWO BEERS How to print YACHTS and corresponding text: DTR> FOR YACHTS PRINT BOAT, ALL REMARK OF DTR> YACHT-TEXTS WITH YACHT-TEXT-KEY BT DTR> BOAT.TYPE|00 AND BOAT.TYPE|99 16 LENGTH OVER MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE REMARK ALBERG 37 MK II KETCH 37 20,000 12 $36,951 ALBIN 79 SLOOP 26 4,200 10 $17,900 ALBIN VEGA SLOOP 27 5,070 08 $18,600 THE VEGA IS NICE BUT TENDS TO CAPSIZE AFTER THE OWNER HAS TWO BEERS ALBIN BALLAD SLOOP 30 7,276 10 $27,500 AMERICAN 26 SLOOP 26 4,000 08 $9,895 . . Please note the concantenation character | I used it because normally I use indexed files and I want each text record to have a unique key (in this case TYPE and SEQ-NO are unique). The remarks could be printed as follows, but an exhaustive search would be made. DTR> DTR> FOR YACHTS PRINT BOAT, ALL REMARK OF DTR> YACHT-TEXTS WITH TYPE=BOAT.TYPE LENGTH OVER MANUFACTURER MODEL RIG ALL WEIGHT BEAM PRICE REMARK ALBERG 37 MK II KETCH 37 20,000 12 $36,951 ALBIN 79 SLOOP 26 4,200 10 $17,900 ALBIN VEGA SLOOP 27 5,070 08 $18,600 THE VEGA IS NICE BUT TENDS TO CAPSIZE AFTER THE OWNER HAS TWO BEERS ALBIN BALLAD SLOOP 30 7,276 10 $27,500 AMERICAN 26 SLOOP 26 4,000 08 $9,895 AMERICAN 26-MS M/S 26 5,500 08 $18,895 BAYFIELD 30/32 SLOOP 32 9,500 10 $32,875 . . . 17 VERSION 1.1 SPR SUMMARY A volunteer is needed to write this column. SPR activity on DATATRIEVE has been low, but someone is needed to monitor them and prepare a summary for the newsletter. Also copies of SPR's submitted by SIG members could be published in this space. VERSION 1.1 UNDOCUMENTED FEATURES (Member contributions solicited) HELP WOMBAT (See cover) HELP ADVANCED WOMBAT (See cover) HELP ME HELP ADVANCED ME ANY - see page 7 OPEN filename - creates copy of terminal I/O CLOSE filename ALL - has more uses than shown in the User's Guide, See page 8. Preview of DATATRIEVE V 2 At the New Orleans DECUS meeting Jim Starkey, the developer of DATATRIEVE, presented a sneak preview of the functional enhancements planned for DATATRIEVE version 2. He was not allowed to make copies of his slides and his plans are not final at this point, but the following items, from my notes at that session, are apparently firm enough to talk about. - DBM-11 - Version 2 will function in a mostly read-only capacity with DBMS-11. The user will not necessarily know if the data is stored in a shema or an indexed sequential file. - VARIABLE LENGTH RECORDS - Version 2 will support the COBOL structures OCCURS and OCCURS DEPENDING ON. This will allow for multiple fields (or trailer records) repeated n times. - VIEW - Version 2 of DATATRIEVE will have a new type of artificial domain called View which will allow a logical record to be scattered among several primitive domains. For those of you familiar with the cumbersome version 1.1 techniques of working with two files, the View concept should be a great improvement. Changes 18 in the DATATRIEVE overlay structure will provide much faster multi- domain access. When the View function is being used it will allow links by value between files in RMS and links be sets in DBMS-11. Another use of the View will be the ability to restrict a user's access to only a portion of the data. For example, you could imagine a personnel file with several Views, each one allowing only access to a given department. The department manager could then use all the DATATRIEVE retrieval functions on the full file but it would appear that the file contained only one department. - HIERARCHIES WITHIN A PRINT LIST - Version 2 will fully support the word ANY in Boolian operations and sublist searching in the PRINT command as well as in FIND. Thus one could say PRINT YACHTS WITH ANY OWNERS WITH TYPE = BOAT.TYPE. - VALIDATION - Version 2 will provide automatic input validation so that in the record definition a user can describe a valid range of data using Boolian expressions. For example, BEAM might be valid if (BT 5 AND 20 AND IF LESS THAN LENGHT-OVER-ALL). All subsequent data entry will be denied if it is outside this range. Furthermore, the rejection of the bad data will be accompanied by a reprompt for correct data. This is an improvement over the current situation in which bad data is accepted with a warning. - EXTRACT - Version 2 will support this new command. It will allow writing of DATATRIEVE definitions such as record definitions, procedure definitions, etc., directly to another file. This will greatly facilitate documentation and backup techniques. - TABLES - In version 2 we will be able to define tables in the data dictionary. This will allow automatic translation of coded entries. For example, one could establish a table with 3 entries, "m" = "MALE", "F" = "FEMALE", any other coede = "OTHER". The user would see the words but the file would contain only the codes. - DATES - Version 2 will support dates, but not time of day. Dates will be stored in the 64 bit binary representation used on the VAX. Subtraction of dates will be supported so that elapsed days can be reported. Also, the dates will be elegantly handled on output. To do this five new words will be added to the records definition syntax. These COBOL-like enhancements or extensions are M, D, Y, J and W. For example, specifying an M in the picture string will cause the first letter of the month of the year to be printed. Specifying MM will cause the first two letters, etc. So each user can structure the display of date information as he chooses. The dates can also be input in rather free form including the recognition of the string TODAY being the system date. 19 - DOCUMENTATION - Hopefully version 2 will be released with a revised and expanded Users Guide and the long awaited Beginner's Guide. - POSSIBILITY - The one item that is not fully resolved for version 2 is the support for temporary variables. There has been a rather vocal demand for temporary variables and they may be included in version 2. - THANK YOU - The special interest group owes a big thank you to Jim Starkey for letting us in on his plans for version 2 at this early date so that we can plan our future applications around these enhancements and so we can refine our wish list for version 3. Chuck Watson DATATRIEVE WISH LIST A wish list was circulated at New Orleans. Jim Starkey, the Datatrieve Language Developer responded orally to this list on Friday morning. I have divided the list into two types: those which are anticipated in Version 2 and those which are not. I interviewed Jim Starkey a few weeks after New Orleans to get calm and considered answers to the items on the second list. DATATRIEVE WISH LIST TYPE 1 - Items which will probably be included in Version 2 Variable Length Records Arrays, i.e. "OCCURS" clause (no subscripts) Easier use of Multiple files More efficient use of multiple files Print line greater than 132 characters Multi file access by report writer NEW USERS GUIDE BEGINNERS GUIDE (See article on Preview of Version 2 elsewhere in this Newsletter) INTERVIEW WITH JIM STARKEY CHUCK - The Datatrieve SIG has prepared a wish list which you graciously discussed item by item at New Orleans. I have placed 2/3 of the items in a list called "TO BE INCLUDED IN VERSION 2" and we need not deal with them here. What I would like to do is get your reaction to the remaining items. JIM - Fine, but first I must point out that I am only the language developer. My management has the ability to change the nature ov Version 2 or even to decide not to market it. I can tell you my reactions to the wish list, but please do not hold me responsible for marketing my comments. 20 CHUCK - OK, I see your position, after all, I have had management override some of my ideas myself. JIM - It's not that I am knocking management. It's just that there are several levels of Product Review which evaluate any new release of DIGITAL software and the final release of Version 2 may be quite different from my oral description at New Orleans. CHUCK - Now that you have worned us not to expect the next version to be a combat killer, we would like your reactions to the wish list. JIM - OK. CHUCK - The item which was wished for most often is temporary variables. Many users appear to need them. I proposed that we ask for just a few with fixed names, and people said that as few as a dozen would be better than none. JIM - Yes, I know you all want to make DATATRIEVE like TICOL. That is not its purpose. But since so many requests have been made we willstrongly consider some sort of scratch variables. CHUCK - I should point out that Dave Nordby has developed a technique of creating variables by using a dummy domain. It is inefficient but gets the job done. JIM - Yes, he showed me his method, and I can see that something more efficient would be helpful. CHUCK - One user requested that DTR be merged with FMS-11. Other users have expressed a need for some sort of screen oriented data entry and/or modification scheme. JIM - We're actively looking at FMS. There's nothing we can do for next release, but after that.... CHUCK - Will that be an enhancement to GUIDE MODE? JIM - I can't answer that. 21 CHUCK - Speaking of GUIDE MODE, when will we be able to uae it with foreign terminals? JIM - Probably never. We have enough trouble with the operating systems and terminals we already have. CHUCK - Well I guess we will leave that one as a project for some user to solve. JIM - -----Silence. CHUCK - Well, moving on, let's talk about the Report Writer. JIM - We have plans to revise that section of the code, it's a bit inefficient. CHUCK - For Version 2? JIM - Possibly. CHUCK - Several users noted that each use of the report writer creates a new report. Often a combined report is the desired product. Current techniques for combining reports result in unnecessary blank pages between reports. JIM - We will take a look at that, but I think it is difficult. CHUCK - I got around it by using the Open feature before starting my reports. Copies of them all went to one file with a minimum of garbage between them. JIM - Yes, but we ought to be able to do it better. CHUCK - Some users would like to intermix BASIC +2 or COBAL dialog with DTR dialog without telling the user which was which. JIM - No, there is no way we can do this. Datatrieve is a task, not an operating system. Besides, I like the visibility. CHUCK - Yes, but couldn't a user eliminate the opening greeting by modification of QUERY.MSG. JIM - Yes, but I am not going to explain how in print. CHUCK - Well, I suppose those who need to figure out how. But we will still have the DTR> prompt. JIM - Yes there is no way to eliminate that, but why bother. CHUCK - One user reported what appears to be a bug. When NO DUP was specified for an alternate key, his entire procedure aborted when a duplicate key was encountered in this sequence. For OLD-DATA STORE NEW-DATA. What he wanted was for DTR to 22 issue a message, abort the store but continue with loop. JIM - Yes that's a bug. Have him send in an SPR and we will fix it. CHUCK - Speaking of messages, several users objected to multiple repetitions of the message RECORD TOO SHORT PADDING WITH BLANKS. They don't think they need all those messages -- they knew what they were doing when they put those short records into those long descriptions. JIM - They may have known, but others might not. Besides its a good message. CHUCK - I have run into a problem and several others mentioned it. We don't know how to MODIFY a field of a collection USING some constant or a field. It seems like a reasonable thing to do. JIM - Yes, you can do it but the syntax is awkward. MODIFY ALL USING WOMBAT = "FURRY" CHUCK - I will give that a try. Now the final item on the list - subscripted arrays. Many users come from FORTRAN or BASIC =2 backgrounds and are frustrated by the lack of subscripted arrays. JIM - We are looking at that, perhaps in a year or so. CHUCK - That's the wish list as of New Orleans. Thank you for your responses. JIM - Thank you for letting me communicate with the SIG members, I enjoy an enthusiastic user group. CHUCK - Thanks again Jim, and be forewarned - we will have an expanded wish list after San Diego. - - - - - - - - - - - SEND WRITTEN WISHLIST ITEMS TO: DATATRIEVE WISH LIST COORDINATOR 23 AN APPROACH TO DATATRIEVE MANAGEMENT Jim McIntyre Battelle Northwest Battelle Boulevard Richland, Washington 99352 Dear Datatrievers, I have received a njmber of requests to summarize my presentation on "AN APPROACH TO DATATRIEVE MANAGEMENT", given at the 1979 SPRING DECUS SYMPOSIUM. I hope the information I plresent below will be useful to those of you who have, or are considering the use of a SINGLE DATATRIEVE MANAGER CONCEPT. I am employed in the BIOMETRICS CENTER along with six other technical support specialists who provide data management and processing support for Life Sciences Research. The BIOMETRICS CENTER has the following hardwire: PDP 11/70; 512 K.bytes of memory; dual 88 megabyte disks; dual diskettes; a magtape drive; 16 interactive ports; 4 batch streams; a line printer; a card reader; and about 50 terminals that are dispersed throughout the labs. The system software that we use include IAS, V3.0; RMS-11K, V1.1; and DATATRIEVE-11, V1.1. The major languages that we use include COBOL, V4.0; FORTRAN IV PLUS, V2.5; and BASIC-PLUS 2, V1.6. We also use various statistics and graphics packages. My role as DATATRIEVE ADMINISTRATOR is to (1) provide technical support, (2) design and maintain DATATRIEVE definitions and (3) design and maintain indexed files. With regard to technical support, I act primarily as a channel for communication. I meet with scientists, for example, to discuss their data handling requirements. I then determine whether or not DATATRIEVE may be the best ligical solution to their data handling requirements. In some cases, the use of DATATRIEVE may be a partial solution to the scientists problem, if so, I then meet with the application programmers to discuss the development of COBOL or FORTRAN programs. I also approach the system manager with any software/hardware problems that I encounte. The system manager and I are supported by general funds while the application programmers and the scientists are supported by research funds. Also, the system manager is the only one with full lprivileges while I am the only one with DATATRIEVE privileges, that is, I am the only one who can add or delete definitions from the data dictionary. The application programmers have "compile and link" privileges and the scientists have only "run" privileges. My job as DATATRIEVE administrator is also to create and maintain DATATRIEVE definitions. I use IAS editor to set up all definitions into indirect command files. For example, YACHTS.DOM is an indirect command file that contains the domain definition for YACHTS: YACHTS.DOM 24 DELETE YACHTS; DEFINE DOMAIN YACHTS USING YACHT ON (200,121)YACHTS.SEQ; DEFINEP YACHTS 2,PW,BOAT,"W" Several commands are contained in YACHTS.DOM including DELETE, DEFINE and DEFINEP. When I invoke YACHTS.DOM in DATATRIEVE the current version of YACHTS will be deleted from the data dictionary before the new definition is created. Also, the password table will be created after the new version of YACHTS is created. If any changes need to be made to an existing definition, all I have to do is edit the indirect command file and then get into DATATRIEVE and invoke that command file. When I delete a lot of definitions from the data dictionary, the dictionary gets bigger because of the indexed pointers that have been left behind. So to compress the size of the dictionary I use the OCPRS utility supplied with DATATRIEVE. I have established naming conventions for DATATRIEVE definitions, indirect command files and data files as follows: DATATRIEVE DEFINITION NAMES 1) Domains are plural, e.g., RATS 2) Records are singular, e.g., RAT 3) 01 record level is singular-rec, e.g., RAT-REC 4) Names of all definitions associated with the same study begin with the same letters, e.g., RATS, RAT AND RATPRINT NAMES OF INDIRECT COMMAND FILES CONTAINING: 1) Domain definitions are plural with extension .DOM, e.g., RATS.DOM 2) Record definitions are singular with extension .REC, e.g., RAT.REC 3) Procedure definitions have extension .PRO, e.g., RATPRINT.PRO 4) The names of all command files containing definitions associated with the same study begin with the same letters, e.g., RAT.REC, RATS.DOM amd RATPRINT.PRO DATA FILES 1) Indexed files accessed by DATATRIEVE have the extension .NDX 2) Sequential files accessed by DATATRIEVE have the extension .SEQ 3) The names of all data files that pertain to the same study begin with the same letters, e.g., RATS.NDX, RATWEIGHT.NDS, etc. I have established a format for defining record definitions so as to improve their readability, to indicate which fields comprise the primary or alternate keys, and to document the contents of each field within the record. For example, the RAT record definition (see below) has been created using spaces and indentation at each sublevel, and tabs have been used between the descriptive phrases, e.g., PIC, QUERY-HEADER, etc., after each field name. To minimize performing exhaustive sequential retrievals from an indexed file I have incorporated the work 'KEY' into any field name that corresponds to a primary key or alternate key. For example, ID-KEY is the field name that 25 refers to the primary key of the RATS indexed file. ID-KEY consists of RAT-NC and SAMPLE-DATE. The other aspect of my record definition format is to chose field names that describe the contents of each field. For example, AGE-IN-DAYS informs me that this field contains the rats age, and that the age of the rat is recorded in days. If I chose, for example, AGE for the field name then I would at least incorporate a QUERY-HEADER to document the full contents of that field. RAT RECORD DEFINITION FORMAT DEFINE RECORD RAT USING 01 RAT-REC. 2 ID-KEY. 3 RAT-NO PIC 9(5). 3 SAMPLE-DATE PIC 9(6). 2 AGE-IN-DAYS PIC 9(3) EDIT-STRING ZZ9. -OR- 2 AGE PIC 9(3) QUERY-HEADER "AGE"/"IN"/"DAYS". 2 WEIGHT-GMS PIC 999V9 EDIT-STRING ZZ9.9 ; To rebuild the data dictionary QUERY.DIC in the event it becomes "wiped out", I would first preallocate a new dictionary file. Then I would invoke an indirect command file called DICTION.ARY in DATATRIEVE (see below) which would then cause the specific command files containing the DATATRIEVE definitions to be invoked, with the result that the definitions would be added to the data dictionary. DICTION.ARY --------------- @RATS.DOM @RAT.REC @RATPRINT.PRO DTR> @DICTION.ARY I also keep printouts of all DATATRIEVE definitions and documentation on each indexed file accessed by DATATRIEVE in a set of notebooks for easy reference. These notebooks are then updated to reflect recent modifications to any definition or indexed file. Another job os mine as DATATRIEVE administrator is to design and maintain indexed files. The first step, in the case of a new file, is to determine the indexed file attributes including bucket size, keys, format and the initial file size needed. The formulas and rules that I follow in defingin an indexed file come from APPENDEX D of the PMS-11 MACRO PROGRAMMERS REFERENCE MANUAL, RMS-11 UTILITIES GUIDE and from notes obtained from RMS presentations at DECUS. Once I terermine the indexed file attributes I then use the RMS-11 Define utility (DFN) to preallocate the indexed file. I would like to mention that the Version 1.5 Define utility the RMS people at DECUS were so proud of is not currently available for those using IAS. So in the meantime the 26 indexed files I create may not be optimal since I cannot, for example, define multiple areas for my files, nor can I specify a fill number in order to leave room in bucket (after initially populating the file) for future additions. There are essentially three ways to populate an indexed file that has been preallocated. One way is through the use of a data entry program. Another method is to use the RMS-11 convert utility (CNV) to merge all records from an existing file into the llpreallocated indexed file. The last method involves the data restructuring technique presented in chapter 7 of the USERS GUIDE TO DATATRIEVE-11. This technique enables me to move certain records or portions of certain records from another file into the preallocated indexed file. Most of the time I use the RMS-11 convert utility to populate an indexed file. Once I populate the preallocated indexed file I use DATATRIEVE to print out different collections of records in the new file to verify the successfulness of the file populating process. I also use DATATRIEVE to print out the count of all records in the new file. I am also responsible for maintaining all indexed files accessed by DATATRIEVE. This involves backing up all indexed files from disk onto tape about once a week. To back up all indexed files I use the RMS-11 Backup utility (BCK). Backups of all indexed files are available for at least the last three weeks. When necessary I may need to restore an indexed file from tape to disk, and to do this I use the RMS-11 Restore utility (RST). During backups, if someone is writing to a particular bucket in an indexed file when the BCK utility is attempting to access that particular bucket, the BCK utility will "bomb off". Before I perform indexed file backups I change the password tables for each Domain definition by first invoking a command file in DATATRIEVE that deletes all passwork tables for each domain. Then I invoke another command file in DATATRIEVE that sets up a "Read" only passwork table for each Domain. Thus, DATATRIEVE users are given only "Read" access to an indexed file while I am backing up these files. Non-DATATREIVE programs also access indexed files, so in order to keep these programs from opening up an indexed file for write access I have made it mandatory that these programs include a subprogram call to read a data file called OK.DTR that I have on my account. Based on the contents of OK.DTR the subprogram call will open an indexed file for write access if OK.DTR contains the letter "Y", otherwise it will only open the indexed file for read access. So before I perform backups I change the contents of OK.DTR to "N". After I have backed up all indexed files I change the contents of OK.DTR to "Y" and then I invoke the command file that deletes the "read" only password tables for all domains from the data dictionary. Then I invoke another command file in DATATRIEVE that recreates the password tables for all domains to their original state. The total time it takes me to perform the indexed file backups, including changing password tables, is about four hours. 27 It is ideal to be able to determine how full an indexed file is in order to anticipate when it will be necessary to allocate more space to insure optimal file growth. However, due to inadequate RMS utilities, there is no technique for doing this. For each of our large indexed files (15000 records or more) we have created a small (200 blocks) sequential audit file which uses the same record descriptor as the large file. During data entry, our COBOL programs write the records to both the indexed file and to the audit file. In the event the indexed file becomes corrupted I can restore the previous version of the large indexed file that has been backed up on tape and then merge the records from the audit file into the restored file. The audit files usually become full in a month and when this happens, a new empty audit file is created and the old audit file is copied off to a floppy for one months storage. We have encountered corrupt index pointers in some of our indexed files which may go unnoticed for several months. So I use DATATRIEVE to check for corrupt primary and alternate indexed pointers on our larger files that have the greatest potential for encountering this problem. To check for corrupt index pointers I do the following: DTR> READY DOMAIN-NAME DTR> PRINT COUNT OF ALL DOMAIN-NAME WITH KEY-NAME BT START AND FINISH NORMALLY--> COUNT VALUE IS PRINTED OUT ERROR--> BAD RRV RECORD IN INDEXED FILE; FILE MAY BE CORRUPT In the event I find an indexed file with a corrupt index pointer all I do is rebuild the file using the techniques presented above. In conclusion, I would like to say that it took us months to develop a datatrieve management concept. We will probably continue to develop new policies and new procedures for improving the management of datatrieve and of our data files. With our DATATRIEVE MANAGEMENT CONCEPT I find it easer to maintain 432,000 records in 61 indexed files, 71 record definitions, 84 domain definitions and 101 procedure definitions. If you have any questions, please give me a call: 509/946-2350 -or- FTS 444-7511 (ask for 946-2350). 28 QUESTIONS AND ANSWERS We have not received any specific written questions. I neglected to write down most of the questions I have answered on the phone. Feel free to submit questions to this column to : DATATRIEVE Q/A COORDINATOR (for now, Chuck Watson) SAN DIEGO SIG member participation is needed for the Fall DECUS meeting in San Diego. Formal or poster papers are needed. The deadline as listed in the Call for Participation is July 20, 1979. Other DATATRIEVERS are interested in your approach to particular problems, and you will view your application differently after you organize your thoughts on paper. SIG members are needed as volunteer tutors to staff the campground room. We hope to increase this activity to 4 hours per day, so we need more volunteers to contribute 1 hour during the week. Scheduling for his activity will be done at the first SIG meeting in San Diego. Finally, anyone with a Bibliography application running on DATATRIEVE please contact Chuck Watson. Bibliography techniques will be featured in one session at San Diego.