What's Up DOCumentation Robelle Consulting Ltd. 8648 Armstrong Rd., R.R.#6 Langley, B.C. Canada V3A 4P9 Telephone: (604) 888-3666 Telex: 04-352848 Fax: (604) 888-7731 Date: October 12, 1988 From: Robert M. Green, President David J. Greer, Research & Development Michael C. Shumko, Customer Support To: Users of Robelle Software Re: News of the HP 3000, 1988 #6 What You Will Find in This News Memo: News Tidbits HP World, a New Magazine New Release of DBAUDIT Technical Tips SUPRTOOL Performance on MPE XL Version 1.1, Part II Using Powerhouse Inside QEDIT About Robelle Robelle Products: Problems, Solutions, and Suggestions News Tidbits Super-Cartridge. 55 fonts on one cartridge, including numerous sizes of Times Roman and Helvetica. Our August News Memo was produced on the Super-Cartridge. Most fonts are available only in the ASCII Symbol set; that is, no Roman-8 characters for European users. We have modified the PCL codes in QEDIT and SUPRTOOL to allow for this. Available for $699 (US) from IQ Engineering, PO Box 60955, Sunnyvale, CA 94086, telephone (408) 733-1161. Childcraft / Series 950. Gary Porto at Childcraft reports that with MPE XL 1.1 the problem of a serial task in a batch job hogging the system is not so bad as it was with 1.0. This problem can occur with SUPRTOOL, QUERY, or any long serial task. The batch job still hogs the system, but at least people can get a minimum of work done. With 1.0, they couldn't even get a colon! Gary reports that he has 65 on-line users on his 64-megabyte Series 950 and that the performance is pretty good - as good as his Series 70. That doesn't sound too bad at all. Dynamic Cache Optimizer. The mysterious DCO program has been turned into a commercial product by its creators. DCO runs as a background job, wakes up periodically to compare disc caching performance against targets, and adjusts the caching parameters dynamically. DCO will even turn caching off on a disc drive if that would improve overall throughput. Available from Computer Tools and Services, PO Box 11226, 1001 GE Amsterdam, Holland, telephone 02159 45031. Spooling Enhancements. A few years ago, the only spooling news we could report was HP's yearly promise, as yet unfulfilled, that they were writing a new spooler for MPE. Then, Holland House (512-287-3417) came out with Unispool, for managing spool files around a network of several HP 3000s and printers. NSD (415-573-5923) released Job Rescue, a spool file analyzer that allows you to print only those $stdlist files with errors in them. Now Unisom (415-968-7511) has entered the market with Spoolmate, a product that combines most of these features, including pre-print analysis, printer control, forms control, distribution lists, and network support, plus audit reports and archiving. Of course, Holland House has been upgrading Unispool all along with many of the same features, and NSD has announced Spool Rescue, a Spook-like program that allows you to look at open spool files (how far has that report gone?). All of a sudden, we have a broad range of alternatives to the standard MPE spooler. HP World, a New Magazine This summer I have been reading a new magazine for European HP users named HP World, which I recommend highly. It has excellent articles that describe in concrete detail how real users, mostly in Europe, are solving real problems. The September issue has page after page of well-written news from the Orlando meeting. Here are some samples: "Early MPE XL Migration Results. London Business School is not a typical installation. Much of their software is written using double precision floating point Fortran which benefits considerably from the Precision Architecture. MIS Director Gordon Miller says "Our straight line performance is up considerably - one program runs 40 times faster - but the performance gains are very application dependent and cannot be accurately forecast beforehand." Keith Howard of Collier-Jackson in Tampa, Florida participated in the Spectrum beta testing and upgraded from a Series 58 to a Series 950 - quite a leap. One application was found to be 6% slower due to constant switching between compatibility and native modes, but in most circumstances the machine was five to ten times faster than the Series 52 and one batch job ran 53 times faster! Glaxo Export has temporarily deferred delivery on its second and third 950 systems due to implementation problems on the initial machine. John Walsh of Glaxo related at the Scarborough conference that the first processor was due to go live earlier this year, but when the company came to do a volume throughput test, the machine was unable to load the full operational workload of the Glaxo Series 70. ... Glaxo's problem was caused by the MPE XL 1.0 Loader table being too small, but Glaxo now has a copy of version 1.1 and is about to re-run the volume throughput test. "Notes from Orlando. HP promises performance improvment for Precision Architecture over the next five years of 40-50% per year. Some of this will be achieved by further tuning of MPE XL -- version 1.1 is said to be at least 20% faster overall. ... There are no plans to port the HP COPYCAT product to the 900 Series. A disc condense utility [for MPE XL] will not be released before 1990. Until then, the only way to reduce disc fragmentation is via a system reload... The 7933XP and 7935XP disc drives are supported under MPE XL 1.0. 7937XP drives will be supported under version 1.1, but in all cases the XP cache is ignored by MPE XL and the disc drives have similar performance profiles to the non-XP drives...MPE XL is designed to handle file names longer than eight characters and this may be implemented in the future - probably when incompatibilities with MPE V are not so important...MPE XL version 1.2 has already been completed and is about to enter testing. Some of the problems encountered in live use with version 1.0 will therefore not be fixed until release 1.3. "Color Lasers. Rumors abounded at Orlando of a color LaserJet in the pipeline. It appears that the main technical problem yet to be resolved is getting the colors correctly lined up. 2680? Heavy duty laser printer replacements for the 2680 printer are not expected until 1992 and are likely to be PCL-based. High end 25Mhz. According to Bob Puette, HP's Worldwide PC Manager, a high end 386 Vectra is to be launched in early 1989. This is likely to be a 25Mhz version. Portable Update. The [HP version of the] Zenith Portable is to be introduced late autumn. The main changes are HP logos, HP-colored keytops, HP PAM, and HP version of MS-DOS. VPLUS. There are no plans to enhance VPLUS to support 132-column wide terminals, due to technical difficulties. HP World offers free subscriptions to qualified users in Europe, Africa, Israel, and the Near East. Contact them for an application: HP World, 7 The Brow, Friston, Eastbourne, East Sussex, England BN20 0ER. Telephone (44) 3215 2051. Fax (44) 3215 3304. New Release of DBAUDIT DBAUDIT is intended to help system managers keep tabs on HP 3000 database transactions through use of IMAGE logfiles. You can use it to monitor a specific user, a troublesome dataset, a dial-in terminal port, or a suspect application program, and see just what changes are being made to your databases. Because transaction dumps can be sorted by up to five keys, including dataset field values, you can use them as audit reports instead of writing custom programs. DBAUDIT provides informative, readable summary reports of database activity, organized by program, user, dataset, and logon device. Using these reports, you can pinpoint performance bottlenecks in your database applications. At Robelle Consulting Ltd., we continue to improve our products, releasing a new, enhanced version each year. This year, DBAUDIT version 1.8 has many additional features, including full compatibility with MPE XL and the logfiles generated by Turbo IMAGE/XL: * Report pages have page numbers, date, and title; to make your account auditors happy, these are ON by default. * Checkpoint/Restart on the Input command simplifies daily audit. You won't have to change parameters for the standard DBAUDIT reports you run every night; DBAUDIT records the name of the last logfile processed and starts the next audit where it left off. * You can exclude selected datasets from the report, and sort all the transactions for a given process together. * New Show command lists database sets, items, and fields. * Options to drop "no data changed" Updates from the report or to select only null updates. * Two JCWs record the number of inconsistencies in your logfile (usually zero, thank goodness), and the number of incomplete logical transactions. You can use these JCWs in a batch job to decide whether a database recovery is possible and necessary. * Shorthand: @ to select any user or program in a specific account; * refers to most recent dataset, field, or record selected. These and other new features are fully explained in the new User Manual, which is included on the update tape so that users may print as many copies as they like, on a lineprinter or LaserJet. This version of DBAUDIT has an updated help file. All users of DBAUDIT covered by service will receive an update tape automatically. Overseas customers - please allow another month. Patience is a virtue. Technical Tips NOBUF/MR. With disc caching enabled, it is always faster to read many blocks at a time, rather than one at a time, regardless of whether the blocksize is an even multiple of 256 bytes. This is also true of MPE XL, where blocks do not have to start on sector boundaries any more (or even page boundaries). Logging Overhead in Turbo IMAGE. Charles Sullivan has done extensive measurements of the overhead imposed by transaction logging. The last major study of this issue that I remember is the one that Dennis Heidner did on pre-Turbo IMAGE. Charles found overhead in Turbo IMAGE, where Dennis found little in IMAGE. The results are in a paper entitled Developing a Faster Image in the proceedings of the Orlando Conference. Possible reason: posting to the disc more often for rollback recovery? Importance of Memory on MPE XL. Charles also measured the time for basic IMAGE calls on the Series 950 (version 1.0 of MPE XL). His surprising results: the 950 was slightly slower than the 70. But, when he increased the size of main memory on the 950, it was much faster than the 70. These MPE XL results were presented in Orlando, but are not in the proceedings. TouchScreen Tip. Don't you hate it when you point at something on your screen, and your cursor moves? Me too. Press Shift-Ctrl-Menu to turn off the touch feature. [mcs] Taming the HP 3000, the Seminar. Bob Lund, who wrote the excellent performance guidebook, Taming the HP 3000, is now giving seminars on this topic. For example, for $250 US you can spend December 6th, 1988 with Bob in Los Angeles, learning how to make your system perform better. Arranged by GLUG. Call (213) 282-0420 for details. Query Bug. One of our users called to report that Query on MPE version G.B3.02 has a very serious bug. When deleting records from a master dataset, after it finds a record that matches the search criteria, it deletes that record and all synonyms that migrate to that location!. MpexHook'd Programs Suspend. Programs that have been hooked using VESOFT's MPEXHOOK procedure can be suspended for later quick reactivation. This is in addition to everything else that the hooks allow, such as access to MPEX commands, MPE XL redo stacks, and keeping files into another account. Just type %% at any program prompt, and the hooked program will suspend back to QEDIT, where it can be re-activated at any time. SUPRTOOL Performance on MPE XL Version 1.1, Part II Last month we reported our first test results on Version 1.1 of MPE XL, the new improved operating system. We found that for straight file copies version 1.1 was 30% faster than version 1.0 of MPE XL and that SUPRTOOL copied as fast, or faster, than the Native-Mode COPY command provided with MPE XL. Our second test measured the time for SUPRTOOL to read a Turbo IMAGE dataset and check each entry against selection criteria (i.e., an IF command); all entries were selected. This test read 38,000 dataset entries of 90 bytes each, blockfactor 13, reading and writing 19 blocks at a time. In order to get a feel for the relative performance of SUPRTOOL access versus standard Turbo IMAGE/XL access, we did two extra tests where we used the SET PRIVMODE OFF command to force SUPRTOOL to use the DBGET intrinsic for reading the dataset. This comparison may not give the total picture of Turbo IMAGE/XL Native-Mode performance, due to the overhead of the CM switch stubs that SUPRTOOL had to call. :run suprtool.pub.robelle >base cust.karnak >get notes >if sort-item=0 or sort-item<>0 >out $null >xeq MPE XL 1.0 MPE XL 1.1 Cpu Elapsed Cpu Elapsed Suprtool 3.0 (CM) 21 21 22 22 Suprtool 3.0 (OCT) 7 7 6 6 Suprtool DBGET (CM) 59 61 62 62 Suprtool DBGET (OCT) 43 43 In this test, MPE XL version 1.1 was slightly slower than version 1.0. This slight difference could be due to different disc drives and main memory on the test machines. What this test does show is that when your CM program such as SUPRTOOL has a lot of processing to do (not just calling the operating system), using the OCT can dramatically improve the performance. On the new MPE XL, we also measured the portion of SUPRTOOL's time spent to read the dataset: 32 seconds with DBGET and .6 seconds with SUPRTOOL's NOBUF/MR techniques. By subtracting this time from the overall time, we can see how much of the time in SUPRTOOL is spent processing the records (i.e., IF command, data conversions, formatting output record, etc.). What we find in these tests is that using the OCT reduced the internal processing time by 63% to 76%, a significant gain. In order to produce an OCT-version of SUPRTOOL on your machine, log on as MGR.ROBELLE and issue this command: :octcomp suprtool,suproct Now run suproct for an Object-Code-Translator version of SUPRTOOL. SUPRTOOL Performance Test #3 -- Query Extract This test was similar to test #2, but also created an output file of record numbers. This allowed us to compare the time to extract data with Query versus the time to extract data with SUPRTOOL. Although SUPRTOOL is not a full replacement for Query, since it does not generate reports, it is a replacement for the use of Query as a data extract tool. MPE XL 1.1 Cpu Elapsed Suprtool 3.0 (CM) 22 22 Suprtool 3.0 (OCT) 6 6 Query (CM) 181 181 Query (OCT) 143 145 Notice that the SUPRTOOL results are basically identical to test #2, showing that the time to write the output file of record numbers is negligible. The OCT version of Query took 24 times longer than SUPRTOOL and the CM version of Query took 8.2 times longer. SUPRTOOL Performance Test #4 -- Sorted Extract We did a more exhaustive and realistic set of tests using SUPRTOOL to create a sorted extract file versus using Query to do the same task. We selected about 1/3 of the entries in the dataset, we sorted by three keys, and we converted the fields to ASCII format for the output file. We designed this test to be a "typical" data extract, where the sorted file would then be fed into a report program for final reporting. :build x2;rec=-96,8,f,ascii;disc=40000 :file qslist=x2,old;dev=disc SUPRTOOL Commands: >base cust.karnak,5 >get notes >if sort-item <= 10150 >output *qslist,erase,ascii >sort serial-no >sort sort-item >exit Query Commands: >base=cust.karnak >mode=5 >set=notes >find sort-item <= 10150 >out=lp >report s2,serial-no s1,sort-item d1,-serial-no,4 d1,product-code,6 d1,text-line,86 d1,sort-item,96 nopage end >exit We ran this test on both a 950 and a Series 37, for comparison. Here is what we found. MPE V U-Mit MPE XL 1.1 Series 37, 2 mbyte Series 950, 32 mbyte Cpu Elapsed Cpu Elapsed Suprtool 3.0 (CM) 126 174 36 36 Suprtool 3.0 (OCT) 23 27 Query (CM) 1227 1380 226 230 Query (OCT) 171 180 On the 950, SUPRTOOL CM was 6.3 times faster than Query CM and SUPRTOOL OCT was 6.6 times faster than Query OCT. On the Series 37, SUPRTOOL was 8 times faster than Query and used 10% of the cpu time. SUPRTOOL on the 950 was much faster than SUPRTOOL on the Series 37, as you would expect: 5 times faster for CM and 6.4 times faster for OCT. What is more surprising is that SUPRTOOL on a Series 37 outperformed Query CM on a 950 by 24% and even beat Query OCT by 3%. Using Powerhouse Inside QEDIT QEDIT works well as a homebase for Powerhouse developers for two reasons: 1) the Powerhouse tools read QEDIT workfiles, and 2) the Powerhouse tools have an option to SUSPEND. You can start by running QEDIT, then use UDCs to run QUIZ, QUICK, QTP, QDD, and QSHOW, holding on to each as you EXIT. The result is that all the Powerhouse tools you need are available for instant activation. You can OPEN a QUIZ report file, modify it, invoke QUIZ to test it, OPEN a related QUICK source file, edit it, invoke QUICK to test it, and so on. All without leaving QEDIT and all without waiting. For Powerhouse 5.01 and the next release, the UDCs are distributed as the file ddudc.current.cognos or ddrudc.current.cognos (if you use Dictionary/3000). The Powerhouse UDCs you may need are QUIZ, QUICK, QTP, QDD, and QSHOW. You should modify these UDCs for best use inside QEDIT. The first change is to add a new parameter, make it the first one, to specify the name of the source file. If you do this, you will be able to invoke the tool on your current workfile without having to type the name of the file. In QEDIT, QUIZ * means invoke the QUIZ UDC and pass the current filename as the first parameter. There are two ways to specify the source file- name: 1) via a FILE equation such as FILE QUIZUSE = !source, or 2) via the AUTO= parameter of the INFO string (e.g., info = "auto = !source". For example, QQUIZ source ... file quizuse=!source The filename for QUICK is QKGO, but the others are obvious: QTPUSE, QDDUSE, and QSHOWUSE. The second change is to specify the SUSPEND parameter in the INFO string (e.g., info = "suspend" or info = "suspend auto = !source"). This will cause the tool to suspend instead of terminate when you EXIT back to QEDIT. QEDIT will ask if it is okay to hold onto the tool, and you should say YES if you might need it again. The next time you specify the UDC, QEDIT will not have to run the tool, only activate it, which is much faster. With these two changes, you will have created an integrated development environment for QEDIT and Powerhouse. About Robelle OEMs, VARs, VABs! Do You Need SUPRTOOL? Please call us if you would like to integrate SUPRTOOL or Speed Demon into your application package. We now have a licensing program that allows you to make the speed and convenience of our tools available to your customers. There is no need for your customers to sign a separate license agreement with us, or pay us any extra support fees. They remain your customers, supported by you. User Manuals. As you probably know, every Robelle distribution tape contains the User Manuals as files that you can print on your system lineprinter. There are two things you may not know, however: 1) you can also print those manuals on the LaserJet with proportional spacing and bold/italics, and 2) you can order printed and bound copies of the manuals if you prefer. The price for printed manuals is $20 US each. Below are the instructions for printing your own copies on a LaserJet: If you have an HP LaserJet and one of the Font Cartridges for which we provide a configuring Include file, you can print the user manual on it with Bold, Italics, Underlining, and Proportional Spacing (certain fonts only). Do listf @.qlibdata.robelle and see if there is an Include file whose name is the same as your Font Cartridge. For example, F92286F is for the "F" cartridge with Times Roman and Helvetica fonts. You will need to know the Include filename, the Device Class of your LaserJet, and the number of copies you want to print. :run qedit.pub.robelle /text manual.doc.robelle /find ".out(lpt" 1 {find OUT command for LPT} /change 1 ".com" {and Comment it out} /find ".out(las" 1 {find OUT command for Laser} /change ".com."." {and Comment it in!} If the LaserJet is attached to your CRT: /change "r-"r+" {enable Record Mode printing} If you want more than one copy: /change "c1"c3" {where 3 is number of copies} /find ".inc" 1 {find the Include file} /modify {insert name of Include file for your Font Cartridge} /file print;dev=devclass {use your device class} /set udc udc.catalog.robelle {defines :PROSE command} /prose *,*print {prepare for a long wait!} /exit The Table of Contents prints last, and must be manually inserted after the title page. For more information on printing documents with Prose, see the Prose User Manual (i.e., prose.qlibdoc.robelle), itself formatted with Prose. If you don't have a Font Cartridge that we support, you can still print the User Manual on your LaserJet -- just follow the instructions for a regular lineprinter. Robelle Products: Problems, Solutions, and Suggestions MPE XL and Security on the Robelle Account. The security of the Robelle account does not give you Read access to the PUB group, only Xeq. A bug in MPE XL forces certain operating system functions to have Read access to the program file. Otherwise, the system may fail! To adjust the Robelle account, enter these commands: :hello mgr.robelle :altgroup pub;access=(r,x,l:any;w,a,s:ac) QEDIT Version 3.6 Migration Toolset Bug. If you are preparing to upgrade to a 925 or 950, HP may run the "Migration Toolset" on your Classic 3000 to analyze programs for compatibility. QEDIT may abort with a privileged instruction trap (program error #6) when you press Control-Y. This is a bug in the Migration Toolset monitor, as QEDIT never uses PM. Workaround: turn off the toolset, :allocate qedit, turn on the toolset again. SET MODIFY VEMODIFY TAE ON. If you see V or 87TR1 printed on your screen when you do a Modify command, it means you are using QZMODIFY with the `Type-Ahead Engine' feature enabled, but don't really have a Telamon Type-Ahead Engine. To disable this, type Ctrl-W Ctrl-T once or twice until QZMODIFY says "no TypeAhead Engine". Then continue modifying your line as usual. DBAUDIT Version 1.8 Spring Ahead, Fall Back. When you set the clock back one hour at the end of October, don't forget that your IMAGE transaction logfile may have a sequence inconsistency in it. This is because the timestamp is supposed to increase from one transaction to the next. But any transactions that occur during the `repeated' hour will have earlier timestamps than those that happened just before the clock was changed. This is important to remember, especially if you use DBAUDIT to report on your logfiles, or if you have a system failure and need to recover from that logfile using DBRECOV. DBAUDIT can ignore the sequence error using set resetonerror off. DBRECOV ignores the timestamp inconsistency using control nostamp. QLIB Version 5.0 DESKQED. Version 0.3 uses the file DESQSCR instead of QEDITSCR as the QEDIT scratch file. PROSE. We have improved command descriptions in the User Manual. QCOPY. Version 3.4 fixes a number of bugs, especially reading files with variable-length records. Such files are now rejected by QCOPY. QHELP. Version 1.4 fixes a bug where QHELP was not closing the LP file when you did a CLOSE command.