What's Up DOCumentation Robelle Consulting Ltd. 8648 Armstrong Rd., R.R.#6 Langley, B.C. Canada V3A 4P9 Telephone: (604) 888-3666 Telex: 04-352848 Date: June 25, 1988 From: Robert M. Green, President David J. Greer, Research & Development Michael C. Shumko, Customer Support To: Users of Robelle Software Re: News of the HP 3000, 1988 #4 What You Will Find in This News Memo: News Tidbits Technical Tips About Robelle Squeezing the Last Bit Out of Your HP 3000 Part Six: Get OMNIDEX for Fast On-line Database Searching European Challenge From SI-IMAGE Using QEDIT for the Occasional Memo Robelle Products: Problems, Solutions, and Suggestions News Tidbits Swedish Users Conference. The recent users conference in Sweden was a great success. This was the first conference to present any real papers on Spectrum performance. Three of us from Robelle attended: Marie Froese, Bob Green, and Mike Shumko, and we had a great time. Many thanks to the conference organizers. Marie set up and ran our booth very capably and looked after the requirements of our European users and prospects. Bob presented a new paper, "The Spectrum Instruction Set: A 3000 Hacker's View", which he will also present at Orlando in August. Hewlett-Packard sponsored a surprise evening on Monday, which consisted of a boat ride to a remote island topped by an immense stone fortress. After climbing about a mile to the peak of the island's fortress, we 900 attendees were treated to a Medieval feast (wine from horns, chunks of lamb and beef eaten from boards, strolling minstrels, etc.). Robelle sponsored Wednesday evening's entertainment for the conference, consisting of a Fishing Boat Party and a visit to the Liseberg amusement park. At Liseberg, Mike Shumko gathered together all the devotees of roller coasters and founded Sig Thrill. SUPRTOOL Performance on 950. We have our first two results from users who have run our suggested SUPRTOOL performance tests on the Spectrum. Please keep in mind that these results are extremely limited -- only two sites, only batch processing -- and may not be transferable to other applications. Leonard Berkowitz of Warren, Gorham & Lamont was the first user to send in test results. He used SUPRTOOL to get 100,000 records from an IMAGE detail dataset with 200-byte records, selecting entries with IF and selecting fields with EXTRACT. He ran the test with and without a SORT field and on both his Series 70 and his Series 950 with 64 megabytes of main memory. The only thing non-typical of his test was that his IF command selected every one of the 100,000 entries read. His results: Task CPU Time (Seconds) Elapsed Time (mm:ss) S/70 S/950 %Chg S/70 S/950 %Chg Extract 51 105 +106% 3:48 2:42 -30% Extract Plus Sort 324 293 -10% 19:22 7:36 -62% For the straight extract, the 950 used twice as much CPU time but completed 30% faster. For the sort task, the 950 used 10% less CPU time and completed in about one third the elapsed time (62% faster). The elapsed time actually spent in the sort routines was reduced from 1151 seconds to 424 seconds (63% less). These tests used MPE XL version A.01.10 and a CM version of SUPRTOOL (not an OCT version). P.A. Vickers of Mallison-Denny Limited also tried our tests on his new 950. His test was similar to the previous one, but his record size was 158 bytes and he selected 157,000 out of 157,000 entries (apparently it isn't as easy as we thought to make up an IF command that will select a subset of a dataset). He also tested the OCT version of SUPRTOOL. Task CPU Time (Seconds) Elapsed Time (mm:ss) S/70 950-CM 950-OCT S/70 950-CM 950-OCT Extract 46 99 36 2:02 1:48 1:01 Extract + Sort 394 334 265 19:55 8:41 6:34 The simple extract without sorting reported 115% more CPU time on the 950 (in compatibility mode), but completed 11% faster. After SUPRTOOL had been run through the Object Code Translater, it reported 22% less CPU time than on the 70 and completed in half the time of the 70 (50% faster). When sorting, the 950-CM version of SUPRTOOL cut Series 70 CPU time by 15% and elapsed time by 56%. The 950-OCT version of SUPRTOOL was even faster, with CPU down by 33% and elapsed down by 67%. From these two tests, it appears the sorting is significantly faster on the 950 and that the OCT does produce a faster program. It is not clear that CPU time can be directly compared between the Series 70 and the Series 950 -- the methods of allocating CPU time to processes may differ between MPE V and MPE XL. A third user called to complain that SUPRTOOL was too fast on his 950 - when it got going it was locking out the on-line users. It appears that this is a bug in the current version of MPE XL that can occur with many programs that do long serial reads. Falcon Disc Drives. One customer installed four EMC Falcon disc drives on his Series 68. He kept getting short hangs which would clear themselves after a few seconds. He also would get long hangs which would go away when he typed DOMAIN commands. EMC looked into the problem and found the answer: the user had a 7933 and two Falcon drives on a high-speed GIC, with the other two Falcon drives on a slow-speed GIC. No one had been aware that there were low- and high-speed GICs. The slow-speed GIC could not handle the throughput of the two Falcon drives. Moving a Laser Printer from the high- to low-speed GICs and the two Falcon drives from the low- to high-speed GICs solved the problem. The user has also discovered that setting the domain to 1024 causes SUPRTOOL batch jobs to crawl, but domain of 9000 makes SUPRTOOL really smoke. His initial benchmarks showed that on his system the Falcons were just a little slower than HP's Eagle disc drives. But the lower price of the Falcons tipped the scales in their favor. [Tom Smyth] HP Vectra RS/20 Wins 386 Shootout! HP finally has a winner in its PC line, according to PC Magazine, June 28, 1988. In a comparison of 13 high-end 386 PCs (20 mhz), the Vectra RS/20 was the winner and only "Editor's Choice", beating out the IBM PS/2 Model 80-111 and the Compaq Deskpro 386/20. PC Magazine summarizes its choice as follows: "The HP Vectra RS/20 is a breakthrough for Hewlett-Packard. With its incredibly fast drives, generous expansion space, and rock-solid construction, the RS/20 stands a step above its competitors. Of course, high quality often carries a high price tag, and the $9,995 (with 150MB hard disk) RS/20 follows the rule." The reviewers were particularly impressed with the disc-caching scheme used in the Vectra. [Could the experience with MPE disc caching have helped?] "The RS/20 starts with an 18-millisecond drive and uses caching software to quadruple its performance on our small-record DOS File Access test, give it more than twice the speed of any other drive tested...Hewlett-Packard's Vectra Disk Cache Program gives the user total control without being overwhelmed and yields the best power boost of the lot...When writes are made, they are collected in cache memory until the next disk rotation brings the correct sector to the read/write head. Volatility is reduced to one disk rotation, with a gain in write speeds of 100 percent." [Write cache is optional.] DCO Now Supported. DCO, Dynamic Cache Optimizer, is a performance booster that we have mentioned in previous What's Up, Docs, and in our award-winning paper, Squeezing the Last Bit Out of Your HP 3000. Until now, DCO has been available only from HP performance specialists on an as-needed unsupported basis. DCO is now available as a fully supported third-party tool from Computer Tools & Services in Holland. CTS's glossy advertisement features the catch phrase, `Squeeze the last bit out of your HP 3000'. Hmmm. Sounds familiar. Quick Reference Card From HP. A new card from HP, the Precision Architecture and Instruction Reference Card has the Spectrum instruction set on it. HP part number 09740-90027. Smallest Spectrum Yet? HP is rumored to be preparing a new, smaller MPE XL machine, code-named "Burgundy", for release as the Series 810 and the Series 915. The CPU is smaller than a Micro-3000 and fits easily on your desk. No word on performance yet. Technical Tips HP Slate Gotcha. If you try to convert an MPE file named SLLIST into Slate format, you will get a system failure 8609. [Jayvis Rowland] HP 150 Insert With Wrap. Shift-Insert-Char on the HP 150 is the same as on the 2393 -- it gives you Insert Mode with character wrap. That is, if you insert so many characters that the line will overflow, characters are wrapped from the end of the line to the next line. Shift-Delete-Char does the opposite, pulling up characters from the next line as you delete characters from the current line. MPE XL vs MPE V File Compatibility. In a recent news memo, we discussed the potential compatibility problems that can occur because MPE XL files can have more than 32 extents and because CM programs are created on MPE XL with two or more extents, instead of just one. Now Evan Rudderow reports that you can avoid these problems when moving files between systems if you use DSCOPY for the copy and use the TRANSPORT option when you STORE files on MPE XL to be RESTOREd on MPE V. MPE XL Tip From VESOFT. Eugene Volokh of VESOFT didn't like the Native-Mode stack trace that the Spectrum produces when a Compatibility-Mode program aborts. Eugene found that the :Setdump command has new options on MPE XL: :setdump ;debug="debug string" Eugune uses the following command before running CM programs: :setdump ;debug="cm;tr" The "cm;tr" parameter forces a CM Symbolic Trace using the FPMAP (when available) on aborts. In fact, you can put any System Debugger commands in the :Setdump string. Saving Disc Space on MPE XL. When you close a file on MPE XL and the EOF is less than the LIMIT, you can give back any disc space beyond the EOF (without reducing the LIMIT) by settings bits (11:2) of the disposition parameter of FCLOSE to 2. This is useful for small data files which will otherwise occupy a minimum of 250 to 500 sectors, regardless of what you specified for EXTENTs. Conditional USE Files. QEDIT version 3.6 supports the :IF and :ELSE commands in USE files as well as UDCs. Qedit commands are always executed, regardless of the :IF and :ELSE commands. This means that you can create USE files that compile your program, but only :PREP it if there are no errors and only :RUN it if the :PREP is successful. For example, comp.use shut;comment close the current workfile, if any purge report1.usl purge $oldpass build temp.david;rec=-80,16,f,ascii; file l=temp.david,old;acc=append setjcw jcw=0 spl report1.source,,*l if jcw=0 then save $oldpass,report1.usl file qmaplist=report1.qmap,old;dev=disc purge report1.qmap build report1.qmap;rec=-81,49,f,ascii;cctl purge report1.pub prep report1.usl,report1.pub;qmap;maxdata=30000;cap=ph if jcw=0 then run report1.pub;lib=p else display Prep of Report1 failed! Run skipped. endif else display Compile of Report1 failed! Prep skipped. endif reset qmaplist reset l open *;comment re-open source file for editing "Pause" Feature. Randy Medd of Telamon reports an extremely odd but useful feature of the PAUSE intrinsic. If you call PAUSE to delay for a time (say 10 seconds) and you have Control-Y enabled, you can do the following. In your Control-Y trap routine, put a zero into the variable that you passed to the PAUSE intrinsic. That action will immediately cancel the "pause" upon exit from the Control-Y routine. A side effect of this feature is that after Control-Y the pause function starts over with whatever value is now in the variable that you originally passed to the PAUSE intrinsic. I always wondered why this parameter was passed by reference instead of by value. You would be wise to ensure that it is global and not local; otherwise, the value after Control-Y could be random! Running HPDESK Within QEDIT. Here is an undocumented feature of HPDESK. We are not certain which release of HPDESK this is in, so don't complain if it doesn't work on your machine. Go into the workarea and create an item named "suspend". The item should contain one line "&suspend". At any prompt, type "xeq suspend". E.g., in tray> xeq suspend If you are running inside QEDIT, HPDESK will suspend. You can re-enter with /:a hpmail About Robelle Party in Orlando. If you are coming to the Interex Conference in Orlando, Florida this August, don't forget the Robelle User Party on Monday night (August 8th). All Robelle users and guests are invited to drop in for appetizers and drinks, in congenial surroundings. Time: 8:00 PM. Place: Grand Ballroom (next door to the vendor fight for 1989 booth space). Squeezing the Last Bit Out of Your HP 3000 Part Six: Get OMNIDEX for Fast On-line Database Searching Last year we did an informal survey of large HP shops to find out how the successful ones avoid topping out the HP 3000 line. An unusual percentage of these sites had OMNIDEX for fast on-line database searching. IMAGE provides calculated read, chained read, and serial read. OMNIDEX, a software package from DISC, adds record selection across multiple fields, generic retrieval and sorted sequential access, multiple keys in masters, and keyword retrieval on text data. It does this by adding another structure to IMAGE's: the binary tree. Traversing this tree is fast. Once some data is reasonably static (i.e., you no longer modify it (much), but you need to ask numerous complex questions about it), OMNIDEX gives you the ability to index everything. The primary cost is disc space. Users also apply OMNIDEX to replace KSAM. The index-sequential part of OMNIDEX (called IMSAM) will reindex about one million keys per hour on a Series 70 (versus 20 to 30 hours with KSAM). Hewlett-Packard uses OMNIDEX in the Response Center to index bug reports. That is how they can find out instantly who else has had a system failure 916 on Series 37 under T-MIT with a full moon. OMNIDEX indexes every word, not just the manually-assigned "keywords" as in the old SSB system. Doug Iles of HP says, "We could enter partial values and/or full values from several different fields and find five qualifying records out of 50,000 in seconds." Consolidated Capital. Kim Everingham reports that they use OMNIDEX extensively in their tracking system for investors and investments. The power of OMNIDEX indexing allows their offical IMAGE structure to be very simple: masters for entities and details for transactions. They have 4.5 million sectors of data, 250 QUICK screens, 12-15 databases, and 35-40 users on a Series 70. Without OMNIDEX the application would require an IBM mainframe. Within 1 or 2 seconds they can identify an investor and the investments he is involved with, even if the investor gives only a vague or partial description of himself (e.g., trust company, Ralph, Minneapolis). They do all updates on-line, including updates of the OMNIDEX indeces; the only exception is the entry of new investments -- that is done in a nightly batch job due to the serious impact on response. They have plenty of horsepower with the Series 70. Kitsap County Government is an HP site that gets a lot of work done without hitting the limits of the HP 3000 line. Jim Kellam, the manager, started with a Series 48, overloaded it, then added a Series 68 and left the 48 for development. He reports that OMNIDEX inquiries are unbelievably fast (`find all the voters named Smith' instantly replies `1200 entries found'), but can be abused, just like any tool. For example, one of their programs opens all eight databases at the start, in case you might need them. Installing OMNIDEX implies an extra open and another extra data segment, the equivalent of 16 DBOPENs per user. The users sometimes get in and out of the application to access other software, so they pay this startup overhead more than once per day. The IMSAM part of OMNIDEX allows you to define concatenated keys with pieces from three different datasets. Jim feels that they may have overused these features, because he observes slow response with some of these bizarre keys. European Challenge From SI-IMAGE An IMAGE database can provide you with the Customer Master information if you provide the exact Customer Number or Name, but it cannot quickly provide you with all the Customers whose Names start with "ARMST". An IMAGE database can provide you with all the Orders for a given Part Number, but it cannot easily provide them in alphabetic order by Customer Name. North American users are about to have their choices expanded when it comes to adding access capabilities to their IMAGE databases. For several years, the German company IABG has been selling and enhancing its product SI-IMAGE in Europe. SI-IMAGE adds indexed sequential, generic key, and multikey access to IMAGE applications. IABG has a large number of installations in West Germany, including a deal with HP to include SI-IMAGE in HP's financial package FA/3000. (Rumor has it that the latest version of FA/3000 has SI-IMAGE built into it, but HP does not tell all customers about it -- only the ones who complain.) Until now, the only serious choice for this type of access in North America was OMNIDEX, a fine product we have described in our "Squeezing" paper. Now IABG has struck a deal with one of the existing third-party tool vendors to market SI-IMAGE in the United States, perhaps world-wide. The competition should improve both products: in order to attract a US distributor, IABG had to add keyword searching and a Powerhouse interface to SI-IMAGE. SI-IMAGE works by adding new modes to the DBFIND and other intrinsics. For example, if you call DBFIND with Mode -110, SI-IMAGE constructs a temporary chain consisting of all entries where the first ten bytes of the key value matches the specified search value. Mode -120 matches for 20 bytes, etc. You then use normal DBGET calls to retrieve the records. Modes 15 and 16 of DBGET allow you to retrieve entries which are greater than or less than the specified search value. To provide this extra access capability, SI-IMAGE maintains B-trees within a special dataset that it adds to your database. An experienced SI-IMAGE user whom we interviewed was very positive about the product, pointing out that with B-trees you do not need master datasets at all. This user found the software reliable and the performance excellent. For more information on SI-IMAGE, contact IABG in West Germany (telephone 089/60 88-30 77) or watch for the US introduction of the product, probably under a new name. Using QEDIT for the Occasional Memo Some users would like to use QEDIT's full-screen editing to write an occasional memo, but find the range of features and options in QEDIT daunting. You can make QEDIT more accessible to such users by providing them with a special UDC and USE file (see listings below). :QMEMO [filename] To invoke QEDIT to create a new memo, use :QMEMO without any parameters. QEDIT will be invoked directly in VISUAL mode, with configuration options suitable for processing a memo. Any existing QEDITSCR file is purged, so that the user will not be faced with an "Okay to clear?" message. When the user presses f8 to exit VISUAL, QEDIT will update the screen, KEEP the new document as the file Memo in the logon group, and ask the user to verify Exit. To invoke QEDIT to modify an existing memo, use :QMEMO filename, where filename is a KEEP file or QEDIT file. When the user presses f8, QEDIT will KEEP the file back into filename, asking if it is "Okay to Purge?". The users should answer YES to save their work or NO to discard it. The text file is configured for 74 columns so that it will fit exactly in the columns available in VISUAL mode. VISUAL mode will save and restore the user's function key labels, if any, and will automatically update the screen when the user presses a function key. You will need to show first-time users how to use DEL-LINE and INS-LINE (etc.) to edit the screen, and f5 or f6 to move backward or forward. You should also teach them the MPE :RENAME command so that they can rename the file Memo. Later you might show them how to use CHANGE and JUSTIFY in the command line (p.s., JUSTIFY is configured with right margin in column 68). Here is the listing of the QMEMO UDC: QMEMO filename = "$NULL" file memofile=!filename run qedit.pub.robelle;parm=68;info="UQ usememo.catalog.robelle" reset memofile *** The QMEMO UDC requires a USE file to drive QEDIT: :comment Usememo.Catalog.Robelle. QEDIT USE file for QMEMO UDC. set list page on set window (upshift) set wraparound on set tabs hp on set shift up 2 down 2 set check on set vis update on save on set justify margin 68 twospaces on :setjcw rlabeldefault=2 set udc open set lang text set length 74 set keep name memo text *memofile set keep num off vis first keep exit Robelle Products: Problems, Solutions, and Suggestions SUPRTOOL Version 3.0 USE Files for DEFINEs. SUPRTOOL DEFINE commands are wonderful for databases where the IMAGE items have sub-structure. Combining the DEFINE command with USE files adds power to SUPRTOOL. You can put all the defines for a dataset or file in a separate USE file, and you can USE it in SUPRTOOL anytime you access that file or dataset (not just at the start of SUPRTOOL, and you don't leave SUPRTOOL after a USE). Remember that DEFINE names can be up to 16 characters, so you can usually make the names in each USE file unique. This allows you to activate several USE files of DEFINEs at the same time without conflict! [Longs Drug Stores] QEDIT Version 3.6 Training Notes. We enclose an introductory guide for new users of QEDIT. It is intended for your novice, non-technical, or occasional users of QEDIT. The QEDIT reference manual can be intimidating to beginners, who may not need all the advanced features. We would like to hear your comments on the training notes. Do you have users it can help? Is it basic enough, or too advanced? Editing Roman-8 Characters. In Sweden recently, several users asked why they saw only dots (.) on the screen when they tried to edit accented European characters using QEDIT VISUAL. The reason is that these are Roman-8 characters; they have decimal values greater than 128 because they use the first bit of the character as a data bit instead of a parity bit. By default, QEDIT thinks that these characters are line noise. However, there is a SET option to inform QEDIT that these are valid characters. Use /set editinput extend on We are considering changing QEDIT so that this option is ON by default. Limiting QEDIT to 80 Columns. When you use VISUAL with Reflection, QEDIT 3.6 automatically widens display memory to accommodate the widest possible line in your file. For Language = Job files, this means that QEDIT will never print the "!" warning at the start of lines which are more than 76 columns long. If you find the "!" to be a useful warning, you can get it back by using this undocumented SET option: /set visual rlimit on HowMessy Version 1.6 Please apply the following patch to version 1.6 of HowMessy. This makes HowMessy a little less paranoid in its database integrity-checking. :hello mgr.robelle,pub :run patch.pub.sys Filename: HowMessy ?m,5,1050 141205,141605 ?{Return} QLIB Version 4.7 QCOPY 3.3. We have produced a new version of QCOPY with a FTN option. When QCOPY is converting a file from KEEP format to QEDIT format, if the file has 80-byte records with sequence numbers in the last eight columns, QCOPY cannot tell whether the file contains an SPL, Pascal, or FORTRAN program. By default, QCOPY converts such files to SPL-type QEDIT workfiles. With the FTN option, QCOPY converts them into FORTRAN-type QEDIT files. SUE. Bob Borwick at Veteran's Administration has discovered that SUE will try to read a command file named INFILE when you start it up. You can put SUE commands in this file to execute `canned' tasks. SUE will stop when the commands are exhausted.