PRODUCT :  R:BASE                  VERSION    :  4.0
     From Dennis Comfort, Microrim Vice President of Development
     WHAT IS R:BASE 4.0
     R:BASE 4.0 is the latest release of R:BASE from Microrim. It is part 
     of the Microrim Upgrade Express program and subscribers will receive 
     it as part of their annual subscription agreement with Microrim. 
     R:BASE 4.0 is packed full of new features and enhanced capabilities, 
     but perhaps more important than the features, R:BASE 4.0 sports major 
     speed improvements throughout the product. We could devote an entire 
     article just to the new features, and in fact we do later in this issue. 
     This article will focus primarily on the memory management technologies 
     which Microrim employed to marshall the speed improvements in this 
     exciting new release.
     R:BASE 4.0 is a 32 bit, protected mode database management system. 
     As such, it exploits the power and performance of Intel 386 and 486 
     processors by: a) using the 32-bit flat memory model available on 
     those processors to ignore previous memory restrictions like 640K 
     DOS boundaries and segmented memory schemes, and b) using the full 
     processor speeds of the 386 and 486 machines by executing full 32-bit 
     instruction sets. The result is a new R:BASE product that runs much 
     faster than ever before!
     When we say 32 bit, protected mode, we are saying a lot. Let's examine 
     this phrase in pieces to understand what is really different about 
     R:BASE 4.0 versus previous versions of R:BASE as well as other products 
     on the market.
     16 Bits versus 32 Bits - What's this all about?
     R:BASE 3.1C and all of it's predecessors and companion products, and 
     most of the DBMS products that are currently shipping for the PC 
     platform are all based upon the Intel 16 bit architecture. They are 
     all designed to run, regardless of how fast, on all Intel machines. 
     Some 16 bit applications still run on the 8088, others with the 80286, 
     and progressing through the 80386 and 80486. They use 16 bit data and 
     memory pathing in the instruc-tions as part of the program. There's 
     nothing wrong with using 16 bits, and in fact, in all processors below 
     the 80386, that's the only choice available.
     On the 80386 and 80486 processors, a program has the option, given 
     some significant program changes, of running either in the processor's 
     real mode (1MB addressable memory), virtual 86 mode (multiple real mode 
     programs running in multiple virtual V86 machines in the same 1MB 
     address space), or protected mode (up to 4GB of addressable memory). 
     In addition, if the program is running in real mode or V86 mode, it is, 
     by definition, a 16 bit application, meaning that it is restricted in 
     how much addressable memory can be made available. It will only utilize 
     the 16 bit instruction set of the processor. In short, you end up 
     wasting a lot of system facilities (including speed) by only using 16 bit 
     addressing on a 32 bit processor. Furthermore, if your application 
     executes in real mode, you restrict how much memory you can use at one
     time. This results, for larger programs, in having to constantly swap 
     code in and out of the real (conventional) memory, causing some perfor-
     mance degradations. Of course, you can mitigate how much swapping 
     occurs and from where by carefully controlling the amount of available 
     real memory (by using an add-on memory manager), and by controlling 
     where the code swapping takes place (from disk or from memory, 
     depending upon what memory manager you use). So, in a real mode 
     environment, you can get respectable performance, but you are still 
     running your 80386 or 80486 as a real fast 8088/8086 or 80286. You 
     aren't utilizing your machine to its full potential. 
     The chart below shows how R:BASE products use the different processing 
     modes of the 80386 and 80486 computers. For comparison purposes, we 
     have also included Paradox 3.5 and the yet-to-be-shipped Paradox 4.0 
     in this chart.
     To use the 32 bit capability of these machines, it is necessary to 
     modify the product to utilize 32 bit addressing, and then rebuild 
     (compile and link) these products with 32 bit tools (compilers, 
     linkers, debuggers, etc). Until recently, there haven't been too many 
     tools available to do this. Even with tools available, we have been 
     lacking any operating system which is capable of running these appli-
     cations. DOS is a real mode, 16 bit operating system which does not 
     take advantage of technologies such as 32 bit addressing, protected 
     modes of the chips, or flat memory models. On the other hand, OS/2 1.X 
     is a protected mode operating system, but it still runs in 16 bit mode. 
     OS/2 2.0, recently shipping, is a 32 bit operating system which is just 
     now gaining some momentum. Windows 3.0 is an operating system of sorts 
     (which lives on DOS), but also is a 16 bit environment. Windows/NT, 
     scheduled to ship someday, is a full 32 bit operating system that may 
     prove fruitful. But today, most of us live exclusively in DOS. 
     OK, You've convinced me that I'm not getting all I can get from my 
     machine. How do I get to use my 32 bit protected mode capabilities?
     Simple. Use R:BASE 4.0. More good news is that you don't have to do 
     anything special to achieve this. R:BASE 4.0 automatically does all 
     of the work for you to use the full potential of your computer. Let's 
     look a little further into what's behind R:BASE to understand how this 
     is done.
     To build R:BASE 4.0, the development group at Microrim switched from 
     our traditional use of the Microsoft C compiler family (16 bit) to the 
     WATCOM C386 32 bit compiler family. This required significant changes 
     to the R:BASE source code to make use of the 32 bit architecture. The 
     result is an executable which consists of full 32 bit instructions so 
     that the speed and architecture of the 32 bit chips (80386 and 80486) 
     can be exploited. But how do we exploit these chips without a 32 bit 
     operating system?
     Before we can answer this, we need to review a few more concepts. Most 
     of us using PCs use the DOS operating system. DOS, under normal 
     conditions, can address a maximum of 640K of (conventional) memory. 
     This means that our applications must live within the 640K limit. 
     Without going into detail on why, suffice it to say that two of the 
     reasons that there is a 640K boundary are: 1) most applications only 
     use 16 bits to represent addresses, thus restricting ourselves to a 
     640K boundary, and 2) since DOS is running the processor in real mode, 
     our application cannot address anything in memory above the one megabyte 
     As you might expect, there are exceptions to the rules. By using add-on 
     memory managers and DOS 5, we can cheat a little bit and actually 
     squeeze a few more bytes out here and there, and move device drivers 
     (such as network and mouse drivers) to some other memory locations 
     beyond the 640K area. However, for all practical purposes, we, along 
     with our applications, live within the confines of the 640K memory 
     barrier.  Large applications, like database managers (R:BASE 3.1X is 
     an example), are far larger than any 640K limit. Therefore, these appli-
     cations must use sophisticated memory swapping and overlay techniques 
     to constantly move program code around within the 640K bounds while the 
     user goes about his business. The result is performance of products 
     which could be improved if we stop the swapping.
     How do you stop code swapping?
     One alternative is to switch to an operating system which provides a 
     flat memory model and much larger address spaces (including no 640K 
     limits) and use them to run our ap- plications. Doing this presumes 
     that the applications have been altered to run in these environments. 
     An example of an operating system which provides such a memory model is 
     OS/2.  OS/2 1.X is a 16 bit protected mode operating system. Microrim 
     ships an OS/2 version of its R:BASE product which therefore is not bound 
     by the 640K limit. However, it is pretty clear now that the industry is 
     not going to drop DOS in favor of OS/2 1.X. IBM has recently started 
     shipping OS/2 2.0, a 32 bit protected mode operating system that also 
     provides a flat memory model. The acceptance of OS/2 2.0 in the market 
     is growing, but does not as of now dominate the PC market. In the mean 
     time, what do DOS application vendors and users do today to get around 
     these limitations and problems brought on by using 16 bits and real mode 
     Simple. Extend DOS to avoid these limitations.
     From the user perspective, this really is simple because the user 
     doesn't have to do anything. All the work must be done by the vendor. 
     To extend DOS, a vendor (such as Microrim) uses a supplemental product 
     called a DOS extender which 'extends' the operating system so that appli-
     cations can address memory beyond the 640K boundary without having to 
     deal with the mechanisms or penalties of swapping. There are several 
     available on the market today. Fortunately, the existence and use of DOS 
     extenders can be left as an implementation detail for the vendors, and 
     virtually invisible from a user standpoint. Customers don't have to buy, 
     load, run, or use DOS extenders. Even better, the customer doesn't have 
     to load any supplemental memory managers at all (like HIMEM.SYS, EMM386, 
     QEMM, or 386MAX, to name a few). It is the job of the vendor to modify 
     their applications to use a DOS extender, and then ship the extender 
     embedded within their own products. We'll examine how this works in a 
     What is a DOS extender and What does it do?
     A DOS extender is consists of a set of program libraries, some of which 
     reside within the extended application (like R:BASE 4.0), while other 
     pieces run external to R:BASE as real mode programs. Vendors of DOS 
     extenders did not want to write an entire new operating system to 
     replace DOS, but they did have to implement many of the services that 
     an operating system typically provides so that those services could be 
     available to pro-tected mode applications. 
     A typical DOS extender provides the following services to the extended 
     1)  Initialization of operating system and processor registers in 
         preparation to switch the processor from real mode to protected mode.
     2)  Service the interrupts from the extended application by providing the 
         interrupt services or passing the requests on to DOS.
     3)  The ability to load the protected mode application from disk into 
         extended memory.
     4)  Memory allocation/de-allocation functions to create and destroy 
         memory (buffers) for the application. These functions must be 
         provided for not only extended memory, but in some cases, also 
         conventional memory.
     5)  Core extender code to allow for communication between the extended 
         application in protected mode and the DOS extender in real mode.
     6)  Transparent access to DOS and BIOS services via the extender to 
         obtain basic services such as file input/output functions.
     7)  Clean up code to terminate the extended application and switch the 
         processor back into the real mode when the program has terminated.
     These functions are provided to extended applications in the form of 
     both 16-bit and 32-bit DOS extenders. Applications built with a 16-bit 
     DOS extender have a maximum addressable memory range of 16MB. Applications
     which use a 32-bit DOS extender have a range up to 32GB. Applications that
     use 32-bit extenders must also be built with 32-bit compilers. R:BASE 4.0 
     is built using the WATCOM C386 32-bit compiler and the WATCOM version of 
     the Rational Systems' 32-bit DOS extender, DOS4GW.
     This completes the basic tutorial on 16-bit usage versus 32-bit usage 
     as well as an introduction to extending DOS. Let's proceed with some 
     specifics on how R:BASE 4.0 operates in this environment.
     Loading R:BASE 4.0 - Why does it take so long?
     When you type RBASE at the DOS prompt, it will take approximately 30 
     seconds for R:BASE to load and be ready to use on your computer. This 
     is considerably longer than it has taken in previous versions of R:BASE. 
     Is something wrong? Why does it do this?
     To answer the first question, nothing is wrong. There is a lot going on 
     behind the scenes after you type RBASE. Before looking at what happens, 
     we should first understand the typical memory layout of a 80386 or 80486 
     computer. Refer to figure 1 below.
                         |   |                    |       4Mb
                         |   |                    |
            Extended Memory  |                    |
                         |   |                    |
                         |   |                    |
                         |   |                    |
       Upper Memory Area  -->|                    |       1Mb
       Video Buffers, etc.   |                    |
                         |   |                    |      640k
        Conventional Memory  |                    |
                         |   +--------------------+
                         |   |        DOS         |
     Figure 1.
     This figure shows the basic memory layout of the computer. It assumes 
     that the only software loaded is DOS, with no special considerations 
     for loading any drivers or tools high in memory. In this diagram, you 
     can see that the conventional memory address space ranges from 0K to 
     640K. It is within this space that DOS typically resides, along with 
     standard 16 bit applications. (Much of DOS 5 can be loaded higher into 
     memory, but it is not germain to this discussion and will not be 
     considered in this article). Above the 640K boundary is a 384K area 
     called the Upper Memory Area. This space has reserved memory locations 
     for the video buffers required for the various video cards (for example, 
     there are reserved locations for monochrome displays, EGA displays, and 
     VGA displays). However, there are spots within this 384K that can be 
     made available to DOS or applications. Both conventional memory and 
     upper memory reside below the 1MB threshold, and therefore is classified 
     as real memory, because it is the only memory that can be directly 
     addressed by the processor when in the real processing mode.
     Above the 1MB boundary is the computer's extended memory. This address 
     starts at approximately 1MB, and goes up to the limit of the memory 
     installed on your machine. Therefore, if you have a computer that has 
     4MB of physical memory installed, then the first 1 MB is directly 
     addressable by DOS and DOS applications as real memory, and the 
     remaining 3 MB is not addressable by DOS or DOS applications unless 
     a) there are special device drivers loaded to use it (like ramdisk 
     drivers or memory managers), or b) there is a protected mode DOS-
     extended application running, capable of addressing that mem-ory. Note 
     that there are several memory managers which can be loaded to provide 
     access to the upper memory area and an area just above that called the 
     high memory area. We will not discuss those issues in this article, as 
     their existence is not necessary for an understanding of what is 
     happening with R:BASE 4.0.
     Now that we have an understanding of the general types of memory and 
     their location, let's see what happens when we load R:BASE (see 
     figure 2). When you type RBASE at the DOS prompt, DOS loads what it 
     thinks is the RBASE program from the RBASE.EXE file. In actuality, it 
     is loading an R:BASE stub program from the front of the executable file. 
     Step 1 shows the stub and DOS loaded into conventional memory. This stub 
     is quite small and is always resident until the user exits from R:BASE. 
     The stub performs a number of initialization functions, including 
     checking to see whether or not certain command line switches were 
     present on the RBASE command line. For example, if the user supplies a 
     -R switch, then the stub will not load the signon screen, and therefore 
     not load the RSIGNON program. If the user omits -R switch from the 
     command line (and from the RBASE.CFG), then the stub will load the 
     RSIGNON program into conventional memory, wait for it to complete, and 
     then the space occupied by RSIGNON program will be released back to DOS 
     and the conventional memory pool. (See Figure 2 below.)
               |  |            ||            ||            ||            |
               |  |            ||            ||            || PROTECTED  |
               |  |            ||            ||            ||    MODE    |
         Extended |            ||            ||            ||   R:BASE   |
         Memory|  |            ||            ||            ||            |
               |  |            ||            ||            ||            |
               |  |            ||            ||            ||            |
               |  |            ||            ||            ||            |
     Upper     -->|            ||            ||            ||            |
     Memory Area  |            ||            ||            ||            |
               |  |            ||            ||            ||            |
               |  |            ||            ||            ||            |
               |  |            ||            |+------------++------------+
     Conventional |            ||            ||   DOS4GW   ||   DOS4GW   |
     Memory    |  |            |+------------++------------++------------+
               |  |            || RBASE STUB || RBASE STUB || RBASE STUB |
               |  +------------++------------++------------++------------+
               |  |    DOS     ||    DOS     ||    DOS     ||    DOS     |
                   Step 1        Step 2        Step 3        Step 4
     Figure 2.
     Once the initialization is complete, the stub program loads the DOS 
     extender. We are using the DOS4GW DOS Extender, a special version of 
     the Rational Systems, Inc. DOS Extender distributed by WATCOM, Inc.  
     The DOS4GW.EXE is a separate file on your distribution disks. Step 2 
     shows DOS4GW loaded above the R:BASE stub in conventional memory. As 
     previously described, there are many important functions which must be 
     performed by the DOS extender. Once the initialization functions are 
     completed by DOS4GW, it loads the protected mode version of R:BASE (also 
     located in this same RBASE.EXE file) from disk into extended memory.  
     This will take several seconds to complete. Once done, there are some 
     final initialization steps, followed by a command by DOS4GW to switch 
     the processor from real to protected mode. Step 3 shows all components 
     loaded in their respective memory areas.
     It should be clear at this point why it takes longer to load R:BASE 4.0. 
     We have executed several other steps which previously were not necessary 
     in the 2.X and 3.X versions of R:BASE. In addition, we are doing a 
     complete load of a 2MB executable, which never was required before for 
     the DOS R:BASE products.
     With all software loaded, you are now ready to use R:BASE.