Hooking in BASIC+ – Using Different Bait

We hope you are enjoying our series on the SRP PreCompiler. Don’t worry, we’ll have more tasty goodness to serve you in just a few days. However, we wanted to take a quick detour and revisit our recent article on hooking, a technique that allows developers a way to intercept calls to stored procedures (including system stored procedures). If you are unfamiliar with this concept then please read our earlier article before proceeding. While the method we discussed has worked well for a very long time, it still leaves a footprint in the environment that is arguably intrusive. If this were only being done in a pure test environment that might be okay. But what if there is a need to add a hook within a mission critical production environment? Perhaps you want the hook to be temporary or only valid for your session. Even if it is only minimal, leaving behind permanent hooking routines bears some risk that something could go wrong and break the application. If only there was a way of hooking without resorting to renaming the original object record and replacing it with you own…

Enter the MD Table

Many of our readers know that OpenInsight (as well as Revelation and ARev) owe their heritage to PICK and various spin-off flavors known collectively as MultiValue databases. Revelation Software products have always marched to a slightly different drum than the other vendors. The lack of an MD (or Master Dictionary) table is one of them. In traditional MultiValue databases, the MD table serves a similar purpose as the VOC table for Revelation and ARev. Chief among these purposes is its role as a pointer system. Various items that were important to the application would be cataloged in the MD table (again, the same as VOC) and this is how the system could locate volumes, tables, procedures, etc., when they were required.

In many ways, the OpenInsight System Repository (SYSREPOS table), along with a few other system tables, serves the same purpose as VOC. This has been offered as one of the reasons for why VOC was never introduced with OpenInsight (that…and the fervent insistence that OpenInsight is not ARev for Windows!) However, while the repository-based storage system offers some rather nice management features, it also removes some of the flexibility that we had with VOC (or MD as with other MultiValue databases).

In 2005, Revelation introduced a new feature of OpenInsight: Character to OpenInsight (CTO). This tool was designed to help traditional (aka green screen) MultiValue applications bridge into OpenInsight. With CTO, developers could create a virtual copy of the MultiValue application (including the character-based UI), but under the hood it is running pure OpenInsight with access to all of its modern wonders.

Okay…so what does CTO have to do with an article on BASIC+ hooking? The answer lies in the introduction of the MD table, which had previously never existed. Chances are that many OpenInsight developers aren’t even aware of its existence…even fewer know how it can be used for OpenInsight’s purposes.

Mind Your Ps and Qs

The Character to OpenInsight Reference Guide is a good place to learn about the usefulness of the MD table. For this article we will focus on Procedure Pointers (P-pointers). Table Pointers, aka Q-pointers, as well as a few others can be reviewed in the aforementioned reference guide.

Procedure Pointers are simple records stored in the MD table with the following layout:

<0> Name of the original stored procedure.

<1> P <- Always the letter "P"
<2>
<3>
<4>
<5> Table where the custom stored procedure object code lives.
<6> Name of the custom stored procedure.

Using the previous example of hooking into the Editor++ commuter module, our P-pointer would look like this:

<0> RTI_EDITOR_EVENTS
 
<1> P
<2>
<3>
<4>
<5> SYSOBJ
<6> HOOK_RTI_EDITOR_EVENTS

Revelation updated the program loader (RTP27) to look first in the MD table for a Procedure Pointer record. If one exists, it loads the referenced object code from the indicated table and stored procedure name. Thus, the original object code remains perfectly intact within the SYSOBJ table. OpenInsight now calls your code instead. Did you catch that it is possible to store your object code anywhere you want? We are much closer to achieving ARev for Windows than ever before.

Bait and Switch

When creating a program hook, the usual practice is to allow the original stored procedure to still execute (as opposed to completely replacing its functionality). In our previous article, we showed how easy this is by simply copying the original object code to a something else and then calling it directly using its new name from within the hook routine. Using the MD approach, this could still be done but it is not optimal. Remember, we are trying to avoid leaving behind a footprint. Also, what if the environment got upgraded? The original object code could be updated but you might forget to copy it again. Thus, your hook could be calling outdated code.

What’s good for the goose is good for the gander. Whatever that means to you, in this context it means that if we can use an MD Procedure Pointer to route a call to our custom hook object code, then we can do the same thing to return the call back to the original object code. We just need to create a second pointer like so:

<0> RTI_EDITOR_EVENTS_ORIG
 
<1> P
<2>
<3>
<4>
<5> SYSOBJ
<6> RTI_EDITOR_EVENTS

Our hooking procedure simply needs to call RTI_EDITOR_EVENTS_ORIG. Remember, there is no $RTI_EDITOR_EVENTS_ORIG record in SYSOBJ. RTP27 nicely loads the real $RTI_EDITOR_EVENTS while our code thinks it is calling something else.

Don’t Rock the Boat Baby

So how does one safely implement a Procedure Pointer in a multi-user environment and avoid the risk of regular users tapping into our hook routine? One approach is to write a routine that writes a P-pointer record to the MD table, call RTP27 to reload the object code, and then promptly delete the P-pointer. This will likely be safe enough, but it is not 100% safe. In a highly active system, there is always the possibility that an end user will trigger the Procedure Pointer before it gets deleted.

An alternative approach is for the developer to make use of a custom MD table. Only this custom MD table would contain the developer Procedure Pointers. However, there is a caveat. RTP27 caches the handle to the original MD table. Even if the developer switches over to another MD table, RTP27 will not make use of it. Fortunately, this problem can be resolved by running a small routine that updates the same global common that RTP27 uses:

Subroutine Refresh_MD_Handle(Void)

// Assumes the new MD table has already been attached.

Common /RTP27COMMON/ MD.FL@, LAST.FL@, HAVE.MDFL@

Open 'MD' to MD.FL@ then
    HAVE.MDFL@ = 1
end else
    MD.FL@     = ''
    HAVE.MDFL@ = 0
end

Return

Catching the Big One

This relatively non-intrusive technique for BASIC+ hooking has great value for tool vendors like SRP. For example, with a Procedure Pointer we can easily track when entities are being updated via the Repository COMPILE method and then feed this information into our toolkit. This now gives us the ability to interact with OpenInsight in real-time while development in other tools is occurring. In fact, we have a major release of one of our most popular tools come out very soon that is taking full advantage of P-Pointers. Stay tuned for more information.