The Case of the Mismanaged MFS
Recently we were asked to install an audit tracking solution on an existing application. This is a fairly common use of MFS technology that many developers have implemented. Naturally we had a plug-in ready system. In fact, our AUDIT_MFS routine has its origins back to AREV and has served us and our clients very well for many years.
So without much thought, we proceeded to install our utility and provided our client with a form that allowed them to add or remove tables for audit tracking. For some of the tables, the system began to work flawlessly, but for others the system would just stall and abort the attempt to write the record to the primary database table.
Fortunately, we were able to quickly observe that those struggling tables had a pre-existing MFS (also provided by SRP.) For this application, the original MFS managed real-time synchronizing of linear hash data with SQL tables. Nevertheless we were still puzzled. All of our custom MFS routines are built upon a standard and time-tested shell. In fact, much of the core of our MFS comes from formerly arcane Revelation documentation, which is now officially published in Chapter 8 of the Programmer’s Reference Guide. So, the $64K question was this: Why would these two MFS routines not get along?
Before we rush to the last chapter of this story, we need to review a popular technique used by many MFS programs for identifying the name of the OpenInsight database table during record based operations (e.g., READ.RECORD, WRITE.RECORD, DELETE.RECORD, etc.). As many developers know, the user-friendly table name (e.g., CUSTOMERS, INVOICES, CONTACTS, etc.) is not normally provided by an MFS in these operations. Instead, these operations provide the handle to the table, which is largely useless to features that require the database table name.
One operation that does provide this name is OPEN.FILE, which ordinarily is only called once during an active session. This is the one opportunity where developers can pair the name of the table with its handle. One technique for preserving this association is maintaining a labelled common. The other technique is to prefix the table handle with the table name using a friendly delimiter (like @TM) after the BFS has been called. Since the MFS is on its way back at this point, the modified handle is safe (for the moment) and preserved for future MFS operations. This logic looks much like this:
OPEN.FILE: // Prefixing table name to handle for future operations. GoSub Call_NextFS RECORD = NAME : @TM : RECORD return
Our traditional approach had been to use the latter method. When using this technique, the MFS has to properly manage the handle. First, it needs to pull the name of the database table out of the handle. Then the handle needs to be restored to its original format before being passed into the BFS. Again, a typical block of code might look like this:
Restore_Handle: // Restore table handle for next MFS/BFS. FileName = Field(HANDLE, @TM, 1) RealHandle = Field(HANDLE, @TM, 2) FS = DELETE(BFS, 1, 1, 1) NEXTFS = FS<1, 1, 1> @FILE.ERROR = '' CALL @NEXTFS(CODE, FS, RealHandle, NAME, FMC, RECORD, STATUS) return
Now back to our contentious MFS routines. Alone they worked fabulously, together they fought like kids unwilling to share their toys. We reversed the order of their execution…still no joy. After a bit of patient debugging we finally observed something very significant: in the second MFS, the value of RealHandle variable before being passed into the CALL @NEXTFS statement was empty. Now we were getting somewhere. Further investigation revealed that the HANDLE argument (from which RealHandle is derived) was also empty.
Finally, the puzzle pieces were starting to fit together. When the first MFS was called, it assumed that the HANDLE argument was already formatted as NAME : @TM : HANDLE and therefore it properly extracted the real handle before passing it into @NEXTFS (because, after all, we cannot assume that other MFS routines or the BFS can manage this modified handle format.) However, the next MFS to be called was not any ordinary MFS, it was another custom MFS that assumed the handle was still using the NAME : @TM : HANDLE format…NOT! In the end we were tripping over our own toes. The first MFS passed through a properly formatted handle to the second MFS, but the second MFS was expecting a modified handle.
The lesson learned was that we cannot assume that there is one and only one MFS in the chain that relies on modified handles. Our quick fix was to make sure that the modified handle format was preserved until only the BFS was left to call. A corollary lesson is that all of our woes probably would have been avoided had we used a variation of the labelled common technique. 🙂
Thanks for sharing
Hmmmm… the quick fix could also fail if a table has another mfs, say system indexing (SI.MFS) in its chain prior to the final BFS call!!
Yes, you are right and we were painfully aware that our own quick fix still had shortcomings, but it was “good enough” for the moment. 🙂 In fact, the table that first stumbled had an index, but SI.MFS was always first in the chain so it did not encounter any problems.
Ironically, we do have a newer MFS template that we use and it stores the table handle and name in memory. However, our utility already had its own, albeit order style, MFS so we just rolled with it. We are now retrofitting our old MFS routines with our new template and all should be good!