In this series of articles, we’ll be exploring how the a precompiler can be used to enhance the BASIC+ language. This first article will teach you the tips and tricks we’ve learned to achieve language enhancement. Subsequent articles will show you how the next release of the SRP Editor will leverage the precompiler to take your programming to new heights.

Precompiling has been in OpenInsight for quite some time. We’ve considered taking advantage of it in the past, but there were still some things we wanted to work out in order to make it work the way we wanted it to. Recent work for a client forced us to dig into this further, and as a result, the SRP Precompiler was born. Later in this series, we’ll show you all the cool things SRP Precompiler will do for you. In the meantime, this first article will give you the technical details you’ll need to pursue your own language enhancements using precompiling.

Precompiler

Precompiling isn’t rocket science. You really don’t need official support if you think about. You could just write a routine that reads the source, changes it, and passes it to the compiler. The SRP Editor, if we so desired, could even do this seamlessly. However, official support is preferred because we want anyone on any editor to successfully compile our enhanced code. What good is an enhanced language if someone down the road has to undo it to make it compile? Thankfully, Revelation added official precompiling several versions ago.

The official support comes in two forms. The first is the CFG_PRECOMPILER record in the SYSENV table. This record holds a list of stored procedure names which will be passed the source code in succession. However, this is a secret always-on approach that we think is only appropriate for non-morphing precompilers such as ones that auto document routines. If we are going to enhance our language, we want programmers to know that something unique is happening. There is a second approach, which will be the focus of this article.

First, you need to create a stored procedure that accepts two parameters: Routine and ProgramName. When the user compiles his or her code, the compiler will pass the code into your precompiler giving you a chance to alter the code before final compilation.

Compile subroutine MY_PRECOMPILER(Routine, ProgramName)

The code itself is passed as an @FM delimited array via the Routine parameter, which your precompiler will alter as desired. The ProgramName parameter contains the name of your program as it appears at the top of your code.

#PRAGMA

The #pragma keyword tells the compiler to use a given precompiler for the current stored procedure. This keyword is known as a compiler directive, which means it gives instructions to the compiler rather than being some kind of runtime logic. In this case, we follow the #pragma keyword with another keyword: precomp. This tells the compiler that we are supplying our own precompiler, which we reference by name:

#pragma precomp MY_PRECOMPILER

This has the advantage of both enabling precompiling on a per-routine basis and clearly documenting what precompiler we are using for the benefit future programmers. Once the #pragma keyword is added, we can test and debug our precompiler to refine our language enhancements. While there are no conflicts with using the debug keyword, there is another helpful #pragma keyword: output.

#pragma output SYSLISTS MY_PRECOMPILER_OUTPUT

This command instructs the compiler to save the results of the precompiler to the given table and key. In the above example, the output will be saved to the SYSLISTS table with the key “MY_PRECOMPILER_OUTPUT”. This is very useful for catching mistakes that are difficult to see in the debugger.

This is all you really need to write your own precompiler. For the curious, there is more to the #pragma keyword, which you can review starting on page 5 of Sprezzatura’s March 2007 SENL.

Playing Nice with the Debugger

The number one set back to using a precompiler is the potential for the source code lines to be out of sync with the code they execute. For example, if your precompiler replaces a single line of source code, such as…

SECURE(Total = CalculateInvoice())

With this…

If UserHasSecurityClearance() then
    Total = CalculateInvoice()
End

Then you just offset all code following your changes by two lines. This becomes obvious in the debugger. Recall that the debugger always displays your original source code. So, if CalcaulateInvoice was originally on line 10 but your precompiler moved it to line 11, the debugger will still show it on line 10. As a result, the debugger will highlight the wrong line. The more line disparity you create in your precompiler, the more impossible it becomes to debug. For this reason, we avoided the precompiler for a long time. Debugging is simply too important to our productivity.

However, we stumbled upon some undocumented keywords: STOPLINECOUNT and STARTLINECOUNT. The compiler uses these when processing the $Insert keyword. The way inserts work is that the compiler does exactly what it sound like: it insert the contents right into your stored procedure. But the insert might have 50 lines of code, which would really throw off the debugger. So, at the top of the inserted code the compiler places a STOPLINECOUNT keyword, and at the bottom it places a STARTLINECOUNT keyword. In so doing, every line of code in the insert points to the same one line.

As it turns out, these keywords can be used in your precompiler as many times as you want. So, using our above example, your precompiler could produce this:

STOPLINECOUNT
If UserHasSecurityClearance() then
    Total = CalculateInvoice()
End
STARTLINECOUNT

This will ensure that the debugger maps all three lines of code to the same line as the original source code. Hooray! We did it. Life is good and we can move on with our lives, right?

F10 Carpel Tunnel

We’ve all been there. We step into a stored procedure during a debug session. The current line is an $Insert. You press F10, but the line doesn’t move. You press it again and it still doesn’t move. That’s when you remember that the insert is actually executing multiple lines of code that you can’t see. Therein lies the caveat with using the STOPLINECOUNT and STARTLINECOUNT keywords. The same thing will happen in your precompiled code. Sure, the debugger stays on one line, but the user has to press F10 multiple times to step over one line of code. This kind of unpredictability is a show stopper as far as we are concerned. It’s come to be expected with inserts, but if a normal line of code needs to be stepped over three times, it will lead to frustration and destroy productivity.

What we really want is a single line of code in our source code to behave like a single line in the debugger. Thankfully, BASIC+ is actually quite forgiving when it comes to placing multiple statements onto a single line of code. It may come as no surprise to anyone that you can do something like this:

A = 1; B = 2; C = 3

But you can also have single line Locate statements:

Locate Event in EventList using "," setting Pos then On Pos Gosub CREATE, READ, WRITE

In fact, you can stack a lot of code onto one line. In so doing, two things happen. First, you no longer need to use STOPLINECOUNT and STARTLINECOUNT. After all, your precompiler is replacing one line with another single (albeit very long) line. Second the debugger steps over the line in a single press of F10. Now we’re talking! We can actually enhance a single line of code to do more behind the scenes without interfering with the development/debug process.

It’s worth noting that the #pragma keyword does support a feature that will collapse many lines of code onto a single line. However, it doesn’t do anything more than swap line breaks with semicolons. Since we are already programmatically generating code, it’s just as well that we put it all on one line ourselves.

Enhancing the Language

The point of all this is you can truly enhance the syntax of the language. As mentioned above, we’ve developed an up-and-coming tool called the SRP Precompiler that will ship with the SRP Editor, and it will bring many language enhancements. Let’s look at a simple feature of our precompiler that’s easy to implement using the techniques described in this article.

It’s not uncommon to see something like this at the end of a Stored Procedure:

If Assigned(Response) else Response = ""
Return Response

Developers do this as a safety precaution to avoid VNAV errors at runtime. Using the SRP Precompiler, you’ll be able to do this at the end of a stored procedure:

Return Response or ""

The SRP Precompiler will recognize the syntax and convert it into this:

Return (If Assigned(Response) then Response else "")

The new syntax is easy to read, easy to interpret, and it executes in a single line. Imagine the possibilities. Better yet, get started on your own precompiler today and enhance your productivity.

Stay tuned for the next article when we show how the SRP Precompiler is bringing high performance and ease of use to a very common BASIC+ task: looping through dynamic arrays.