Picking the Correct XMLHTTP Object
Many OpenInsight applications integrate HTTP calls to extend its capabilities and embrace the power of web services. Whether using the Google Map API to get geolocation information or the UPS APIs to track shipments, applications are doing far more than they ever could before. We were recently approached by ChargeItPro, a leader in payment processing, to help them out. ChargeItPro was going to retire an older web API and replace it one with based on a RESTful architecture. Since ChargeItPro has a number of customers using OpenInsight, they maintain BASIC+ stored procedures and provide them to developers who want to integrate their services. We were asked to convert the logic to work with their new API. This seemed like a fun project, especially since much of the work we are now doing is heavily centered on REST APIs and web services.
On the whole, the conversion to the new API was straight forward. The API follows a very simple flow:
We had the code converted fairly quickly. However, during testing we discovered that there was something that wasn’t working quite right: only the first GET request would actually reach the server. All subsequent attempts would simply return the same response…but it was clear that the server was not receiving any new requests.
After a fair amount of research we discovered that the problem was due to the original XMLHTTP object in the code provided to us:
Before we reveal the nature of the problem, it should be noted that this is a perfectly acceptable XMLHTTP object. Also, in case you didn’t know it, there are two others that can be used:
Of course, once you start to add in different versions of each object, the list starts to be somewhat lengthy. Nevertheless, the above represent the latest versions, which is what we should be using anyway.
Why are there three different objects? Part of the answer is that WinHttp.WinHttpRequest.5.1 (or WinHTTP for short) has given way to Msxml2.ServerXMLHTTP.6.0 (or ServerXMLHTTP for short). Thus, if we ignore WinHTTP, there are really only two objects. Okay, fine, then why are there two different objects? Good question, especially since it is directly relevant to the problem we were having.
Microsoft’s online documentation states that Msxml2.XMLHTTP.6.0 (or XMLHTTP for short) has a very short description of its purpose:
Provides client-side protocol support for communication with HTTP servers.
This matches our expectations. But what about ServerXMLHTTP? Here’s what its documentation states:
Provides methods and properties that enable you to establish an HTTP connection between files or objects on different Web servers.
The ServerXMLHTTP object offers functionality similar to that of the XMLHTTP object. Unlike XMLHTTP, however, the ServerXMLHTTP object does not rely on the WinInet control for HTTP access to remote XML documents. ServerXMLHTTP uses a new HTTP client stack. Designed for server applications, this server-safe subset of WinInet offers the following advantages:
- Reliability — The HTTP client stack offers longer uptimes. WinInet features that are not critical for server applications, such as URL caching, auto-discovery of proxy servers, HTTP/1.1 chunking, offline support, and support for Gopher and FTP protocols are not included in the new HTTP subset.
- Security — The HTTP client stack does not allow a user-specific state to be shared with another user’s session. ServerXMLHTTP provides support for client certificates.
So…what does all this mean? First, despite using the same properties and methods, they are built around two different technologies: WinInet (XMLHTTP) and WinHTTP (ServerXMLHTTP). It also means that XMLHTTP and ServerXMLHTTP are designed for two different purposes. XMLHTTP is designed for apps (i.e., clients) that need basic browser like functionality from within their code. ServerXMLHTTP is designed for…wait for it…servers! That is to say, when a server is talking to another server, it needs a robust and reliable connection. Clients, on the other hand, are optimized for performance and therefore have quicker timeouts and they can cache results.
It was that last bit of detail that helped to explain why our GET requests were no longer reaching the server. Basically, XMLHTTP did its job and cached our URL request. Armed with this knowledge, we switched the object over to ServerXMLHTTP. Our GET requests were reaching the server as often as we sent them. Problem solved and project completed.
Still, we thought there had to be a way to use the XMLHTTP object and force it to not cache. After doing a bit of searching on this issue, we discovered that this is a common problem that developers from various languages have run into. Fortunately, we found that the If-Modified-Since header helps in this situation, which we implemented like this:
rv = OLECallMethod(Object, "setRequestHeader", "If-Modified-Since", "Sat, 1 Jan 2000 00:00:00 GMT")
This header effectively forces the XMLHTTP object to check in with the server to see if there have been any changes since the date specified in the value. So, both XMLHTTP and ServerXMLHTTP can be used with little or no changes in the code.
We should also report that we had tested the OLE_GetWebPage function and found that it also worked perfectly well. A little deeper digging revealed that this function relies upon the ServerXMLHTTP object, which confirmed our previous conclusions.