It would be difficult -- in fact, it would be downright impossible -- to get some things done in NT without resorting to the use of undocumented functions.  We often resort to the use of undocumented functions in our software development projects here at OSR. However, we try to limit our use to those functions for which we have relatively clear examples.  The undocumented functions used by the file systems in the Microsoft Windows 2000 IFS kit, for example, fall into this category.

Still, we always warn people about the dangers of using undocumented functions.  "It's not always obvious how they're supposed to work," we often say.  "The implementation can change from release to release or service pack to service pack," we warn.  Most people understand and heed our warnings.  Others think we're just being annoying, or trying to kiss up to Microsoft.

A good example of how small changes in undocumented functions can come back to haunt you surfaced in a network driver we were developing recently.  One of our drivers makes use of the functions RtlFindClearBits(...) and RtlFindSetBits(...).  These functions have been used in the example file systems in the IFS kit forever.  These functions are documented for the first time in the Win2K DDK.

Both of these functions are used to find a range of bits in a bitmap.  Both take a pointer to a bitmap header, a number of consecutive clear or set bits to find (depending on the function called), and a "hint index" that hints at the point at which to start searching the bitmap.  Pretty useful, if you need to do that sort of thing, right?  And how hard can these functions be to use?

Well, it turns out using these functions can be just hard enough -- hard enough to break your code that is.  It seems that the implementation of these functions changed between NT V4 and Win2K.  Prior to Win2K, our use of these functions indicated that if you passed in a "hint index" that was past the bit range for which you were searching, the functions would return to the beginning of the bitmap and start searching from the beginning of the bitmap.  That is, the "hint index" was used as a guess; if the guess turned out to be wrong, and pointed beyond the only range in the bitmap that would satisfy your request, the function searched from beginning of the bitmap and returned a matching range.  This does not appear to be the case in the Win2K implementation of this function, even though the documentation seems to indicate that this is the way the function works.

In Win2K, our experience with these functions has been that if the only matching range in the bitmap appears before the "hint index", the functions no long return to search the bitmap from the beginning, and the functions now return 0xFFFFFFFF (to indicate that a suitable range in the bitmap was not found).  Arrrgh!  Is this a bug or is it a planned changed in the way these functions work?  Or, perhaps, we're using these functions wrong?  There's really no way to know.  The functions were undocumented prior to WIN2K, so you can't exactly complain to Microsoft that they changed from their previous behavior.

Fortunately, these changes turned-up in a driver that we were actively re-writing, with the goal of moving it from NT V4 to Win2K.  In testing the new implementation of the driver, it was only a matter of a few hours of debugging and annoyance to discover that these functions seemed to be behaving differently than we had anticipated.  It wasn't pleasant, but at least we found it in our lab and not at a customer site!

The point here isn't as much to highlight that these functions have changed (although it might be nice to note that), but also to illustrate just how subtle and wide-ranging changes to undocumented function can be.  I mean, seriously: Who would have anticipated a change to a function like RtlFindClearBits(...)??  I mean, it must have been working the way it worked for years, right?  Why change it now?  Only the developer knows for sure.  

So, be careful of those undocumented functions.  Even the benign looking ones can cause you problems. Remember that Russian phrase that the actor, turned president, turned Alzheimer’s patient popularized: "Trust, yet verify."  Sounds like good advice to me.