Here at OSR, we have taken a more complex implementation approach to using file system filter contexts in our own filters. This is because we must support a broad range of OS versions, including those on which filter contexts are not available. We prefer to provide a uniform context model, so that even if the underlying file system does not support filter contexts on some files (for example the paging file) we fall back to the old model.

Of course, when filter contexts are not supported, we use a separate lookup table with either the FsContext or SectionObjectPointers structure address as the lookup key - each is unique on a per-file basis. Separately, we also track per-instance information by using the address of the FileObject itself - again, via a lookup table. Thus, on a system that generally supports filter contexts, we first check to see if we can find the requisite information from the filter context and if not, we then check our separate lookup table. This dual lookup is wrapped with our own functions so as to keep the interface clean - and isolates our drivers from the differences between various versions of Windows, including service packs.

Back to the issue at hand. Filter context support is determined by using the function FsRtlSupportsPerStream Contexts. This is actually a macro that we can find in ntifs.h, see below.

Note that this macro really doesn't validate that it is looking at an advanced header - it just checks bits assuming that it is an advanced header. What we found is that under some circumstances this macro would return TRUE, but when we called FsRtlLookupStreamContext, the system would crash. After doing some digging around, we found that one of the standard Microsoft file system drivers (no name calling here) was actually setting FsContext to point to a UNICODE_STRING structure. We discussed this issue for a while and since we were in "fix it and keep going" mode, we decided to special case this particular file system so that we know we can't rely upon stream contexts.

However, after further discussion and reflection it occurs to us that there's a fundamental (implicit) assumption in the filter context model that bears additional caution here: the filter context model assumes that the FsContext pointer will always refer to a structure arranged in the common header format. Thus, this either now becomes a requirement for all file systems (something that Microsoft developers have publicly stated in the past and for the record is not the case) or the filter context model has a fundamental flaw.

In the case of the Microsoft FSD, we looked at, the FsContext value actually referred to a UNICODE_STRING structure. On the x86 platform, this structure consists of 64 bits of information; Casting it to an FSRTL_ADVANCED_FCB_ HEADER and then testing the Flags2 field to determine if the FSRTL_FLAG2_ SUPPORTS_FILTER_CONTEXTS bit is set, means that the macro is testing a bit somewhere in the address of the buffer itself. Of course, whether this bit is set or cleared does seem to be a bit arbitrary - change the pool allocation function, and code that previously worked might stop working because these bits no longer are set quite the same way.

We spent some time thinking about how we could make this more robust. We want to guarantee that our own filters can take advantage of filter contexts on file systems where they are supported, and yet ensure we do not rely upon a feature that might not be present on other file systems. Naturally, we try to be conservative in our development efforts in this area because a failure to do this properly will typically lead to a system crash - a situation that we consider to be unacceptable. Thus, the thought that we could track just the exceptions to this rule works if we assume that we will only be called upon to filter the Windows file systems.

One possibility is that we could add additional checks (or validation) to the FCB header itself. For example, we note there is "node" information: a NodeTypeCode and NodeByteSize field. Compiling a list of valid values for this might at least provide us with some additional surety (for example, ensuring the NodeByteSize field is at least large enough to contain the stream context field). If we examine the FAT file system code, we can find that it uses several different values for both of these fields. This makes validation a rather costly affair since we would need to perform this check each time we wanted to consider looking at the stream context information associated with a given FILE_OBJECT.

For example, the FAT file system defines its "node types" in the header file nodetype.h. The values range from 0x500 through 0x508. This field would correspond to the Length field in our UNICODE_STRING structure. While all of these values are large, none of the even values would be invalid. Presumably we can also find the comparable range for the other file systems, either by inspecting the source code (CDFS uses the range from 0x301 through 0x309) or by observing them in actual use (NTFS appears to use values in the 0x700 range).

Heuristic data structure validation techniques might work most of the time, but they are merely a way of "papering over" the underlying problem - that there really is no way that we can determine, by looking at the file object, if the file system is using the FSRTL_COMMON_FCB_HEADER. Ideally, we'd want to have some sort of flag set in the file object itself (perhaps FO_USES_COMMON_FCB_HEADER) that we could use to validate the assumption that the common header is present. Once we know the common header is present, the other techniques used for validation (reading Flags2 for example) work correctly. Unfortunately, to implement this would require changing all of the existing file systems to support this flag.

In the interim, we've discussed this at length and come to the conclusion that filter contexts in their current incarnation are not really safe to use. This is very unfortunate because using a lookup table is an expensive operation. We optimize the lookup operation as best we can to minimize this cost, but in our development efforts we've concluded that expensive is always preferable to incorrect.

For those of you that are willing to rely upon heuristic approaches, we have come up with a variety of suggestions on how to mitigate these issues. They include:

• Checking for the type of the file system against a list of drivers known to support filter contexts. The alternative (checking for file systems known not to support filter contexts) isn't reliable, but might be sufficient if you never filter anything other than a standard Windows file system.
• Checking the node information; one observation is that the NodeTypeCode value is generally larger than the NodeByteSize field and this would at least allow us to rule out the case we saw - where a UNICODE_STRING structure was being used.
• Checking to ensure the NodeByteSize field is greater than the FSRTL_ADVANCED_FCB_HEADER. If it is not, then no matter what the Flags2 field indicates, it is quite possible that the structure is not big enough to contain the requisite field.

No doubt there are other heuristic approaches that can also be concocted, and yet each of them leaves us with the fear that some other file system - whether from Microsoft or from some third party vendor - will not fit into our heuristic model. This will result in either incorrect behavior or a system crash.

Thus, for the foreseeable future we have reached the unfortunate conclusion that filter contexts, while good in theory, can't be used in the hostile world in which we operate. Other developers should make sure they are also aware of these hazards and (at a minimum) take precautions or eschew using filter contexts until such time as Microsoft's development team finds a way to make them more robust.