By Rod Widdowson

The availability of the IFS Kit for Windows 2003 SP1 in spring 2005 marked the official release of the Filter Manager. A beta version of the kit, that included the Filter Manager, had been available in beta format for quite some time. As a result, many people have had significant success developing minifilters. Since the documentation is now in excellent shape, rather than present an exhaustive "how to" article, this article provides some hints and tips based on our own experience.

"Quick to Write" Does Not Mean "Easy to Design"

It is vital to realize that just because the Filter Manager makes writing a file system filter faster, it does not make designing a file system filter easier. The Filter Manager does simplify the effort required to building a file system filter. It includes a body of "boiler plate" code that you no longer need to write. This makes it possible to go from design to working filter in weeks rather than months.

However, don't be lulled into thinking that the difficult parts of filter design are not still difficult. For example, you still have to understand how a file system filter fits into Windows and how it will interoperate with the various subsystems and other filters it will encounter. No matter how easy it is to construct your filter initially, you must still deal with the realities and responsibilities of "being" a file system filter driver.

Put another way, the Filter Manager is a great productivity tool for a skilled engineer. But like many such tools, the Filter Manager can be dangerous in the hands of an unskilled engineer. The world has enough bad software and, notwithstanding the opportunities it would give Peter to pontificate, it would be a shame if the Filter Manager was abused to create more bad software at a faster pace than before.

If You're Writing a Lot of Code ‑ Check the Documentation

The Filter Manager comes with a rich library of support functions including support functions for most operations a filter will need. For example, if you need to find out the file name of a renamed file, you can use FltGetDestinationFile NameInformation(). If you need to know the name of the share from which a file is being accessed, then you can use FltParseFileNameInformation().

My rule is if you find yourself writing a lot of code that is not specific to the job of your minifilter, the chances are there is an Flt function for you to call.

As a corollary, avoid Zw calls. There is nearly always an Flt version that will do the same job or even a better job. For instance, by default, Filter Manager library calls never re-enter the top of the stack, which means strange re-entrancy rules do not exist. Looking toward Windows Vista, there is room in the Filter Manager's structures to make it easier for minifilters to support Transactional NTFS whereas legacy filters will have to "go it alone".

It is equally important not to underestimate the work the Filter Manager can do for you and check the documentation carefully. Quite often it just "does the right thing", which surprised us more than once.  For instance, when we started writing our first minifilter, we wrote a great deal of code to enumerate all the file systems on startup. We wrote more code to explicitly tear down our contexts and instances on shutdown. Later we discovered that calling FltStartFiltering provokes multiple callbacks at InstanceCallbackSetup and that FltUnregisterFilter tears down all the instances and, hence, all the contexts. This discovery allowed us to significantly simplify startup and shutdown.

We know from conversations with others that this sort of "over programming" is not uncommon with people who are used to developing legacy filters.

Unload Frequently During Development

A frequent question on NTFSD is, "When I detach my minifilter, it just hangs. What has gone wrong?" In 99% of the cases the filter has not returned some structure such as contexts or other structures to the Filter Manager. Debugging this sort of problem when your filter is substantially complete is slow and incredibly frustrating. Therefore, we suggest the first thing you write when developing a new filter is the detach and unload code. 

Once your filter unloads cleanly, always unload after every test run. That way, as soon as you introduce a pool leak, the Driver Verifier will let you know. Similarly, as soon as you forget to dereference a context, the Filter Manager will let you know. We all know that these kinds of bugs are easier to find when they are "fresh".

Use Checked Kernel and Checked Filter Manager

Using the checked kernel, should be as second nature to you as using Driver Verifier. Here's a hint to make sure this is the case ‑ use the Filter Manager from the checked build. The Filter Manager is rich in ASSERTs that usually fire early enough to assist in bug tracking. See Figure 1 for a favorite ASSERT.

Figure 1 - A Favorite ASSERT

Again, the best help the checked build Filter Manager gives is on unload. As mentioned above, if you left references on any contexts, the free build Filter Manager just hangs. If you are using the checked build version, it will list all the contexts that are still referenced. Microsoft promises that future releases of the Filter Manager will provide even more diagnostics (the "filter verifier"). We can't wait!

How Context Management Works

Context management is probably one of the most frustrating parts of maintaining a minifilter. Currently there are limited diagnostics to help if a context reference is held too long. This means finding a leaking context "in the field" is likely to be difficult, if not impossible. As noted above, your best protection is to unload frequently during development which helps find the bugs early. However, before development even starts, you should have a good understanding of what the calls do that manipulate contexts. Since the documentation of this has historically been weak, a brief summary is provided below.

Contexts are referenced data structures. The Filter Manager arranges that a context will continue to exist for as long as its reference count is non zero. As soon as something reduces the reference count to zero, the appropriate ContextCleanup Callback will be called (if you have registered one), and then the context will be deallocated. Hence, it is important to understand which operations have what effect on the reference count.

Creating, looking up, and explicitly referencing a context will all add a reference to the context. However, often times a context will also have a reference because it has been attached to a system data structure via one of the FltSetXXXContext calls. The reference count is decremented when such a context is detached. Since this often results in the count dropping to zero, the context will usually be cleaned up and deallocated.

Incrementing the reference count prior to it being manipulated (via FltReferenceContext or, more often, one of the FltGetContext calls) protects your code from having a context disappear while operations are ongoing. Protecting data structures in a multi-threaded environment is an important reason for reference counting.

Given that FltSetContext will attach a context to a data structure and add a reference count, what are the cases that will detach it? There are, in fact, four such cases.

1. The attached to system structure is about to go away. For example, when the file system calls FsRtlTeardownPer StreamContexts as part of tearing down the FCB, the Filter Manager will detach any attached stream contexts and dereference them.
2. The filter instance associated with the context is being detached.  Again taking the stream context example, during instance teardown after the InstanceTeardown callbacks have been made the filter manager will detach any stream contexts associated with this instance from their associated ADVANCED_FCB_HEADER and dereference them.
3. The minifilter itself detaches the context by calling FltDeleteContext or one of its variants.
4. The minifilter itself detaches the context by calling the appropriate FltSetXXXContext functions, and specifying FLT_SET_CONTEXT_REPLACE_ IF_EXISTS. In this case the old context is not dereferenced, which means it will not go away before the caller has a chance to inspect it. Therefore, the caller has to perform the dereference.

The documentation indicates that cases 1, 2, and 4 listed above are the most common reasons detaching occurs. Therefore, it is usually not necessary to call the FltDeleteXXXContext routines.

For most filters, every function that references a context will usually dereference it before returning. A notable exception is when a context is allocated in a pre-create call and is then associated to the FCB in the post-create call. If you can construct code to ensure that references do not span function calls, your code will be easier to understand and inspection will be a whole lot easier.

If you find situations when you need to hold a reference on a context for a long time, you should reexamine your structures and move whatever requires you to keep the context referenced into a secondary (possibly reference counted) structure.

For example, if a filter has a need to keep information on a per-file (rather than a per-stream) basis, then they need per-file contexts. (This feature will be available in Windows Vista.)

A naive implementation might have the per-file information in the stream context, and then link together all the stream contexts that share the file. They will need to be referenced for as long as they are linked in order to protect to the linked contexts.

A better solution would be to have an explicit per-file structure that contains the per-file information and a reference count. This per-file structure should be pointed to by every appropriate stream context. In the StreamContextCleanup Callback, the reference to the per-file structure is decremented, and, if required, the structure is deallocated.

In this way, the time during which stream contexts are referenced is kept to a minimum.

Remember, if you have a reference on a context you will not be told when the underlying system structure goes away. For example, in the case of Stream Contexts, as long as your filter has no reference on the context, the StreamContextCleanup Callback will be associated with the file system tearing down its FCB as described above. This is important if you want to keep information associated with the stream beyond the call to CLEANUP.

Instance Initialization and Teardown

As we have seen, instance initialization can either happen:

• Explicitly - via FLTMC, the FltAttachVolume kernel function, or the FilterAttach user mode function; or
• Automatically - as a result of a minifilter being loaded or a volume becoming otherwise available

Depending on precisely what triggers the call of InstanceSetupCallback, the process context will vary. This requires you to be careful before making any assumptions about the context in which your filter is being called.

Let's look at two examples.

1. FLTMC ATTACH myfilter C - InstanceSetup Callback will be called in the context of the process running fltmc.
2. FLTMC LOAD myfilter - InstanceSetup Callback will be called in the context of the system process.

This has several repercussions; many of them are quite subtle. For example, consider what impact the previous mode may have on probe. Therefore, it is important to test at least these two variants, along with the associated detach and teardown homologs.

Summary

The Filter Manager is currently available on Windows 2000 (with appropriate service packs and hot fixes), Windows XP SP2, and Windows Server 2003 SP1. At this point, unless the target customers are on NT or they cannot (or will not) move up to a version of the OS that supports the Filter Manager (in which case ‑ good luck!), there is no reason why most file systems filters should not be developed using the Filter Manager. 

The team that put together the minifilter framework has thought a great deal about the problems of building file system filters. Their efforts have provided a Filter Manager that will undoubtedly save a great deal of time. However, as I cautioned earlier, do not be seduced by the ease of writing the code. The design stage is still strewn with architectural pitfalls.

Rod Widdowson is Consulting Partner with Steading System Software LLP, a file system consultancy.  He can be reached at: rdw@steadingsoftware.com