Filter drivers vs hooking

NTFSD
1

Hi Nick/All,

I have ‘known’ that writing filter drivers is the
recomended method to hooking filesystem requests.

But I don’t fully understand the reasons.
Would someone mind explaining it?

Thanks in advance!

Regards,
Manoj

-----Original Message-----

Subject: Re: Why can’t hook CreateFile?
From: “Nick Ryan” <nryan_at_nryan_dot_com>
> Date: Wed, 4 Jun 2003 09:38:22 -0700
>
>
> PLEASE don’t give advice like this to new list
members. If he wants to
> hook filesystem requests, first tell him about the
DOCUMENTED method (a
> filesystem filter). Hooking ZwCreateFile is ugly,
undocumented, and
> completely unnecessary if you write a filesystem
filter. If he insists
> the IFSKIT is too expensive and he’s just coding a
one-off hack, then
> fine, but if he’s trying to develop a commercial
product (which is what
> it sounds like here), it benefits us all to make
sure it will be as
> reliable as possible. :slight_smile:
>
> - Nick Ryan

________________________________________________________________________
Yahoo! India Matrimony: Find your partner online. http://yahoo.shaadi.com/india-matrimony/</nryan_at_nryan_dot_com>

2

“Manoj Paul Joseph” wrote in message news:xxxxx@ntfsd…

I have ‘known’ that writing filter drivers is the
recomended method to hooking filesystem requests.

But I don’t fully understand the reasons.

Manoj,

First off this is system call table hooking, there are other types of
hooking with various impacts. My list against system call table hooking is
a follows:

  1. Many of the calls are undocumented

Bottom line is if you are doing anything more than the simplest hook,
you will be wandering into areas of undocumented and changeable calls.

  1. Call numbers change

While this isn’t bad for ZwCreateFile or others that are exposed in
the kernel, there are plenty of calls that are not exposed getting the right
index for the hook is messy.

  1. No locking on applying the hook

This is a serious problem. If driver A and driver B both are hooking,
there is nothing stopping a situation of:

System call 1 - Call A’s hook, then Call B’s hook, then call the
system
System call 2 - Call B’s hook, then Call A’s hook. then call the
system

Depending what A and B are doing this is a disaster. And this assumes
that the drivers are smart on their hooking and use interlocked exchange.
Otherwise you can have situations where A & B stomp on the others hook.

  1. Don’t even think about unloading a hook driver

While filter drivers are rarely unloaded, unloading a hook driver is
a disaster. If anyone hooks above you then that driver will still try to
call your code with the disasterous results.

Bottomline is that there is absolutely no need to do something this stupid
when Microsoft gives you a blessed way of handling things that avoids the
above problems.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

3

Hi Don,

Thanks a lot for the answers! One more question.

If MS were to provide an API for hooking, wouldn’t
things be different?
Not as messy at any rate… Right?

Or are filtering just that and more?
If so, more in what way?

Regards,
Manoj

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf
Of Don Burn
Sent: Tuesday, April 13, 2004 6:06 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Filter drivers vs hooking

“Manoj Paul Joseph” wrote in message
news:xxxxx@ntfsd…

I have ‘known’ that writing filter drivers is the
recomended method to hooking filesystem requests.

But I don’t fully understand the reasons.

Manoj,

First off this is system call table hooking,
there are other types of
hooking with various impacts. My list against system
call table hooking is
a follows:

  1. Many of the calls are undocumented

Bottom line is if you are doing anything more
than the simplest hook,
you will be wandering into areas of undocumented and
changeable calls.

  1. Call numbers change

While this isn’t bad for ZwCreateFile or others
that are exposed in
the kernel, there are plenty of calls that are not
exposed getting the right
index for the hook is messy.

  1. No locking on applying the hook

This is a serious problem. If driver A and
driver B both are hooking,
there is nothing stopping a situation of:

System call 1 - Call A’s hook, then Call
B’s hook, then call the
system
System call 2 - Call B’s hook, then Call
A’s hook. then call the
system

Depending what A and B are doing this is a
disaster. And this assumes
that the drivers are smart on their hooking and use
interlocked exchange.
Otherwise you can have situations where A & B stomp on
the others hook.

  1. Don’t even think about unloading a hook driver

While filter drivers are rarely unloaded,
unloading a hook driver is
a disaster. If anyone hooks above you then that
driver will still try to
call your code with the disasterous results.

Bottomline is that there is absolutely no need to do
something this stupid
when Microsoft gives you a blessed way of handling
things that avoids the
above problems.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

Yahoo! India Matrimony: Find your partner online. http://yahoo.shaadi.com/india-matrimony/

4

“Manoj Paul Joseph” wrote in message
news:xxxxx@ntfsd…
> Hi Don,
>
> Thanks a lot for the answers! One more question.
>
> If MS were to provide an API for hooking, wouldn’t
> things be different?
> Not as messy at any rate… Right?
>
> Or are filtering just that and more?
> If so, more in what way?
>
> Regards,
> Manoj
>

Microsoft did provide a system call hooking interface for registry stuff for
RegMon. They officially haven’t documented it but one of their operations
put the draft spec on the web. On a file system hooking can be harder, you
will not see the I/O for mapped memory with hooking, but you do with a
filter.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

5

Let me add something to Don’s comments. This is basically a pragmatic
argument.

Filtering will provide various guarantees that things will work, and the
programming model is sometimes simpler. For example, with WDM filtering, OS
support for the model will see to it that other drivers in the stack are
called in proper order. Another example is that with NDIS IM drivers
(admittedly not of the filesystem domain), NDIS ensures that certain things
won’t happen concurrently.

Doing hooking oneself, on the other hand, it is up to you to get all such
things right. This may not be important if you’re just experimenting, but it
is crucial for distributed code.

Although it can be very time-consuming to learn the particulars of whatever
filter model one is using, I believe it is a lesser burden – over the long
term in particular – than getting everything right by oneself.

–
James Antognini
Windows DDK Support

This posting is provided “AS IS” with no warranties, and confers no rights.

“Manoj Paul Joseph” wrote in message
news:xxxxx@ntfsd…
> Hi Nick/All,
>
> I have ‘known’ that writing filter drivers is the
> recomended method to hooking filesystem requests.
>
> But I don’t fully understand the reasons.
> Would someone mind explaining it?
>
> Thanks in advance!
>
> Regards,
> Manoj

6

I addition to Don’s excellent points I wish to add a couple more:

  • Starting with Window XP the kernel code is write protected. Of course
    a kernel mode driver can work around this by directly manipulating the
    mapping registers but this widens the window for potential interop
    issues as described in pint #3 by Don.

  • In future versions of Windows the system is going to detect these
    types of changes and will do something to either prevent it or bug check
    the system.

  • I am seeing issues with drivers that do API hooking not working with
    XP SP2 because the function prolog in XP SP2 has changed and they have
    hard code what the function prolog looks like.

The bottom line is that direct hooking of APIs is a bad and unsupported
thing that in the long term will not be allowed. The supported way of
“hooking” file systems is by writing a file system filter.

If you are presently doing API hooking please start making plans now to
eliminate this. There will come a time when your product will no longer
work with windows.

Note that we are in the process of making the Registry “hooking” APIs
publicly available.

Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of James Antognini
[MSFT]
Sent: Tuesday, April 13, 2004 10:19 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Filter drivers vs hooking

Let me add something to Don’s comments. This is basically a pragmatic
argument.

Filtering will provide various guarantees that things will work, and the
programming model is sometimes simpler. For example, with WDM filtering,
OS
support for the model will see to it that other drivers in the stack are
called in proper order. Another example is that with NDIS IM drivers
(admittedly not of the filesystem domain), NDIS ensures that certain
things
won’t happen concurrently.

Doing hooking oneself, on the other hand, it is up to you to get all
such
things right. This may not be important if you’re just experimenting,
but it
is crucial for distributed code.

Although it can be very time-consuming to learn the particulars of
whatever
filter model one is using, I believe it is a lesser burden – over the
long
term in particular – than getting everything right by oneself.

–
James Antognini
Windows DDK Support

This posting is provided “AS IS” with no warranties, and confers no
rights.

“Manoj Paul Joseph” wrote in message
news:xxxxx@ntfsd…
> Hi Nick/All,
>
> I have ‘known’ that writing filter drivers is the
> recomended method to hooking filesystem requests.
>
> But I don’t fully understand the reasons.
> Would someone mind explaining it?
>
> Thanks in advance!
>
> Regards,
> Manoj

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

7

> ----------

From: xxxxx@lists.osr.com[SMTP:xxxxx@lists.osr.com] on behalf of Neal Christiansen[SMTP:xxxxx@windows.microsoft.com]
Reply To: Windows File Systems Devs Interest List
Sent: Tuesday, April 13, 2004 7:50 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Filter drivers vs hooking

  • In future versions of Windows the system is going to detect these
    types of changes and will do something to either prevent it or bug check
    the system.

Which can prevent many useful developers tools from working. For example, System Internals DebugView and their other tools. It would be nice if there is a supported way for hooking as was in w9x. I don’t mean file system filtering (where current supported way is right) but general API hooking. Hooking itself isn’t bad and supported way would solve most of mentioned problems.

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http:://www.upek.com]

8

Speaking as someone who comes from a fault tolerant background, there isn’t
much they can do besides bug check. Essentially they have detected a
modification of a critical data structure, they cannot just shutdown, the
hooks could prevent it. So either you accept the fact that things are hosed
(since no one is supposed to do this) and crash, or you say to heck with
security and data reliability and let the hooker run free.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

“Brian Hutchison” wrote in message news:xxxxx@ntfsd…
>>- In future versions of Windows the system is going to detect these
>>types of changes and will do something to either prevent it or bug check
>>the system.

> Speaking as someone who has had to reinstall windows after installing a
driver that bug
> checked on
> boot up, PLEASE, PLEASE, PLEASE don’t bug check if hooking is detected.
There must
> be a better approach.

9

Well you cannot be sure you will display the message since the hook could
stop it. So I guess you advocate, letting something hook the system suck
the confidential data dry, trash the world so you can never get at your data
again, and then crash unexpectedly! As I said I’m from an fault tolerant
background and their you never want to crash, but bottom line is this is one
of these cases that it is the only safe thing to do.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

“Brian Hutchison” wrote in message news:xxxxx@ntfsd…
>> So either you accept the fact that things are hosed
> > (since no one is supposed to do this) and crash, or you say to heck with
> > security and data reliability and let the hooker run free

> So is the question “whether it’s better to crash now or maybe crash
later”? If so, I’ll go with
> “maybe crash later” every time. There was a great “Peter Pontificates”
article in an issue of
> NT Insider that talked about just this topic.

> If it can be detected, why not display a message box saying “Windows has
detected that
> non-Microsoft software has performed an illegal operation. Please
uninstall whatever you
> have recently installed until this message box goes away.” Microsoft can
also refuse to
> provide support when they detect this has been done.

10

Neal Christiansen wrote:

> Note that we are in the process of making the Registry “hooking” APIs
> publicly available.

Hello Neal, I’m curious about the registry hooking APIs and would
appreciate if you could answer some questions.

When will they actually be available? Or better, what do I have to do to
receive a preliminary version asap?

Will there be documented ways to filter other functions besides the
registry related?

Thanks
Tobias

11

Don,

you say to heck with security and data reliability
> and let the hooker run free.

Given that driver planting a hook requires local admin privilege to
install, just like a file system driver, and given that fully blessed
file-system filters like some AV junk still use so much stack that there
is little else left, what exactly makes you think that a filter is less
of a risk than a hook?

Essentially, the TCB of Windows (and Linux, and, and, and) is way too
large; nothing short of a major redesign will ever change that. And as
long as we merrily add stuff to that trusted base, the risk is the same.

Now if filters were run isolated, while hooking drivers were part of the
TCB, I might agree with you …

Cheers,
Felix.

12

“Felix Kasza” wrote in message news:xxxxx@ntfsd…
> Given that driver planting a hook requires local admin privilege to
> install, just like a file system driver, and given that fully blessed
> file-system filters like some AV junk still use so much stack that there
> is little else left, what exactly makes you think that a filter is less
> of a risk than a hook?

Felix,

I am not arguing that a filter is less of risk than a hook. The
original argument was if you consider a hook to be an “illegal operation”
then what is the correct response. My point is that once you assume it is
illegal the only correct responses are: undo the hooks if you can, and
crash if you cannot.

There was a proposal in the discussion to put up a message box, the
reason that is ill advised is the message box (assuming anyone is at the
console or that there is a console) requires a number of operations and so
is relatively easy to hook away. As any of us who write drivers know, it is
very easy to crash a system, and you don’t need a lot of OS support to do
it.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

13

“Felix Kasza” wrote in message news:xxxxx@ntfsd…
> Don,
>
> > you say to heck with security and data reliability
> > and let the hooker run free.
>
> Given that driver planting a hook requires local admin privilege to
> install, just like a file system driver, and given that fully blessed
> file-system filters like some AV junk still use so much stack that there
> is little else left, what exactly makes you think that a filter is less
> of a risk than a hook?
>

I am sick to death of discussions like this.

You’re missing the point: There’s an architected and supported way to do
certain things, and there’s a hacked, lazy, poor engineering way.

The dev who uses other than the architected way, violates the assumptions of
every other dev who’s writing kernel code on that system. When you’re
writing your code, how can you possibly take into account every possible
weird hack that somebody else might do? And if you don’t, and some weird
hack IS present on the same system as your code, how do you know your code
will work? Answer: You don’t.

On the other hand, it certainly IS possible to take into account architected
interfaces, and do your best to insure that your code works properly in all
architected instances.

See, it’s not necessarily the risk of the hacked implementation screwing up.
It’s the risk the hacked implementation creates in other implementations,
and the inability of those implementations to foresee the hacks.

Burn said it all, correctly, a day or two ago: System call hooking for a
product is stupid engineering (OK, he didn’t say that exactly – Don’s way
too polite – but that’s my paraphrase).

Now, HAVING SAID ALL THAT… There are certainly cases when you’re writing
diagnostic code and have no alternative. In that case, all bets are off.
Just be sure you warn people that you’ve got a massive hack going on before
you load your code. There’s still a chance that the system will crash, but
at least forewarned is forearmed.

Peter
OSR

14

“Peter Viscarola” wrote in message news:xxxxx@ntfsd…
> Burn said it all, correctly, a day or two ago: System call hooking for a
> product is stupid engineering (OK, he didn’t say that exactly – Don’s way
> too polite – but that’s my paraphrase).

Actually, I’m not polite, but the OP was asking why he shouldn’t and I will
give the benefit of the doubt to people who ask why.

Thinking about this discussion, I did realize that it is desirable to have a
way to do this for diag purposes. So a request to Neal and the folks at
Microsoft looking at protecting against hooking. Consider making the code
that tests for hooks and restore/crashes prompt in the debugger in the
checked build similar to asserts, so that developers do have a way of using
this for diagnostic purposes.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

15

If the OS can detect the “hooking” how about just “unhooking” it?

I agree that there are some very useful dev utilities that I hate to see no
longer work.

-----Original Message-----
From: Don Burn [mailto:xxxxx@acm.org]
Sent: Tuesday, April 13, 2004 4:54 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Filter drivers vs hooking

Well you cannot be sure you will display the message since the hook could
stop it. So I guess you advocate, letting something hook the system suck
the confidential data dry, trash the world so you can never get at your data
again, and then crash unexpectedly! As I said I’m from an fault tolerant
background and their you never want to crash, but bottom line is this is one
of these cases that it is the only safe thing to do.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

“Brian Hutchison” wrote in message news:xxxxx@ntfsd…
>> So either you accept the fact that things are hosed
> > (since no one is supposed to do this) and crash, or you say to heck with
> > security and data reliability and let the hooker run free

> So is the question “whether it’s better to crash now or maybe crash
later”? If so, I’ll go with
> “maybe crash later” every time. There was a great “Peter Pontificates”
article in an issue of
> NT Insider that talked about just this topic.

> If it can be detected, why not display a message box saying “Windows has
detected that
> non-Microsoft software has performed an illegal operation. Please
uninstall whatever you
> have recently installed until this message box goes away.” Microsoft can
also refuse to
> provide support when they detect this has been done.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@legato.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

16

What about something similar as SFP protection? It “works” only if WinDbg is
attached. WBR Primoz

-----Original Message-----
From: Don Burn [mailto:xxxxx@acm.org]
Sent: Wednesday, April 14, 2004 4:43 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Filter drivers vs hooking

“Peter Viscarola” wrote in message news:xxxxx@ntfsd…
> Burn said it all, correctly, a day or two ago: System call hooking for a
> product is stupid engineering (OK, he didn’t say that exactly – Don’s way
> too polite – but that’s my paraphrase).

Actually, I’m not polite, but the OP was asking why he shouldn’t and I will
give the benefit of the doubt to people who ask why.

Thinking about this discussion, I did realize that it is desirable to have a
way to do this for diag purposes. So a request to Neal and the folks at
Microsoft looking at protecting against hooking. Consider making the code
that tests for hooks and restore/crashes prompt in the debugger in the
checked build similar to asserts, so that developers do have a way of using
this for diagnostic purposes.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@hermes.si
To unsubscribe send a blank email to xxxxx@lists.osr.com

17

Well,

I do tend to agree here with Don, Felix. And youeself talking about TCB
in this post makes me think you are looking at this problem of filter vs
hooking
from a security point of view, which is not by far the only aspect to be
considered.

Dan

----- Original Message -----
From: “Felix Kasza”
To: “Windows File Systems Devs Interest List”
Sent: Wednesday, April 14, 2004 4:09 PM
Subject: RE: [ntfsd] Filter drivers vs hooking

> Don,
>
> > you say to heck with security and data reliability
> > and let the hooker run free.
>
> Given that driver planting a hook requires local admin privilege to
> install, just like a file system driver, and given that fully blessed
> file-system filters like some AV junk still use so much stack that there
> is little else left, what exactly makes you think that a filter is less
> of a risk than a hook?
>
> Essentially, the TCB of Windows (and Linux, and, and, and) is way too
> large; nothing short of a major redesign will ever change that. And as
> long as we merrily add stuff to that trusted base, the risk is the same.
>
> Now if filters were run isolated, while hooking drivers were part of the
> TCB, I might agree with you …
>
> Cheers,
> Felix.
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

18

Dan,

you are right about my point of view – security. IMHO, it subsumes
issues of bugginess; a malfunctioning part of the TCB is a security
risk, too. (I should mention that I dabble in security for my daily
bread.)

Give me an extensible security model in NT, and the hooks become
unnecessary. Linux, incidentally, has a similar problem, slightly worse.

How hard can it be, at the time one designs an OS, to provide documented
and well-defined authorisation hooks for all operations that access or
modify data? (And privilege checks for all ops that do not involve a
securable object.)

In the meantime, I received an emailed reply from another list member,
hinting at a possible way to obviate the need for a hook in our stuff.
Keep your fingers crossed!

But my point remains: Not everything can be done with a file system
filter. And if a customer, say, of the military persuasion, prefers
security at the price of using undocumented OS internals to no security,
but that fully documented, then one does what one has to.

Cheers,
Felix.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Dan Partelly
Sent: Wednesday, 14 April 2004 18:54
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Filter drivers vs hooking

Well,

I do tend to agree here with Don, Felix. And youeself talking about TCB
in this post makes me think you are looking at this problem of filter vs
hooking
from a security point of view, which is not by far the only aspect to be
considered.

Dan

----- Original Message -----
From: “Felix Kasza”
To: “Windows File Systems Devs Interest List”
Sent: Wednesday, April 14, 2004 4:09 PM
Subject: RE: [ntfsd] Filter drivers vs hooking

> Don,
>
> > you say to heck with security and data reliability
> > and let the hooker run free.
>
> Given that driver planting a hook requires local admin privilege to
> install, just like a file system driver, and given that fully blessed
> file-system filters like some AV junk still use so much stack that
there
> is little else left, what exactly makes you think that a filter is
less
> of a risk than a hook?
>
> Essentially, the TCB of Windows (and Linux, and, and, and) is way too
> large; nothing short of a major redesign will ever change that. And as
> long as we merrily add stuff to that trusted base, the risk is the
same.
>
> Now if filters were run isolated, while hooking drivers were part of
the
> TCB, I might agree with you …
>
> Cheers,
> Felix.
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@mvps.org
To unsubscribe send a blank email to xxxxx@lists.osr.com

19

I’d like to apologize for posting to this list using my badly configured
mail client. Please allow to post my questions again, since they’re of
great interest for me. So here it goes…

Neal Christiansen wrote:

> Note that we are in the process of making the Registry “hooking” APIs
> publicly available.

Hello Neal, I’m curious about the registry hooking APIs and would
appreciate if you could answer some questions.

When will they actually be available? Or better, what do I have to do to
receive a preliminary version asap?

Will there be documented ways to filter other functions besides the
registry related?

Thanks
Tobias

20

Yes someplaces dont care of “godLike attitude”, they just want to get the security done, at any and every cost. Hey, for doing that they will pay a very high price, but giving them some standard architectural bullshit is to annoy a bunch of credible people who definitely knows what is this damn, security, and how to circumvent… Really a joke, when some of us talk about security …

-pro