minifilter cache question

NTFSD
1

We’re seeing some performance degradation when we copy files across the network and our minifilter is installed. I’ve traced this to the FltGetFileNameInformation call (for the normalized name) by montioring the SMB traffic with Wireshark. We always call this in the same way:

status = FltGetFileNameInformation( Data,
FLT_FILE_NAME_NORMALIZED |
FLT_FILE_NAME_QUERY_DEFAULT,
&pNameInfo );

My understanding is that when a call is made to FltGtFileNameInformation, the name should be stored in the name cache. We currently call this function from the PreCallbacks on IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION, IRP_MJ_CREATE and IRP_MJ_CLEANUP but it seems to go through the whole nightmare of retrieving the normalized name on each call. I would have expected to take the hit the first time around but then retrieve it from the cache on subsequent calls. As an experiment I tried modifying the flags to FLT_FILE_NAME_QUERY_CACHE_ONLY in the cleanup but this failed with STATUS_FLT_NAME_CACHE_MISS.

Is the name cache entry just valid for the duration of the current IRP (in otherwords, is it valid for other minifilters but not other IRPs) or is it reasonable to assume that the cache entries might (but not necessarily will) persist across two or more of my dispatch functions? Also, does the cache entry lifetime behave differently for network files ?? I know that FltGetFileNameInformation will queue requests off to a worker thread if there is less than 50% of stack space remaining (on x86 at least). Is it possible that a name cache entry is tied to a specific thread and we are seeign the effects of that?

I understand that querying for the normalized name is an expenseive operation but we have to use the normalized name. We can always implement our own cache of course but in theory at least this seems unnecessary.

Regards

Mark

2

I suspect that at least part of the issue is that there’s no simple way for filter manager to know when the name goes stale on a network drive and thus the cache lifetime is relatively short. Presumably one of those familiar with the innards of the name caching in FM can confirm or deny this.

I was curious about one of your statements:

This is a hard problem over the network, since all you are going to get is a name anyway. Unless one builds a server-side assist component, you won’t be able to disambiguate names anyway.

First, there’s the name aliasing problem:

\192.168.1.47\library\blah\foo.txt
\myserver\apple\bar.txt

These could be the same file - I’ve combined several different screw cases here: IP address versus name (could be a netbios name or a dns name, for example), the fact that share names are themselves non-overlapping AND the fact that the far end might support hard links.

Then there’s the issue that redirector (last I checked, haven’t looked to see if SMB2 does this) returns the opened name for a file, not the current name - even if you did a rename through the local path.

So, all I have to do to screw up your normalized name logic is to open the file, rename it, memory map it, and work on it. The rename won’t be reflected in the “normalized name” anyway.

Frustrating (to me) is that techniques for disambiguation that we can use on local drives (e.g., the FID on the volume) don’t work across the network, because you get a simulated FID. Many years ago we built a server-side add-in that would take an FSCTL and return the real FID - that allowed us to disambiguate. The same trick would allow us to do the (correct) normalized name.

Tony
OSR

3

Thanks Tony.
I was doing a bit more testing this morning and observed that the cache doesn’t seem to be working correctly for local files either (the second and and third IRPs fail if I specify the FLT_FILE_NAME_QUERY_CACHE_ONLY option) . Digging through the event log I found the following errors shortly after a reboot

Name caching for File System Filters has been disabled on volume ‘\Device\Harddisk0\DR0’.
Name caching for File System Filters has been disabled on volume ‘\Device\Mup’.
Name caching for File System Filters has been disabled on volume ‘\Device\HarddiskVolume1’.

Has anybody else seen this before or can anybody hazard a guess as to why I might be seeing this (low memory conditions for example)

Regards

Mark

4

You say you call it in your PreCallbacks. You should move that to the PostCallbacks, as it’s far more likely to be in the name cache then. But it’s not guaranteed, and if you are the only one calling it, you’ll still incur the penalty. If someone else does it for you, then you get it for (almost) free.

Phil

Not speaking for LogRhythm

Phil Barila | Senior Software Engineer
720.881.5364 (w)
LogRhythm, Inc.
A LEADER 2012 SIEM Magic Quadrant
WINNER of SC Magazine’s 2012 SIEM Best Buy

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Thursday, October 25, 2012 6:57 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] minifilter cache question

We’re seeing some performance degradation when we copy files across the network and our minifilter is installed. I’ve traced this to the FltGetFileNameInformation call (for the normalized name) by montioring the SMB traffic with Wireshark. We always call this in the same way:

status = FltGetFileNameInformation( Data,
FLT_FILE_NAME_NORMALIZED |
FLT_FILE_NAME_QUERY_DEFAULT,
&pNameInfo );

My understanding is that when a call is made to FltGtFileNameInformation, the name should be stored in the name cache. We currently call this function from the PreCallbacks on IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION, IRP_MJ_CREATE and IRP_MJ_CLEANUP but it seems to go through the whole nightmare of retrieving the normalized name on each call. I would have expected to take the hit the first time around but then retrieve it from the cache on subsequent calls. As an experiment I tried modifying the flags to FLT_FILE_NAME_QUERY_CACHE_ONLY in the cleanup but this failed with STATUS_FLT_NAME_CACHE_MISS.

Is the name cache entry just valid for the duration of the current IRP (in otherwords, is it valid for other minifilters but not other IRPs) or is it reasonable to assume that the cache entries might (but not necessarily will) persist across two or more of my dispatch functions? Also, does the cache entry lifetime behave differently for network files ?? I know that FltGetFileNameInformation will queue requests off to a worker thread if there is less than 50% of stack space remaining (on x86 at least). Is it possible that a name cache entry is tied to a specific thread and we are seeign the effects of that?

I understand that querying for the normalized name is an expenseive operation but we have to use the normalized name. We can always implement our own cache of course but in theory at least this seems unnecessary.

Regards

Mark

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

5

Well my logic (correct or otherwise) is that when I call it first time for IRP_MJ_CREATE (Pre or Post) I’ll incur the hit but at least it will be in the cache. Subsequent IRPs - IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION and IRP_MJ_CLEANUP should pull it from there but this doesn’t appear to be happening - presumably because name caching appears to have been disabled for some reason. Think I need to get to the bottom of why caching has been disabled first. My VM only had 1GB of RAM so I’ve increased this to 4GB and am still seeing these “Name caching for File System Filters has been disabled on volume …” errors.

6

This is likely because filter manager is not attached to the file system base device object. If some other legacy filter is attached between fltmgr and the file system, filter manger does not cache names for the volume. To confirm look for the EnableNameCaching flag in the output of !fltkd.volume.

Scott [MSFT]
This posting is provided “AS IS” with no warranties, and confers no rights.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Friday, October 26, 2012 10:53 AM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] minifilter cache question

Well my logic (correct or otherwise) is that when I call it first time for IRP_MJ_CREATE (Pre or Post) I’ll incur the hit but at least it will be in the cache. Subsequent IRPs - IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION and IRP_MJ_CLEANUP should pull it from there but this doesn’t appear to be happening - presumably because name caching appears to have been disabled for some reason. Think I need to get to the bottom of why caching has been disabled first. My VM only had 1GB of RAM so I’ve increased this to 4GB and am still seeing these “Name caching for File System Filters has been disabled on volume …” errors.

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

7

That’s exactly what the problem is.

kd> !drvobj ntfs
Driver object (85a59858) is for:
\FileSystem\Ntfs
Driver Extension List: (id , addr)

Device Object list:
860cc020 85a92640
kd> !devstack 860cc020
!DevObj !DrvObj !DevExt ObjectName
860cbbb8 \FileSystem\FltMgr 860cbc70
860cb358 \Driver<thirdpartydriver> 860cb410
> 860cc020 \FileSystem\Ntfs 860cc0d8

Thanks Scott. Very much appreciated