Driver Problems? Questions? Issues?
Put OSR's experience to work for you! Contact us for assistance with:
  • Creating the right design for your requirements
  • Reviewing your existing driver code
  • Analyzing driver reliability/performance issues
  • Custom training mixed with consulting and focused directly on your specific areas of interest/concern.
Check us out. OSR, the Windows driver experts.

On-Access, Transparent, Per-File Data Encryption:

OSR's File Encryption Solution Framework (FESF) provides all the infrastructure you need to build a transparent file encryption product REALLY FAST.

Super flexible policy determination and customization, all done in user-mode. Extensive starter/sample code provided.

Proven, robust, flexible. In use in multiple commercial products.

Currently available on Windows. FESF for Linux will ship in 2018.

For more info: https://www.osr.com/fesf

Go Back   OSR Online Lists > ntfsd
Welcome, Guest
You must login to post to this list
  Message 1 of 31  
10 Aug 12 02:45
Prasad Dabak
xxxxxx@vmware.com
Join Date: 09 Jun 2011
Posts To This List: 125
Mini filter peformance issues while accessing network files

Hello, We have hit into serious performance issues while accessing network files with our mini filter driver and we have narrowed it down to 3 filter manager calls that are causing significant performance degradation viz. FltGetFileNameInformation (normalized path), FltQuerySecurityObject, (owner/group and dacl/sacl information), FltQueryInformationFile (basic and standard information) In order to narrow down the scope for the performance experiments, we tweaked our filter driver to just trap in PostOpCreate and make those 3 calls. We also wrote a simple test application (win32 console application) that just opens and closes the specified file specified number of times and measure the performance in controlled environment. When we run our test application with our filter driver loaded, For network files: FltGetFileNameInformation causes 6x slowness. FltQuerySecurityObject causes 3x slowness. FltQueryInformationFile causes 2x slowness. The cumulative slowness is around 10X. For local files: The cumulative slowness is around 2X. Significant overhead is coming from FltGetFileNameInformation. We are calling it with NameOptions = FLT_FILE_NAME_NORMALIZED | FLT_FILE_NAME_QUERY_ALWAYS_ALLOW_CACHE_LOOKUP We found similar threads on the forum in the context of calling it from PreOpCreate. However, we are calling it from PostOpCreate. Further, some threads mention that this was an issue in earlier versions of Windows (e.g. XP). However, we are finding performance degradation in 64-bit Windows 2008 R2 as well All the three calls are mandatory for functioning of our filter driver. Are there any optimizations that we can look for? On other note, in our original filter (i.e. not the one tweaked for performance measurements) we cache the name information that we receive from FltGetFileNameInformation call in our stream context (allocated using FltAllocateContext with FLT_STREAM_CONTEXT). This ensures that for subsequent opens, we don't call the API again if the stream context is around. For local files, we see that Windows keeps the stream context around. However, for network files, we find that stream context is released as soon as the file is closed. Hence, with our test application, we get better numbers if we do open, open, open, close, close, close sequence instead of open,close,open,close,open,close sequence. Please advice. Thanks. -Prasad
  Message 2 of 31  
10 Aug 12 04:09
Amritanshu Johri
xxxxxx@gmail.com
Join Date: 05 Jul 2012
Posts To This List: 51
Mini filter peformance issues while accessing network files

What is the protocol server end speaking? what kind of pathnames ? On Fri, Aug 10, 2012 at 12:15 PM, <xxxxx@vmware.com> wrote: > Hello, > > We have hit into serious performance issues while accessing network files= with our mini filter driver and we have narrowed it down to 3 filter manag= er calls that are causing significant performance degradation viz. FltGetFi= leNameInformation (normalized path), FltQuerySecurityObject, (owner/group a= nd dacl/sacl information), FltQueryInformationFile (basic and standard info= rmation) > > In order to narrow down the scope for the performance experiments, we twe= aked our filter driver to just trap in PostOpCreate and make those 3 calls.= We also wrote a simple test application (win32 console application) that j= ust opens and closes the specified file specified number of times and measu= re the performance in controlled environment. > > When we run our test application with our filter driver loaded, > > For network files: > FltGetFileNameInformation causes 6x slowness. > FltQuerySecurityObject causes 3x slowness. > FltQueryInformationFile causes 2x slowness. > The cumulative slowness is around 10X. > > For local files: <...excess quoted lines suppressed...> ling it with NameOptions =3D > FLT_FILE_NAME_NORMALIZED | FLT_FILE_NAME_QUERY_ALWAYS_ALLOW_CACHE_LOOKUP > > We found similar threads on the forum in the context of calling it from P= reOpCreate. However, we are calling it from PostOpCreate. Further, some thr= eads mention that this was an issue in earlier versions of Windows (e.g. XP= ). However, we are finding performance degradation in 64-bit Windows 2008 R= 2 as well > > All the three calls are mandatory for functioning of our filter driver. = Are there any optimizations that we can look for? > > On other note, in our original filter (i.e. not the one tweaked for perfo= rmance measurements) we cache the name information that we receive from Flt= GetFileNameInformation call in our stream context (allocated using FltAlloc= ateContext with FLT_STREAM_CONTEXT). This ensures that for subsequent opens= , we don't call the API again if the stream context is around. For local fi= les, we see that Windows keeps the stream context around. However, for netw= ork files, we find that stream context is released as soon as the file is c= losed. Hence, with our test application, we get better numbers if we do ope= n, open, open, close, close, close sequence instead of open,close,open,clos= e,open,close sequence. > > Please advice. > > Thanks. > -Prasad > > > --- > NTFSD is sponsored by OSR > .osronline.com/page.cfm?name=3DListServer
  Message 3 of 31  
10 Aug 12 15:06
ntfsd member 5961
xxxxxx@alfasp.com
Join Date:
Posts To This List: 1448
Mini filter peformance issues while accessing network files

Use OPENED file names if you can - or suffer the performance penalty :) There are no workarounds. xxxxx@vmware.com wrote: > Hello, > > We have hit into serious performance issues while accessing network files with our mini filter driver and we have narrowed it down to 3 filter manager calls that are causing significant performance degradation viz. FltGetFileNameInformation (normalized path), FltQuerySecurityObject, (owner/group and dacl/sacl information), FltQueryInformationFile (basic and standard information) > > In order to narrow down the scope for the performance experiments, we tweaked our filter driver to just trap in PostOpCreate and make those 3 calls. We also wrote a simple test application (win32 console application) that just opens and closes the specified file specified number of times and measure the performance in controlled environment. > > When we run our test application with our filter driver loaded, > > For network files: > FltGetFileNameInformation causes 6x slowness. <...excess quoted lines suppressed...> -- Kind regards, Dejan (MSN support: xxxxx@alfasp.com) http://www.alfasp.com File system audit, security and encryption kits.
  Message 4 of 31  
10 Aug 12 16:52
Christian Allred
xxxxxx@microsoft.com
Join Date: 26 Jan 2009
Posts To This List: 104
Mini filter peformance issues while accessing network files

Unfortunately Dejan is pretty much right. The slowdown from FltGetFileNameInformation is because it has to open every component in the path and query for the long name in order to build the normalized name. That is, if the path is \\server\share\foo\bar\baz\file.txt, then a call to FltGetFileNameInformation results in: Open \\server\share\foo\bar\baz, QueryDirectory( file.txt, FileNamesInformation ) Open \\server\share\foo\bar, QueryDirectory( baz, FileNamesInformation ) Open \\server\share\foo, QueryDirectory( bar, FileNamesInformation ) Open \\server\share, QueryDirectory( foo, FileNamesInformation ) As you can imagine, this is time consuming on the network and much faster locally. I'm curious about your cache. Filter Manager already implements a name cache. Calling FltGetFileNameInformation populates this cache the first time around, and draws from the cache on subsequent calls. Filter Manager also takes care of the other work needed to properly maintain this cache (e.g. invalidating it when necessary). Did you implement your own cache in response to seeing poor performance? In Windows 8 local queries are even faster because Filter Manager is able to use the FileNormalizedNameInformation IRP_MJ_QUERY_INFORMATION class to get the full normalized name in one shot. We weren't able to get support for that in remote names this time around, but it is something we want to do in the future. -- Christian [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
  Message 5 of 31  
11 Aug 12 08:41
Prasad Dabak
xxxxxx@vmware.com
Join Date: 09 Jun 2011
Posts To This List: 125
Mini filter peformance issues while accessing network files

Thank you all for your responses. @Amritanshu, we are accessing files on a remote Windows share. The paths are of the form \\servername\sharename\filepath. @Dejan, that's unfortunate to hear. Is "short names not expanded" the only difference between FLT_FILE_NAME_NORMALIZED and FLT_FILE_NAME_OPENED? In that case, can we use some heuristic? e.g. Use FLT_FILE_NAME_OPENED and if the returned path contains any tildes then use FLT_FILE_NAME_NORMALIZED. Agreed, it will cause additional overheads for files accessed using short paths OR if the file genuinely contains tilde character. But at least for files that are accessed using long names, we can optimize. @Christian, no, we are not maintaining cache specifically for this performance issue. Our filter already maintains some other data in the stream context and one of it is nameinfo returned by FltGetFileNameInformation call. We reference it when we initially create stream context and release it when stream context is destroyed. For network files, we observe that stream context is released as soon as file is closed and hence nameInfo is also released and apparently it is removed from filter manager cache as well and hence for each subsequent open we get a performance overhead. That's the reason we get better performance for open,open,close,close pattern vs. open, close, open, close pattern for the same file. For local files, stream context remains around and hence the nameinfo as well. Does filter manager also release name information when last reference to the file goes away? Thanks. -Prasad
  Message 6 of 31  
13 Aug 12 09:55
Amritanshu Johri
xxxxxx@gmail.com
Join Date: 05 Jul 2012
Posts To This List: 51
Mini filter peformance issues while accessing network files

Hello Prasad, I am not sure how to quantify the 6x performance degradation. At the risk of stating the obvious, 2k8 onwards SMB2 is used for 2k8 <-> 2k8/Vista or 2k8 <-> Win7/R2, it has a redirector cache this normally is suppose to speed enumeration and query info operation for SMB2 clients. Is that enabled for you? here is a small link for your reference: http://technet.microsoft.com/en-us/library/ff686200(v=ws.10).aspx what kind of traffic are you seeing on your netmon/ws trace, too many disconnects? tree connects? regards, Amritanshu.
  Message 7 of 31  
14 Aug 12 06:00
Prasad Dabak
xxxxxx@vmware.com
Join Date: 09 Jun 2011
Posts To This List: 125
Mini filter peformance issues while accessing network files

@Amritanshu, yes, you are right. If both ends (client and server) involve Windows 2008/Windows 7 machines, the degradation is much lower. I did another experiment *without* our filter loaded. The test application gets the start tick, opens and closes the file on a network share 500 times, gets the end tick and prints the diff between end tick and start tick. I ran the application on Windows 2008 R2 client. When the network share is hosted on Windows 2003 server, the application took around 850 ticks to complete. However, when the share is hosted on Windows 7 machine, the application took just 47 ticks to complete. That's significant.... Thanks. -Prasad
  Message 8 of 31  
14 Aug 12 13:53
ntfsd member 8
xxxxxx@osr.com
Join Date:
Posts To This List: 2736
Mini filter peformance issues while accessing network files

<quote> I ran the application on Windows 2008 R2 client. When the network share is = hosted on Windows 2003 server, the application took around 850 ticks to com= plete. However, when the share is hosted on Windows 7 machine, the applicat= ion took just 47 ticks to complete. That's significant.... </quote> Welcome to the wonderful world of caching (and/or deferral). Tony OSR
  Message 9 of 31  
14 Aug 12 14:10
Christian Allred
xxxxxx@microsoft.com
Join Date: 26 Jan 2009
Posts To This List: 104
Mini filter peformance issues while accessing network files

The file server on Windows 7/Server 2008 takes advantage of a much improved oplock mechanism that allows it to cache more effectively than previous versions. > Does filter manager also release name information when last reference to the file goes away? Yes. -- Christian [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
  Message 10 of 31  
15 Aug 12 12:24
ntfsd member 8
xxxxxx@osr.com
Join Date:
Posts To This List: 2736
Mini filter peformance issues while accessing network files

<quote> The file server on Windows 7/Server 2008 takes advantage of a much improved= oplock mechanism that allows it to cache more effectively than previous ve= rsions. </quote> Handle oplocks. Further, in Windows 8, handle oplocks are now supported on directories; pre= sumably this should allow caching of even more information (e.g., directory= contents). As an aside, it is interesting that while handle oplocks are supported in W= indows 7, they're actually a bit of a challenge to test as the ifs kit test= s don't actually exercise them. Tony OSR
  Message 11 of 31  
16 Aug 12 03:29
Prasad Dabak
xxxxxx@vmware.com
Join Date: 09 Jun 2011
Posts To This List: 125
Mini filter peformance issues while accessing network files

Thank you all for the responses. Unfortunately, we cannot expect all Windows machines in a network to run Vista/Windows 7(+) to benefit from the optimizations. Hence, revisiting one of the question that I asked earlier: Is "short names not expanded" the only difference between FLT_FILE_NAME_NORMALIZED and FLT_FILE_NAME_OPENED? In that case, can we use some heuristic? e.g. Use FLT_FILE_NAME_OPENED and if the returned path contains any tildes then use FLT_FILE_NAME_NORMALIZED. Agreed, it will cause additional overheads for files accessed using short paths OR if the file genuinely contains tilde character. But at least for files that are accessed using long names, we can optimize? Thanks. -Prasad
  Message 12 of 31  
16 Aug 12 03:46
ntfsd member 5961
xxxxxx@alfasp.com
Join Date:
Posts To This List: 1448
Mini filter peformance issues while accessing network files

Normalized names also resolve mount points. xxxxx@vmware.com wrote: > Thank you all for the responses. > > Unfortunately, we cannot expect all Windows machines in a network to run Vista/Windows 7(+) to benefit from the optimizations. > > Hence, revisiting one of the question that I asked earlier: Is "short names not expanded" the only difference between FLT_FILE_NAME_NORMALIZED and FLT_FILE_NAME_OPENED? In that case, can we use some heuristic? e.g. Use FLT_FILE_NAME_OPENED and if the returned path contains any tildes then use FLT_FILE_NAME_NORMALIZED. Agreed, it will cause additional overheads for files accessed using short paths OR if the file genuinely contains tilde character. But at least for files that are accessed using long names, we can optimize? > > Thanks. > -Prasad -- Kind regards, Dejan (MSN support: xxxxx@alfasp.com) http://www.alfasp.com File system audit, security and encryption kits.
  Message 13 of 31  
16 Aug 12 09:42
ntfsd member 8
xxxxxx@osr.com
Join Date:
Posts To This List: 2736
Mini filter peformance issues while accessing network files

I'm not sure how this would work with open by file ID or object ID. Of cou= rse, the simplest thing to do would be to simply try it. That's what Ladis= lav's excellent little testing tool is for - so you can open a file in vari= ous ways to see what happens, quickly and without extra programming effort. Tony OSR
  Message 14 of 31  
16 Aug 12 10:28
Alex Carp
xxxxxx@gmail.com
Join Date: 23 Feb 2010
Posts To This List: 981
Mini filter peformance issues while accessing network files

This won't work because not all short names have the tilde character. You wo= uld need to treat any name shorter than 8 characters as potentially a short n= ame.=20 What do you use the name for?=20 Thanks, Alex Sent from my iPhone On Aug 16, 2012, at 6:43 AM, Tony Mason <xxxxx@osr.com> wrote: > I'm not sure how this would work with open by file ID or object ID. Of co= urse, the simplest thing to do would be to simply try it. That's what Ladis= lav's excellent little testing tool is for - so you can open a file in vario= us ways to see what happens, quickly and without extra programming effort. >=20 > Tony > OSR >=20 >=20 > --- > NTFSD is sponsored by OSR >=20 > For our schedule of debugging and file system seminars visit:=20 > http://www.osr.com/seminars <...excess quoted lines suppressed...> osronline.com/page.cfm?name=3DListServer
  Message 15 of 31  
16 Aug 12 14:11
mm
xxxxxx@gmail.com
Join Date: 24 May 2010
Posts To This List: 121
Mini filter peformance issues while accessing network files

Are you back? Mm On Aug 16, 2012 7:29 AM, "Alex Carp" <xxxxx@gmail.com> wrote: > This won't work because not all short names have the tilde character. You > would need to treat any name shorter than 8 characters as potentially a > short name. > > What do you use the name for? > > Thanks, > Alex > > Sent from my iPhone <...excess quoted lines suppressed...> --
  Message 16 of 31  
16 Aug 12 15:41
Alex Carp
xxxxxx@gmail.com
Join Date: 23 Feb 2010
Posts To This List: 981
Mini filter peformance issues while accessing network files

I am , but i'm in redmond at plugfest Sent from my iPhone On Aug 16, 2012, at 11:10 AM, "Martin O'Brien" <xxxxx@gmail.com> wrote: > Are you back? > > Mm > > On Aug 16, 2012 7:29 AM, "Alex Carp" <xxxxx@gmail.com> wrote: > This won't work because not all short names have the tilde character. You would need to treat any name shorter than 8 characters as potentially a short name. > > What do you use the name for? > > Thanks, <...excess quoted lines suppressed...> --
  Message 17 of 31  
16 Aug 12 15:50
mm
xxxxxx@gmail.com
Join Date: 24 May 2010
Posts To This List: 121
Mini filter peformance issues while accessing network files

Outstanding. mm From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alex Carp Sent: Thursday, August 16, 2012 12:41 PM To: Windows File Systems Devs Interest List Cc: Windows File Systems Devs Interest List Subject: Re: [ntfsd] Mini filter peformance issues while accessing network files I am , but i'm in redmond at plugfest Sent from my iPhone On Aug 16, 2012, at 11:10 AM, "Martin O'Brien" <xxxxx@gmail.com> wrote: Are you back? Mm On Aug 16, 2012 7:29 AM, "Alex Carp" <xxxxx@gmail.com> wrote: This won't work because not all short names have the tilde character. You would need to treat any name shorter than 8 characters as potentially a short name. What do you use the name for? Thanks, Alex Sent from my iPhone On Aug 16, 2012, at 6:43 AM, Tony Mason <xxxxx@osr.com> wrote: > I'm not sure how this would work with open by file ID or object ID. Of course, the simplest thing to do would be to simply try it. That's what Ladislav's excellent little testing tool is for - so you can open a file in various ways to see what happens, quickly and without extra programming effort. > > Tony > OSR > > > --- > NTFSD is sponsored by OSR > > For our schedule of debugging and file system seminars visit: <...excess quoted lines suppressed...> --- NTFSD is sponsored by OSR For our schedule of debugging and file system seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer --- NTFSD is sponsored by OSR For our schedule of debugging and file system seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer --- NTFSD is sponsored by OSR For our schedule of debugging and file system seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer --
  Message 18 of 31  
17 Aug 12 05:36
Prasad Dabak
xxxxxx@vmware.com
Join Date: 09 Jun 2011
Posts To This List: 125
Mini filter peformance issues while accessing network files

@Dejan, ok, so, we may end up reporting the name without resolving mount points. I am thinking of using the heuristic only for network files. @Tony, thanks for the pointer. I will give it a try. So you suspect that if the file is opened using file id, opened name may not return us the filename. @Alex, are you saying that you can have long file names (that don't fit in 8.3) whose corresponding short filename does not contain tilde character? I looked at http://en.wikipedia.org/wiki/8.3_filename and that doesn't seem to be the case? We use the name in display and we may re-open the file with that name from our mini-filter. Thanks. -Prasad
  Message 19 of 31  
17 Aug 12 09:24
Alex Carp
xxxxxx@gmail.com
Join Date: 23 Feb 2010
Posts To This List: 981
Mini filter peformance issues while accessing network files

Yup, that's what i'm saying. Please don't forget that it's possible for a us= er to set the short name for a file directly as well. In my opinion if all you do is display the name and use it to re-open then y= ou can just the opened name and not the normalized one. I view the normalize= d name as necessary only when you're trying to 'understand' the namespace, l= ike when you need to compare paths or possibly do things based on the path (= for path-based policy and such). =20 Thanks, Alex. Sent from my iPhone On Aug 17, 2012, at 2:35 AM, xxxxx@vmware.com wrote: > @Dejan, ok, so, we may end up reporting the name without resolving mount p= oints. I am thinking of using the heuristic only for network files. >=20 > @Tony, thanks for the pointer. I will give it a try. So you suspect that i= f the file is opened using file id, opened name may not return us the filena= me. >=20 > @Alex, are you saying that you can have long file names (that don't fit in= 8.3) whose corresponding short filename does not contain tilde character? I= looked at http://en.wikipedia.org/wiki/8.3_filename and that doesn't seem t= o be the case? We use the name in display and we may re-open the file with t= hat name from our mini-filter. >=20 > Thanks. > -Prasad >=20 >=20 >=20 >=20 >=20 > --- > NTFSD is sponsored by OSR <...excess quoted lines suppressed...> osronline.com/page.cfm?name=3DListServer
  Message 20 of 31  
21 Aug 12 07:21
Prasad Dabak
xxxxxx@vmware.com
Join Date: 09 Jun 2011
Posts To This List: 125
Mini filter peformance issues while accessing network files

@Alex, thanks for pointing this out. However, I guess this is possible only if the end user explicitly sets the short name directly? If the short name is OS generated, then, it will always contain tilde (assuming that the file name doesn't fit in 8.3) right? My concern was actually around backward compatibility. Our filter is consumed by third parties and so far we were returning them normalized names. With the change, we will start returning opened names instead and hence I wanted to understand the differences and see if it can affect them. We are trying to keep the deviation minimal. We are returning opened names only for network files. If the opened name contains tilde, then, we return normalized name. Based on the discussion so far, there are few cases which we need to worry about. 1. An explicit short name was set which doesn't not contain tilde and the file was originally opened using short name. We will end up sending short filename in this case. 2. Mount points are not resolved. We will end up returning path containing mount point instead of resolved path. 3. If the file is opened by id. I looked at Ladislav's sample to open file by id. However, apparently, it's not possible to open the file on network with id? Hence, this is probably a mute point? 4. Any other case that you can think off? Thanks. -Prasad
  Message 21 of 31  
21 Aug 12 12:48
Alex Carp
xxxxxx@gmail.com
Join Date: 23 Feb 2010
Posts To This List: 981
Mini filter peformance issues while accessing network files

1. I'm not sure whether the function used by the OS to generate short names (RtlGenerate8dot3Name) is guaranteed to always return names containing ~. I can't find anything in the documentation and the disassembly looks rather nasty :(, but from what I see (again, not 100% certain) it does seem to only generate names with ~. 2. since you're calling it from PostOpCreate (I assume you mean successful creates here) I'm not sure why you'd not see the mount points not resolved. I expect they'd be resolved by the IO manager by the time the IRP_MJ_CREATE you're looking at was issued. 3. as far as I can tell you can't open a file by fileID or objectID over SMB (I'm looking at the 2.2.13 SMB2 CREATE Request document on MSDN -> http://msdn.microsoft.com/en-us/library/cc246502(v=prot.13).aspx, which states that for the FILE_OPEN_BY_FILE_ID create option "This bit SHOULD be set to 0 and the server MUST fail the request with a STATUS_NOT_SUPPORTED error if this bit is set.<34>" ). Thanks, Alex.
  Message 22 of 31  
21 Aug 12 13:54
Prasad Dabak
xxxxxx@vmware.com
Join Date: 09 Jun 2011
Posts To This List: 125
Mini filter peformance issues while accessing network files

Thanks Alex. Yes, we are calling it from PostOpCreate. Dejan mentioned earlier that "Normalized names also resolve mount points.". Hence, I thought that opened name do not resolve it. I will confirm this though. Thanks. -Prasad
  Message 23 of 31  
21 Aug 12 14:07
Christian Allred
xxxxxx@microsoft.com
Join Date: 26 Jan 2009
Posts To This List: 104
Mini filter peformance issues while accessing network files

RtlGenerate8dot3Name happens to always generate names with ~ in them for now. This is an implementation detail that should not be relied upon to remain true. The algorithm for generating short names has changed in the past, and may change in the future. See the "Remarks" section of http://msdn.microsoft.com/en-us/library/windows/hardware/ff544633(v=vs.85).aspx for the definitions of what constitutes a "normalized", "opened", and "short" name. That section notes that mount points are not resolved in a query for FLT_FILE_NAME_OPENED. After all, the "opened name" is the name that the caller used to open the file. If the Filter Manager resolved mount points in a query for the opened name, it would no longer be the name the caller used. Mount points do of course get resolved during the open so that the actual object being opened can be accessed, but that doesn't have a bearing on the name string you get back when requesting the opened name from Filter Manager. -- Christian [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
  Message 24 of 31  
21 Aug 12 14:37
Alex Carp
xxxxxx@gmail.com
Join Date: 23 Feb 2010
Posts To This List: 981
Mini filter peformance issues while accessing network files

Wait, so my assumption up to this point has been that that opened name is not exactly "the name the caller used" but rather the "the name the IO manager used when issuing the IRP_MJ_CREATE". So for a case where you have \Device\HarddiskVolume1\foo.txt which reparses to \Device\HarddiskVolume2\bar.txt, the name the caller used might have been "c:\foo~1.txt" or "\Device\HarddiskVolume1\foo~1.txt" and the opened name for the first IRP_MJ_CREATE would be "\Device\HarddiskVolume1\ foo~1.txt" and the opened name for the second IRP_MJ_CREATE would be "\Device\HarddiskVolume2\bar.txt". Is this not the case? Thanks, Alex.
  Message 25 of 31  
21 Aug 12 15:08
ntfsd member 5961
xxxxxx@alfasp.com
Join Date:
Posts To This List: 1448
Mini filter peformance issues while accessing network files

I never actually tested it with Opened, since we require Normalized for any successful Post op. xxxxx@vmware.com wrote: > Thanks Alex. > > Yes, we are calling it from PostOpCreate. Dejan mentioned earlier that "Normalized names also resolve mount points.". Hence, I thought that opened name do not resolve it. I will confirm this though. > > Thanks. > -Prasad > > --- > NTFSD is sponsored by OSR > <...excess quoted lines suppressed...> -- Kind regards, Dejan (MSN support: xxxxx@alfasp.com) http://www.alfasp.com File system audit, security and encryption kits.
  Message 26 of 31  
21 Aug 12 15:27
Christian Allred
xxxxxx@microsoft.com
Join Date: 26 Jan 2009
Posts To This List: 104
Mini filter peformance issues while accessing network files

Alex, I spent some time wading through NTFS. The documentation must be wrong (where it says that in an opened name "Mount points are not resolved. "). So you're right, the reparse points are resolved in the opened name. This has to be the case because Filter Manager gets the opened name by sending a FileNameInformation query to the file system. NTFS, for example, services this by returning the name stored in its CCB (i.e. FsContext2 structure). That structure isn't created until the create is about to be completed (after reparse processing), and the name it uses came from the FileObject during the create. If the FileObject didn't have a no-reparse-points name in it then NTFS wouldn't have been able to complete the open. -- Christian [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
  Message 27 of 31  
21 Aug 12 15:42
Bill Wandel
xxxxxx@bwandel.com
Join Date: 14 Sep 2010
Posts To This List: 58
Mini filter peformance issues while accessing network files

What about in pre create? I didn't think that mount points were resolved if the request was for "opened name". Bill Wandel -----Original Message----- From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@microsoft.com Sent: Tuesday, August 21, 2012 3:26 PM To: Windows File Systems Devs Interest List Subject: RE:[ntfsd] Mini filter peformance issues while accessing network files Alex, I spent some time wading through NTFS. The documentation must be wrong (where it says that in an opened name "Mount points are not resolved. "). So you're right, the reparse points are resolved in the opened name. This has to be the case because Filter Manager gets the opened name by sending a FileNameInformation query to the file system. NTFS, for example, services this by returning the name stored in its CCB (i.e. FsContext2 structure). That structure isn't created until the create is about to be completed (after reparse processing), and the name it uses came from the FileObject during the create. If the FileObject didn't have a no-reparse-points name in it then NTFS wouldn't have been able to complete the open. -- Christian [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights. --- NTFSD is sponsored by OSR For our schedule of debugging and file system seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
  Message 28 of 31  
21 Aug 12 16:08
Alex Carp
xxxxxx@gmail.com
Join Date: 23 Feb 2010
Posts To This List: 981
Mini filter peformance issues while accessing network files

Hi Christian, Thanks for looking into it! That's pretty much what I thought was the case. Bill, I think even in preCreate things are similar, except there are multiple preCreates for each open that traverses several mountpoints and so it's harder to know if you're in the final preCreate or not. If my understanding is correct, in order for FltMgr to actually return a name that's not coming from the FILE_OBJECT it would have to look into the OPEN_PACKET to get it and I'm pretty sure it's not doing that. So the name must have all the mountpoints resolved up to that point in the path, regardless of whether it's in pre or postCreate. Thanks, Alex.
  Message 29 of 31  
21 Aug 12 18:31
Christian Allred
xxxxxx@microsoft.com
Join Date: 26 Jan 2009
Posts To This List: 104
Mini filter peformance issues while accessing network files

In pre-create the name is built from the FILE_OBJECT for non-open-by-ID opens. Therefore mount points are not resolved in that case. So, as Alex noted, you'd have to know if you're in the final pre-create to know whether the name you got back was completely free of mount points. Furthermore, if it is a relative open then Filter Manager queries the name from FILE_OBJECT.RelatedFileObject and then adds the portion of the path from the FILE_OBJECT. Since RelatedFileObject has to already be open for a relative open to work, the portion of the path that is covered by RelatedFileObject *does* have mount points resolved but the remainder doesn't. The moral of that story is: if you care about mount points wait until post-create before you go asking for an opened name. BTW, for open-by-ID Filter Manager actually opens the file (bypassing lower filters), asks for the name, and closes the file. This doesn't involve mount points, but I'm including it here for completeness. -- Christian [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.
  Message 30 of 31  
21 Aug 12 19:45
ntfsd member 8
xxxxxx@osr.com
Join Date:
Posts To This List: 2736
Mini filter peformance issues while accessing network files

<quote> BTW, for open-by-ID Filter Manager actually opens the file (bypassing lower= filters), asks for the name, and closes the file. This doesn't involve mo= unt points, but I'm including it here for completeness. </quote> That's nice if you're a filter driver implementing a name space extension t= hat includes object IDs. There's no clean fix for a filter that bypasses other filters, unfortunatel= y.=20 Tony OSR
  Message 31 of 31  
21 Aug 12 19:47
ntfsd member 8
xxxxxx@osr.com
Join Date:
Posts To This List: 2736
Mini filter peformance issues while accessing network files

I'm curious (since you've been looking) how NTFS would resolve cross-volume= symbolic links? In fact, we've just been discussing this very issue (reparse points) becaus= e we're looking at what happens when we start encrypting the data within th= e symbolic link (we're encrypting the names of files and directories, so th= e question of what we should return here gets quite interesting actually). Tony OSR
Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You must login to OSR Online AND be a member of the ntfsd list to be able to post.

All times are GMT -5. The time now is 04:15.


Copyright ©2015, OSR Open Systems Resources, Inc.
Based on vBulletin Copyright ©2000 - 2005, Jelsoft Enterprises Ltd.
Modified under license