Driver Problems? Questions? Issues?
Put OSR's experience to work for you! Contact us for assistance with:
  • Creating the right design for your requirements
  • Reviewing your existing driver code
  • Analyzing driver reliability/performance issues
  • Custom training mixed with consulting and focused directly on your specific areas of interest/concern.
Check us out. OSR, the Windows driver experts.

On-Access, Transparent, Per-File Data Encryption:

OSR's File Encryption Solution Framework (FESF) provides all the infrastructure you need to build a transparent file encryption product REALLY FAST.

Super flexible policy determination and customization, all done in user-mode. Extensive starter/sample code provided.

Proven, robust, flexible. In use in multiple commercial products.

Currently available on Windows. FESF for Linux will ship in 2018.

For more info: https://www.osr.com/fesf

Go Back   OSR Online Lists > ntfsd
link FltCreateFile and NotifyChangeDirectory
Welcome, Guest
You must login to post to this list
  Message 1 of 10  
05 Jan 17 13:42
Pablo Glombovsky
xxxxxx@speedy.com.ar
Join Date: 19 Aug 2016
Posts To This List: 26
FltCreateFile and NotifyChangeDirectory
Hi all, I wrote a minifilter that redirects files that are new to another folder in the same volume. Suppose you have a new created file c:\temp\file.txt then I redirect it to c:\temp\redirected\ABA_file.txt and at the same time I create some internal .dat file in c:\temp\redirected\ ) What I do is to apply the reparse logic (STATUS_REPARSE/IO_REPARSE) in the PreCreate event and I also handle the rename event. This basically works OK. The problem is when the files are in a network volume (\device\mup). Here the redirection works pretty nice... but if the user in explorer right clicks on a folder and select "new text document" then first explorer calls Create with the "New Text Document.txt" (here I do the redirection from "\\mypc\folder\New Text Document.txt" to "\\mypc\folder\Redirected\ABA_New Text Document.txt" AND I CREATE A FILE \\mypc\folder\Redirected\instruction.dat), then explorer shows the edit control allowing the user to edit that, then after few seconds if the user does not do anything, explorer finishes editing the file not allowing the user to change it name. I discovered the problem comes from the fact that I create the \\mypc\folder\Redirected\instruction.dat file. This triggers a NotifyChangeDirectory, and explorer does something that forces it to finish editing the file. I used process monitor and I saw that the QueryDirectory event was triggered passing the path of \\mypc\folder\New Text Document.txt and in this case in the PRE event I create \\mypc\folder\New Text Document.txt and in the post event I delete \\mypc\folder\New Text Document.txt just to force explorer to believe that it exists... however this does not help. Any other thoughts?
  Message 2 of 10  
06 Jan 17 10:25
Peter Scott
xxxxxx@kerneldrivers.com
Join Date: 17 Feb 2012
Posts To This List: 669
FltCreateFile and NotifyChangeDirectory
In the dir control, instead of creating/deleting a file, why not just=20 update the dir info buffer from your name to the name it should be? This= =20 additional create/delete might also be triggering something that is=20 causing a refresh of the directory before Explorer allows the user to=20 edit the name of the file. As well, are you removing your .dat file from the dir info buffer? There= =20 are several things here which could be triggering that problem you are=20 seeing. I would suggest starting simple, not creating your .dat file,=20 and see if the issue exists. And proceed from there. Pete -- Kernel Drivers Windows File System and Device Driver Consulting www.KernelDrivers.com <http://www.kerneldrivers.com/> 866.263.9295 ------ Original Message ------ From: xxxxx@speedy.com.ar To: "Windows File Systems Devs Interest List" <xxxxx@lists.osr.com> Sent: 1/5/2017 11:40:10 AM Subject: [ntfsd] FltCreateFile and NotifyChangeDirectory >Hi all, >I wrote a minifilter that redirects files that are new to another=20 >folder in the same volume. >Suppose you have a new created file c:\temp\file.txt then I redirect it= =20 >to c:\temp\redirected\ABA_file.txt and at the same time I create some=20 >internal .dat file in c:\temp\redirected\ ) >What I do is to apply the reparse logic (STATUS_REPARSE/IO_REPARSE) in=20 >the PreCreate event and I also handle the rename event. > >This basically works OK. The problem is when the files are in a network= =20 >volume (\device\mup). >Here the redirection works pretty nice... but if the user in explorer=20 >right clicks on a folder and select "new text document" then first=20 >explorer calls Create with the "New Text Document.txt" (here I do the=20 >redirection from "\\mypc\folder\New Text Document.txt" to=20 >"\\mypc\folder\Redirected\ABA_New Text Document.txt" AND I CREATE A=20 >FILE \\mypc\folder\Redirected\instruction.dat), then explorer shows the= =20 >edit control allowing the user to edit that, then after few seconds if=20 >the user does not do anything, explorer finishes editing the file not=20 >allowing the user to change it name. > >I discovered the problem comes from the fact that I create the=20 >\\mypc\folder\Redirected\instruction.dat file. This triggers a=20 >NotifyChangeDirectory, and explorer does something that forces it to=20 >finish editing the file. I used process monitor and I saw that the=20 >QueryDirectory event was triggered passing the path of=20 >\\mypc\folder\New Text Document.txt and in this case in the PRE event I= =20 >create \\mypc\folder\New Text Document.txt and in the post event I=20 >delete \\mypc\folder\New Text Document.txt just to force explorer to=20 >believe that it exists... however this does not help. > >Any other thoughts? > >--- >NTFSD is sponsored by OSR > > <...excess quoted lines suppressed...>
  Message 3 of 10  
06 Jan 17 11:11
Scott Noone
xxxxxx@osr.com
Join Date: 10 Jul 2002
Posts To This List: 920
List Moderator
FltCreateFile and NotifyChangeDirectory
Activity that causes directory change notifications (DCNs) to trigger when they wouldn't otherwise can certainly cause application compatibility issues. We have been burned by this enough times* that I have a little app I run that tells me whenever a DCN fires, useful during development to make sure I'm not doing something to inadvertently cause a DCN. There's not always a good answer to this, it's not like you can get Explorer to change its behavior for you. With respect to your situation, I lost you a bit at the end. Are you saying the DCN buffer indicated that \\mypc\folder\New Text Document.txt was modified? Does this file actually get created or just the redirected files? Have you tried reproducing the Explorer behavior without your filter? If the problem really is that you're creating a file in a subdirectory while the edit control is active, you should be able to reproduce this behavior without any of your code present. While this doesn't give you an answer, it would confirm that this is indeed a DCN problem (and that the DCN isn't just a red herring). *We had a bug (once!) where simply opening a file caused us to trigger a DCN. This caused Explorer to think that the file changed and open the file again. Which caused us to trigger a DCN...Which caused Explorer to think that the file changed... -scott OSR @OSRDrivers wrote in message news:102062@ntfsd... Hi all, I wrote a minifilter that redirects files that are new to another folder in the same volume. Suppose you have a new created file c:\temp\file.txt then I redirect it to c:\temp\redirected\ABA_file.txt and at the same time I create some internal .dat file in c:\temp\redirected\ ) What I do is to apply the reparse logic (STATUS_REPARSE/IO_REPARSE) in the PreCreate event and I also handle the rename event. This basically works OK. The problem is when the files are in a network volume (\device\mup). Here the redirection works pretty nice... but if the user in explorer right clicks on a folder and select "new text document" then first explorer calls Create with the "New Text Document.txt" (here I do the redirection from "\\mypc\folder\New Text Document.txt" to "\\mypc\folder\Redirected\ABA_New Text Document.txt" AND I CREATE A FILE \\mypc\folder\Redirected\instruction.dat), then explorer shows the edit control allowing the user to edit that, then after few seconds if the user does not do anything, explorer finishes editing the file not allowing the user to change it name. I discovered the problem comes from the fact that I create the \\mypc\folder\Redirected\instruction.dat file. This triggers a NotifyChangeDirectory, and explorer does something that forces it to finish editing the file. I used process monitor and I saw that the QueryDirectory event was triggered passing the path of \\mypc\folder\New Text Document.txt and in this case in the PRE event I create \\mypc\folder\New Text Document.txt and in the post event I delete \\mypc\folder\New Text Document.txt just to force explorer to believe that it exists... however this does not help. Any other thoughts?
  Message 4 of 10  
06 Jan 17 11:31
Pablo Glombovsky
xxxxxx@speedy.com.ar
Join Date: 19 Aug 2016
Posts To This List: 26
FltCreateFile and NotifyChangeDirectory
Thanks Scott for your answers. If I do not create the .dat files in a subfolder then the issue does not arise. I do not create and delete the file anymore in the QueryDirectory events since it was also causing the issue. The problem is that I must create the .dat files in the folder since it's a way I have to tell a worker process to process that file that was just redirected to that folder. For example the user creates a file in c:\temp\myfile.txt then this creation is redirected (reparse) to c:\temp\Redirected\ABA_myfile.txt.... at this point this is OK but the problem comes because I use FltCreateFile to create a file c:\temp\Redirected\instructions.dat. This triggers the Notify event that alerts explorer that the c:\temp\myfile.txt does not exist anymore. (this happens only in UNC paths) I tried creating files (e.g. in another computer) while explorer had the edit control being shown and it's OK. Of course if I delete the file being edited then explorer stops showing the edit control. So this confirms that the issue is because explorer queries if file.txt still exists when it receives a notify event. Is there any way to: + Either call FltCreateFile without triggering any notification + Intercept the notification event (there is one) and "prevent it from being processed". I tried returning STATUS_PENDING without success. At this point what I am doing is to create the .dat file somewhere else but it's an big effort and risk only for this "little issue". Thanks again.
  Message 5 of 10  
06 Jan 17 16:32
Scott Noone
xxxxxx@osr.com
Join Date: 10 Jul 2002
Posts To This List: 920
List Moderator
FltCreateFile and NotifyChangeDirectory
That makes sense (in a twisted Shell sort of way)...Explorer "expects" the first DCN and is happy. But the second one triggers and now Explorer realizes that it has an active edit control on a missing file and aborts it. The DCN is really just a side effect of the real problem, which is that the application thinks it successfully created a file but the file no longer exists. Adding the file entry to the directory enumeration buffer would also presumably solve this problem. To your questions: + There is no way to create a new file and not trigger the DCN. + You can filter IRP_MJ_QUERY_DIRECTORY/IRP_MN_NOTIFY_CHANGE_DIRECTORY to modify the response that the application gets back. Having done this before, I can attest to it being a major pain. The associated buffer can be director, buffered, or neither by the time you look at. You also need to decide if you just want to remove your entries, remove all entries, fail the request, resubmit the request, etc. There is much madness down this path. I guess I have two more questions: 1. Do you ever eventually create "c:\temp\myfile.txt"? Or does that file never exist? If the file WILL eventually exist maybe you can hold up the DCN until the file is created (or change the order of operations so that the .dat file is created after the "real" file?) 2. Does this really matter for you? Is this an expected use case (Right Click->New File on a network share) for your product? Not that I'm advocating for create weird Shell behaviors with your filter present, but just confirming that it matters for your environment and customers. -scott OSR @OSRDrivers wrote in message news:102066@ntfsd... Thanks Scott for your answers. If I do not create the .dat files in a subfolder then the issue does not arise. I do not create and delete the file anymore in the QueryDirectory events since it was also causing the issue. The problem is that I must create the .dat files in the folder since it's a way I have to tell a worker process to process that file that was just redirected to that folder. For example the user creates a file in c:\temp\myfile.txt then this creation is redirected (reparse) to c:\temp\Redirected\ABA_myfile.txt.... at this point this is OK but the problem comes because I use FltCreateFile to create a file c:\temp\Redirected\instructions.dat. This triggers the Notify event that alerts explorer that the c:\temp\myfile.txt does not exist anymore. (this happens only in UNC paths) I tried creating files (e.g. in another computer) while explorer had the edit control being shown and it's OK. Of course if I delete the file being edited then explorer stops showing the edit control. So this confirms that the issue is because explorer queries if file.txt still exists when it receives a notify event. Is there any way to: + Either call FltCreateFile without triggering any notification + Intercept the notification event (there is one) and "prevent it from being processed". I tried returning STATUS_PENDING without success. At this point what I am doing is to create the .dat file somewhere else but it's an big effort and risk only for this "little issue". Thanks again.
  Message 6 of 10  
09 Jan 17 06:51
Pablo Glombovsky
xxxxxx@speedy.com.ar
Join Date: 19 Aug 2016
Posts To This List: 26
FltCreateFile and NotifyChangeDirectory
Hi Scott, "c:\temp\myfile.txt" can or not exists later. The minifilter redirects "c:\temp\myfile.txt" to "c:\temp\Redirected\ABA_myfile.txt" and it creates c:\temp\Redirected\instructions.dat. Then there is a user mode app that reads this .dat file and does things in ABA_myfile.txt... it can create a kind of c:\temp\myfile.txt.processed" or must copy the .txt back to c:\temp\myfile.txt. The UNC paths must also work in this product. I cannot change the order since instructions.dat tells the user mode app what to do. You said "you can hold up the DCN until the file is created " In my case I know when the user finished editing the file name.... is it possible to hold on the DNC? Thanks
  Message 7 of 10  
09 Jan 17 09:44
Scott Noone
xxxxxx@osr.com
Join Date: 10 Jul 2002
Posts To This List: 920
List Moderator
FltCreateFile and NotifyChangeDirectory
Honestly I think the better option is to preserve the existence of the new file in the original location by adding it to the directory listing. That way the directory listing is correct and you save yourself from additional interop issues down the line (e.g. applications sometimes get the attributes of a file by listing the directory). That being said, adding an entry to a directory listing is frustratingly complicated. The easiest thing to do is create the c:\temp\myfile.txt stub file and leave it while you do all of your additional processing. That preserves the original directory structure while you do your work. My idea with holding up the DCN would be to register a completion routine and prevent the request from completing back to the app until you're done with your processing. Presumably you would allow the DCN to complete when you have finally gone back and created the original file, at which points things would "just work." Though if you never create the original file you're always going to have this problem unless you never let the DCN complete (e.g. by resubmitting it back to the file system). The problem that you're going to have though is that you can't know if the DCN is being triggered by your activity (in which case you want to suppress it) or someone else's activity (in which case you don’t) unless you look at the data being returned. To complicate things further, the returned data may contain both your activity and someone else's activity, in which case you would need to filter out just your activity. This all quickly gets complicated. Better to just preserve the app's expectation that a file just created will exist in the specified directory. -scott OSR @OSRDrivers wrote in message news:102070@ntfsd... Hi Scott, "c:\temp\myfile.txt" can or not exists later. The minifilter redirects "c:\temp\myfile.txt" to "c:\temp\Redirected\ABA_myfile.txt" and it creates c:\temp\Redirected\instructions.dat. Then there is a user mode app that reads this .dat file and does things in ABA_myfile.txt... it can create a kind of c:\temp\myfile.txt.processed" or must copy the .txt back to c:\temp\myfile.txt. The UNC paths must also work in this product. I cannot change the order since instructions.dat tells the user mode app what to do. You said "you can hold up the DCN until the file is created " In my case I know when the user finished editing the file name.... is it possible to hold on the DNC? Thanks
  Message 8 of 10  
09 Jan 17 10:21
Pablo Glombovsky
xxxxxx@speedy.com.ar
Join Date: 19 Aug 2016
Posts To This List: 26
FltCreateFile and NotifyChangeDirectory
Thanks Scott. In fact the problem is that the PreCreate event for "New Text Document.txt" happens BEFORE explorer creates the edit control to let the user edit the file. At this time I already redirected that path so the file is in c:\temp\Redir\ABA_file.txt. What I did was to detect that Windows explorer is editing a file and I copy c:\temp\Redir\ABA_file.txt back to c:\temp\Redir\file.txt but in this case explorer still stops editing the file (seems reasonable since the file name is the same but the id is different). The ideal solution is as you say to not redirect it and to do it when the user finishes editing it... question is how to do this. What I did was to implement a .dll that does function interception and it intercepts the CreateWindow API and when it finds that such API is called with the "Edit" class the I assume it may be in this situation. As I told you, the problem is that explorer first creates "New Text Document.txt" and then it calls CreateWindow.... so it's too late. If I do the reparse at this stage I must keep doing the reparse. The ideal is to know the fact that the PreCreate event is triggered due to a "New/Text document" action. My minifilter redirects all IO to that folder. Thanks
  Message 9 of 10  
11 Jan 17 11:55
Scott Noone
xxxxxx@osr.com
Join Date: 10 Jul 2002
Posts To This List: 920
List Moderator
FltCreateFile and NotifyChangeDirectory
I didn't mean to suggest that you should take some action when the edit box is finished. Surely the right solution does not involve injecting code into Explorer... Taking this way back to first principals: you have created a solution where an application successfully creates a file BUT if the application looks for the file it does not actually exist. This has manifested itself in this particular Explorer behavior, but you have a generic issue that could pop up in any application an cause any amount of unexpected behaviors. There are lots of ways out of this. Off the top of my head: 1. Create a stub "New Text Document.txt" (with or without a reparse point to your actual file, depending on your design) 2. Virtualize the existence of "New Text Document.txt", including adding it to directory queries by Explorer 3. Play games with the DCN, though this is a spot fix that doesn't really solve your design problem Without knowing more about your design and requirements I can't really provide any more specific guidance. You might want to look at the NameChanger sample. It doesn't do exactly what you're doing, but it's pretty close and might give you some ideas as to the complexities and trade offs involved. -scott OSR @OSRDrivers wrote in message news:102076@ntfsd... Thanks Scott. In fact the problem is that the PreCreate event for "New Text Document.txt" happens BEFORE explorer creates the edit control to let the user edit the file. At this time I already redirected that path so the file is in c:\temp\Redir\ABA_file.txt. What I did was to detect that Windows explorer is editing a file and I copy c:\temp\Redir\ABA_file.txt back to c:\temp\Redir\file.txt but in this case explorer still stops editing the file (seems reasonable since the file name is the same but the id is different). The ideal solution is as you say to not redirect it and to do it when the user finishes editing it... question is how to do this. What I did was to implement a .dll that does function interception and it intercepts the CreateWindow API and when it finds that such API is called with the "Edit" class the I assume it may be in this situation. As I told you, the problem is that explorer first creates "New Text Document.txt" and then it calls CreateWindow.... so it's too late. If I do the reparse at this stage I must keep doing the reparse. The ideal is to know the fact that the PreCreate event is triggered due to a "New/Text document" action. My minifilter redirects all IO to that folder. Thanks
  Message 10 of 10  
11 Jan 17 13:04
Pablo Glombovsky
xxxxxx@speedy.com.ar
Join Date: 19 Aug 2016
Posts To This List: 26
FltCreateFile and NotifyChangeDirectory
Thanks Scott, In fact the product I am coding needs to process a file in real-time BEFORE that file is "visible" for end users or applications. Suppose that a file c:\folder\file.txt is created because of a copy operation, a save as, or an external application creating it. The file that is in c:\folder will finally be c:\folder\file.processed. So in case you do in a command prompt: > Copy file.txt c:\folder Then when you do: > Dir c:\folder you should see: file.processed This can be anything.... it can be for example a PDF convertor (here .processed can be .pdf). The way I do is to do a reparse and to redirect the creation to another folder... then an external application (user mode) sees there is a request and it finally copies file.processed to c:\folder So it's not a full virtualization since explorer should really see the real content in c:\folder, and external applications should also see the real content. I can understand it's not an easy product since suppose the user has a .bat file with echo hello > c:\temp\file.txt cat c:\temp\file.txt it should work... but I already covered such cases in the code... So my concern is that if the user makes a new text document in \\myshare\folder then this problem happens. Today I discovered that what explorer does is to do QueryDirectory for c:\temp and then it analyzes all files and it discovers that "New Text document.txt" is no longer here. This QueryDirectory is triggered mainly because of IRP_MN_NOTIFY_CHANGE_DIRECTORY. What I saw is that if in IRP_MN_NOTIFY_CHANGE_DIRECTORY I detect that I am in explorer and that the edit control is being shown, I pause in the PRE event until the edit control finishes and this does the trick. Hope this helps.
Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You must login to OSR Online AND be a member of the ntfsd list to be able to post.

All times are GMT -5. The time now is 02:39.

Contact Us - Osr Online Homepage - Top

Copyright ©2015, OSR Open Systems Resources, Inc.
Based on vBulletin Copyright ©2000 - 2005, Jelsoft Enterprises Ltd.
Modified under license