FltCreateFile and NotifyChangeDirectory

Hi all,
I wrote a minifilter that redirects files that are new to another folder in the same volume.
Suppose you have a new created file c:\temp\file.txt then I redirect it to c:\temp\redirected\ABA_file.txt and at the same time I create some internal .dat file in c:\temp\redirected\ )
What I do is to apply the reparse logic (STATUS_REPARSE/IO_REPARSE) in the PreCreate event and I also handle the rename event.

This basically works OK. The problem is when the files are in a network volume (\device\mup).
Here the redirection works pretty nice… but if the user in explorer right clicks on a folder and select “new text document” then first explorer calls Create with the “New Text Document.txt” (here I do the redirection from “\mypc\folder\New Text Document.txt” to “\mypc\folder\Redirected\ABA_New Text Document.txt” AND I CREATE A FILE \mypc\folder\Redirected\instruction.dat), then explorer shows the edit control allowing the user to edit that, then after few seconds if the user does not do anything, explorer finishes editing the file not allowing the user to change it name.

I discovered the problem comes from the fact that I create the \mypc\folder\Redirected\instruction.dat file. This triggers a NotifyChangeDirectory, and explorer does something that forces it to finish editing the file. I used process monitor and I saw that the QueryDirectory event was triggered passing the path of \mypc\folder\New Text Document.txt and in this case in the PRE event I create \mypc\folder\New Text Document.txt and in the post event I delete \mypc\folder\New Text Document.txt just to force explorer to believe that it exists… however this does not help.

Any other thoughts?

In the dir control, instead of creating/deleting a file, why not just
update the dir info buffer from your name to the name it should be? This
additional create/delete might also be triggering something that is
causing a refresh of the directory before Explorer allows the user to
edit the name of the file.

As well, are you removing your .dat file from the dir info buffer? There
are several things here which could be triggering that problem you are
seeing. I would suggest starting simple, not creating your .dat file,
and see if the issue exists. And proceed from there.

Pete


Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com http:</http:>
866.263.9295

------ Original Message ------
From: xxxxx@speedy.com.ar
To: “Windows File Systems Devs Interest List”
Sent: 1/5/2017 11:40:10 AM
Subject: [ntfsd] FltCreateFile and NotifyChangeDirectory

>Hi all,
>I wrote a minifilter that redirects files that are new to another
>folder in the same volume.
>Suppose you have a new created file c:\temp\file.txt then I redirect it
>to c:\temp\redirected\ABA_file.txt and at the same time I create some
>internal .dat file in c:\temp\redirected\ )
>What I do is to apply the reparse logic (STATUS_REPARSE/IO_REPARSE) in
>the PreCreate event and I also handle the rename event.
>
>This basically works OK. The problem is when the files are in a network
>volume (\device\mup).
>Here the redirection works pretty nice… but if the user in explorer
>right clicks on a folder and select “new text document” then first
>explorer calls Create with the “New Text Document.txt” (here I do the
>redirection from “\mypc\folder\New Text Document.txt” to
>“\mypc\folder\Redirected\ABA_New Text Document.txt” AND I CREATE A
>FILE \mypc\folder\Redirected\instruction.dat), then explorer shows the
>edit control allowing the user to edit that, then after few seconds if
>the user does not do anything, explorer finishes editing the file not
>allowing the user to change it name.
>
>I discovered the problem comes from the fact that I create the
>\mypc\folder\Redirected\instruction.dat file. This triggers a
>NotifyChangeDirectory, and explorer does something that forces it to
>finish editing the file. I used process monitor and I saw that the
>QueryDirectory event was triggered passing the path of
>\mypc\folder\New Text Document.txt and in this case in the PRE event I
>create \mypc\folder\New Text Document.txt and in the post event I
>delete \mypc\folder\New Text Document.txt just to force explorer to
>believe that it exists… however this does not help.
>
>Any other thoughts?
>
>—
>NTFSD is sponsored by OSR
>
>
>MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>software drivers!
>Details at http:
>
>To unsubscribe, visit the List Server section of OSR Online at
>http:</http:></http:>

Activity that causes directory change notifications (DCNs) to trigger when
they wouldn’t otherwise can certainly cause application compatibility
issues. We have been burned by this enough times* that I have a little app I
run that tells me whenever a DCN fires, useful during development to make
sure I’m not doing something to inadvertently cause a DCN.

There’s not always a good answer to this, it’s not like you can get Explorer
to change its behavior for you.

With respect to your situation, I lost you a bit at the end. Are you saying
the DCN buffer indicated that \mypc\folder\New Text Document.txt was
modified? Does this file actually get created or just the redirected files?

Have you tried reproducing the Explorer behavior without your filter? If the
problem really is that you’re creating a file in a subdirectory while the
edit control is active, you should be able to reproduce this behavior
without any of your code present. While this doesn’t give you an answer, it
would confirm that this is indeed a DCN problem (and that the DCN isn’t just
a red herring).

*We had a bug (once!) where simply opening a file caused us to trigger a
DCN. This caused Explorer to think that the file changed and open the file
again. Which caused us to trigger a DCN…Which caused Explorer to think
that the file changed…

-scott
OSR
@OSRDrivers

wrote in message news:xxxxx@ntfsd…

Hi all,
I wrote a minifilter that redirects files that are new to another folder in
the same volume.
Suppose you have a new created file c:\temp\file.txt then I redirect it to
c:\temp\redirected\ABA_file.txt and at the same time I create some internal
.dat file in c:\temp\redirected\ )
What I do is to apply the reparse logic (STATUS_REPARSE/IO_REPARSE) in the
PreCreate event and I also handle the rename event.

This basically works OK. The problem is when the files are in a network
volume (\device\mup).
Here the redirection works pretty nice… but if the user in explorer right
clicks on a folder and select “new text document” then first explorer calls
Create with the “New Text Document.txt” (here I do the redirection from
“\mypc\folder\New Text Document.txt” to “\mypc\folder\Redirected\ABA_New
Text Document.txt” AND I CREATE A FILE
\mypc\folder\Redirected\instruction.dat), then explorer shows the edit
control allowing the user to edit that, then after few seconds if the user
does not do anything, explorer finishes editing the file not allowing the
user to change it name.

I discovered the problem comes from the fact that I create the
\mypc\folder\Redirected\instruction.dat file. This triggers a
NotifyChangeDirectory, and explorer does something that forces it to finish
editing the file. I used process monitor and I saw that the QueryDirectory
event was triggered passing the path of \mypc\folder\New Text Document.txt
and in this case in the PRE event I create \mypc\folder\New Text
Document.txt and in the post event I delete \mypc\folder\New Text
Document.txt just to force explorer to believe that it exists… however
this does not help.

Any other thoughts?

Thanks Scott for your answers.
If I do not create the .dat files in a subfolder then the issue does not arise.
I do not create and delete the file anymore in the QueryDirectory events since it was also causing the issue.
The problem is that I must create the .dat files in the folder since it’s a way I have to tell a worker process to process that file that was just redirected to that folder.
For example the user creates a file in c:\temp\myfile.txt then this creation is redirected (reparse) to c:\temp\Redirected\ABA_myfile.txt… at this point this is OK but the problem comes because I use FltCreateFile to create a file c:\temp\Redirected\instructions.dat. This triggers the Notify event that alerts explorer that the c:\temp\myfile.txt does not exist anymore. (this happens only in UNC paths)

I tried creating files (e.g. in another computer) while explorer had the edit control being shown and it’s OK. Of course if I delete the file being edited then explorer stops showing the edit control. So this confirms that the issue is because explorer queries if file.txt still exists when it receives a notify event.

Is there any way to:

  • Either call FltCreateFile without triggering any notification
  • Intercept the notification event (there is one) and “prevent it from being processed”. I tried returning STATUS_PENDING without success.

At this point what I am doing is to create the .dat file somewhere else but it’s an big effort and risk only for this “little issue”.

Thanks again.

That makes sense (in a twisted Shell sort of way)…Explorer “expects” the
first DCN and is happy. But the second one triggers and now Explorer
realizes that it has an active edit control on a missing file and aborts it.
The DCN is really just a side effect of the real problem, which is that the
application thinks it successfully created a file but the file no longer
exists. Adding the file entry to the directory enumeration buffer would also
presumably solve this problem.

To your questions:

  • There is no way to create a new file and not trigger the DCN.

  • You can filter IRP_MJ_QUERY_DIRECTORY/IRP_MN_NOTIFY_CHANGE_DIRECTORY to
    modify the response that the application gets back. Having done this before,
    I can attest to it being a major pain. The associated buffer can be
    director, buffered, or neither by the time you look at. You also need to
    decide if you just want to remove your entries, remove all entries, fail the
    request, resubmit the request, etc. There is much madness down this path.

I guess I have two more questions:

  1. Do you ever eventually create “c:\temp\myfile.txt”? Or does that file
    never exist? If the file WILL eventually exist maybe you can hold up the DCN
    until the file is created (or change the order of operations so that the
    .dat file is created after the “real” file?)

  2. Does this really matter for you? Is this an expected use case (Right
    Click->New File on a network share) for your product? Not that I’m
    advocating for create weird Shell behaviors with your filter present, but
    just confirming that it matters for your environment and customers.

-scott
OSR
@OSRDrivers

wrote in message news:xxxxx@ntfsd…

Thanks Scott for your answers.
If I do not create the .dat files in a subfolder then the issue does not
arise.
I do not create and delete the file anymore in the QueryDirectory events
since it was also causing the issue.
The problem is that I must create the .dat files in the folder since it’s a
way I have to tell a worker process to process that file that was just
redirected to that folder.
For example the user creates a file in c:\temp\myfile.txt then this
creation is redirected (reparse) to c:\temp\Redirected\ABA_myfile.txt… at
this point this is OK but the problem comes because I use FltCreateFile to
create a file c:\temp\Redirected\instructions.dat. This triggers the Notify
event that alerts explorer that the c:\temp\myfile.txt does not exist
anymore. (this happens only in UNC paths)

I tried creating files (e.g. in another computer) while explorer had the
edit control being shown and it’s OK. Of course if I delete the file being
edited then explorer stops showing the edit control. So this confirms that
the issue is because explorer queries if file.txt still exists when it
receives a notify event.

Is there any way to:

  • Either call FltCreateFile without triggering any notification
  • Intercept the notification event (there is one) and “prevent it from being
    processed”. I tried returning STATUS_PENDING without success.

At this point what I am doing is to create the .dat file somewhere else but
it’s an big effort and risk only for this “little issue”.

Thanks again.

Hi Scott,
“c:\temp\myfile.txt” can or not exists later. The minifilter redirects “c:\temp\myfile.txt” to “c:\temp\Redirected\ABA_myfile.txt” and it creates c:\temp\Redirected\instructions.dat.
Then there is a user mode app that reads this .dat file and does things in ABA_myfile.txt… it can create a kind of c:\temp\myfile.txt.processed" or must copy the .txt back to c:\temp\myfile.txt.

The UNC paths must also work in this product.

I cannot change the order since instructions.dat tells the user mode app what to do.

You said "you can hold up the DCN until the file is created "
In my case I know when the user finished editing the file name… is it possible to hold on the DNC?

Thanks

Honestly I think the better option is to preserve the existence of the new
file in the original location by adding it to the directory listing. That
way the directory listing is correct and you save yourself from additional
interop issues down the line (e.g. applications sometimes get the attributes
of a file by listing the directory).

That being said, adding an entry to a directory listing is frustratingly
complicated. The easiest thing to do is create the c:\temp\myfile.txt stub
file and leave it while you do all of your additional processing. That
preserves the original directory structure while you do your work.

My idea with holding up the DCN would be to register a completion routine
and prevent the request from completing back to the app until you’re done
with your processing. Presumably you would allow the DCN to complete when
you have finally gone back and created the original file, at which points
things would “just work.” Though if you never create the original file
you’re always going to have this problem unless you never let the DCN
complete (e.g. by resubmitting it back to the file system).

The problem that you’re going to have though is that you can’t know if the
DCN is being triggered by your activity (in which case you want to suppress
it) or someone else’s activity (in which case you don’t) unless you look at
the data being returned. To complicate things further, the returned data may
contain both your activity and someone else’s activity, in which case you
would need to filter out just your activity.

This all quickly gets complicated. Better to just preserve the app’s
expectation that a file just created will exist in the specified directory.

-scott
OSR
@OSRDrivers

wrote in message news:xxxxx@ntfsd…

Hi Scott,
“c:\temp\myfile.txt” can or not exists later. The minifilter redirects
“c:\temp\myfile.txt” to “c:\temp\Redirected\ABA_myfile.txt” and it creates
c:\temp\Redirected\instructions.dat.
Then there is a user mode app that reads this .dat file and does things in
ABA_myfile.txt… it can create a kind of c:\temp\myfile.txt.processed" or
must copy the .txt back to c:\temp\myfile.txt.

The UNC paths must also work in this product.

I cannot change the order since instructions.dat tells the user mode app
what to do.

You said "you can hold up the DCN until the file is created "
In my case I know when the user finished editing the file name… is it
possible to hold on the DNC?

Thanks

Thanks Scott.
In fact the problem is that the PreCreate event for “New Text Document.txt” happens BEFORE explorer creates the edit control to let the user edit the file. At this time I already redirected that path so the file is in c:\temp\Redir\ABA_file.txt. What I did was to detect that Windows explorer is editing a file and I copy c:\temp\Redir\ABA_file.txt back to c:\temp\Redir\file.txt but in this case explorer still stops editing the file (seems reasonable since the file name is the same but the id is different).

The ideal solution is as you say to not redirect it and to do it when the user finishes editing it… question is how to do this.

What I did was to implement a .dll that does function interception and it intercepts the CreateWindow API and when it finds that such API is called with the “Edit” class the I assume it may be in this situation. As I told you, the problem is that explorer first creates “New Text Document.txt” and then it calls CreateWindow… so it’s too late. If I do the reparse at this stage I must keep doing the reparse. The ideal is to know the fact that the PreCreate event is triggered due to a “New/Text document” action. My minifilter redirects all IO to that folder.

Thanks

I didn’t mean to suggest that you should take some action when the edit box
is finished. Surely the right solution does not involve injecting code into
Explorer…

Taking this way back to first principals: you have created a solution where
an application successfully creates a file BUT if the application looks for
the file it does not actually exist. This has manifested itself in this
particular Explorer behavior, but you have a generic issue that could pop up
in any application an cause any amount of unexpected behaviors.

There are lots of ways out of this. Off the top of my head:

  1. Create a stub “New Text Document.txt” (with or without a reparse point to
    your actual file, depending on your design)

  2. Virtualize the existence of “New Text Document.txt”, including adding it
    to directory queries by Explorer

  3. Play games with the DCN, though this is a spot fix that doesn’t really
    solve your design problem

Without knowing more about your design and requirements I can’t really
provide any more specific guidance.

You might want to look at the NameChanger sample. It doesn’t do exactly what
you’re doing, but it’s pretty close and might give you some ideas as to the
complexities and trade offs involved.

-scott
OSR
@OSRDrivers

wrote in message news:xxxxx@ntfsd…

Thanks Scott.
In fact the problem is that the PreCreate event for “New Text Document.txt”
happens BEFORE explorer creates the edit control to let the user edit the
file. At this time I already redirected that path so the file is in
c:\temp\Redir\ABA_file.txt. What I did was to detect that Windows explorer
is editing a file and I copy c:\temp\Redir\ABA_file.txt back to
c:\temp\Redir\file.txt but in this case explorer still stops editing the
file (seems reasonable since the file name is the same but the id is
different).

The ideal solution is as you say to not redirect it and to do it when the
user finishes editing it… question is how to do this.

What I did was to implement a .dll that does function interception and it
intercepts the CreateWindow API and when it finds that such API is called
with the “Edit” class the I assume it may be in this situation. As I told
you, the problem is that explorer first creates “New Text Document.txt” and
then it calls CreateWindow… so it’s too late. If I do the reparse at this
stage I must keep doing the reparse. The ideal is to know the fact that the
PreCreate event is triggered due to a “New/Text document” action. My
minifilter redirects all IO to that folder.

Thanks

Thanks Scott,
In fact the product I am coding needs to process a file in real-time BEFORE that file is “visible” for end users or applications.
Suppose that a file c:\folder\file.txt is created because of a copy operation, a save as, or an external application creating it. The file that is in c:\folder will finally be c:\folder\file.processed.
So in case you do in a command prompt:

Copy file.txt c:\folder
Then when you do:
Dir c:\folder
you should see:
file.processed

This can be anything… it can be for example a PDF convertor (here .processed can be .pdf).

The way I do is to do a reparse and to redirect the creation to another folder… then an external application (user mode) sees there is a request and it finally copies file.processed to c:\folder

So it’s not a full virtualization since explorer should really see the real content in c:\folder, and external applications should also see the real content.

I can understand it’s not an easy product since suppose the user has a .bat file with
echo hello > c:\temp\file.txt
cat c:\temp\file.txt

it should work… but I already covered such cases in the code…

So my concern is that if the user makes a new text document in \myshare\folder then this problem happens.

Today I discovered that what explorer does is to do QueryDirectory for c:\temp and then it analyzes all files and it discovers that “New Text document.txt” is no longer here. This QueryDirectory is triggered mainly because of IRP_MN_NOTIFY_CHANGE_DIRECTORY.

What I saw is that if in IRP_MN_NOTIFY_CHANGE_DIRECTORY I detect that I am in explorer and that the edit control is being shown, I pause in the PRE event until the edit control finishes and this does the trick.

Hope this helps.