In my minifilter I’m seeing post-cleanup operations where (IoGetTopLevelIrp() != NULL) - as a result my call FltQueueDeferredIoWorkItem fails.
I can’t explain this because in my pre-cleanup, if (IoGetTopLevelIrp() != NULL) I return FLT_PREOP_SUCCESS_NO_CALLBACK. Could anyone shed any light on why I am seeing post-cleanup callbacks with a top-level IRP?
I must admit it’s mostly curiosity. By design I am happy to ignore file operations with a top-level IRP so it would be simple to also ignore them in the post-cleanup callback. But it is bothering me that I can’t explain why I am seeing these post-cleanup ops!
Welcome to the whacky world of minifilters. My experience is that there are
a lot of filters and components (third party and others) that do not pay as
much attention to clearing the top level irp as you might like. It is not
unusual to see it set in precreate even (which nakes getting file names in
post create a pain).
You can try to diagnose this by setting a write-access breakpoint on the
address in the KTHREAD and seeing who sets it and doesn’t clear it.
Of course another (possibly even more likely) possibility is that some
subsidiary operation has happened using the same IRP and the completion for
that is happening with that field being set (for very good reasons) and that
that in turn is calling your post callback in a context you didn’t expect.
Did you try FLT_PREOP_SYNCHRONIZE?
Thanks! hadn’t considered FLT_PREOP_SYNCHRONIZE as a way to avoid having to do a FltQueueDeferredIoWorkItem in the post-op callback (as you can guess I’m already doing FltQueueDeferredIoWorkItem in the pre-op)
Bit of a follow-up on this.
It only seems to affect file operations on a removal USB drive, and it only affects the post-cleanup (not post-create or post-set-information)
Unfortunately neither FLT_PREOP_SYNCHRONIZE or FltDoCompletionProcessingWhenSafe help me here as I specifically need to be at PASSIVE_LEVEL for my post operation work due to a dependence on RtlStringCchCopyUnicodeString (though arguably this could be worked around)
I’ve dug up this old post:
If the pre-operation callback came in as passive level and you return
FLT_PREOP_SYNCHRONIZE the post-operation callback is guaranteed to be at
passive level since internally we are synchronizing back to that same
thread.
Returning FLT_PREOP_SYNCHRONIZE is only a PERF issue for read/write
operations which are marked asynchronous. I have not seen a perf issue
synchronizing any other operations.
Neal Christiansen
Microsoft NTFS Development Lead
This posting is provided "AS IS" with no warranties, and confers no
Rights
Does this mean FLT_PREOP_SYNCHRONIZE from a pre operation that was deferred via FltQueueDeferredIoWorkItem will mean the post-op is always PASSIVE_LEVEL ?
You can’t FLT_PREOP_SYNCHRONIZE and FltQueueDeferredIoWorkItem the same
request.
FLT_PREOP_SYNCHRONIZE forces the operation to be synchronous. This gives you
the guarantee that your PostOp callback will be called in the same context
and at the same IRQL as your PreOp callback.
FltQueueDeferredIoWorkItem posts the work to a System worker thread. You’re
called back at IRQL PASSIVE_LEVEL in the context of the System process.
You can’t pass FLT_PREOP_SYNCHRONIZE to FltCompletePendedPreOperation
because that ship has sailed. I could certainly argue that you *should* be
able to (and thus get called back in the same context and IRQL as
FltCompletePendedPreOperation), but it doesn’t work that way.
If you’re worried about your PostOp being called at DISPATCH_LEVEL you can
use FltDoCompletionProcessingWhenSafe. I don’t think this takes TopLevelIrp
into account. Though I’d be pretty surprised to ever see an IRP_MJ_CLEANUP
PostOp at DISPATCH_LEVEL (but architecturally I suppose it could happen.
-scott
OSR
@OSRDrivers
> It only seems to affect file operations on a removal USB drive, and it
only affects the post-cleanup (not post-create or post-set-information)
Have you checked the Flags sent to post callback. The whole
FLT_PREOP_SYNCRHONIZE thing falls apart during tear down. AFAIR It doesn’t
matter that you said syncrhonize you are going to be called in an arbitrary
thread. This can make things exciting (though why toplevel irp might be set
is mystery).
As an aside this makes it distinctly iffy to be holding ERESOURCES between
pre-and post calls. There are a gazillion good reasons not to do this
anyway.
R
Scott:
Apologies, I was unclear with my explanation. I of course call FLT_PREOP_PENDING after deferring the IO work item. What I meant was calling FltCompletePendedPreOperation(data, FLT_PREOP_SYNCHRONIZE, context) from the deferred IO work item handler.
Rod:
I make no assumptions about the level I am at until I have checked the FLTFL_POST_OPERATION_DRAINING flag. Is this what you are referring to? I am not holding any ERESOURCES between pre and post-calls (unless that is what SYNCHRONIZE is doing under the covers?) - just passing a stream handle context about.
aaand I’ve just read FltCompletePendingPreOperation msdn page a bit closer…
“The status value that the minifilter driver is returning for this I/O operation. Cannot be FLT_PREOP_PENDING, FLT_PREOP_SYNCHRONIZE, or FLT_PREOP_DISALLOW_FASTIO. Must be one of the following FLT_PREOP_CALLBACK_STATUS values. For more information about the effect of these values, see the Remarks section of the reference entry for PFLT_PRE_OPERATION_CALLBACK.”
Oops. Back to the drawing board 
> I make no assumptions about the level I am at until I have checked the
FLTFL_POST_OPERATION_DRAINING flag. Is this what you are referring to?
It was… Thanks
I am not holding any ERESOURCES between pre and post-calls (unless that is
what SYNCHRONIZE is doing under the covers?)
Not targetted at you, just a general observation.
/R
To back up a bit, what are you doing in your PostCleanup that you need to
queue a work item?
-scott
OSR
@OSRDrivers
FltQueryInformationFile (querying for whether file was deleted)
FltSendMessage (“push” auditing of sorts. Refactoring to a pull mechanism would not be straightforward but might have to be the answer. Doesn’t avoid need for FltQueryInformationFile though afaics)
FltGetDestinationFileNameInformation
RtlStringCchCopyUnicodeString (possibly avoidable by having fixed sized strings in contexts but would rather not)
Possibly a few other <= APC_LEVEL calls
Check the delete sample from Microsoft.
That’s not how you should check for deletes. Sample also supports
transacted deletes as well.
Also get the insider rs4 WDK and check for interesting post cleanup values
in the Information bit of the IoStatus block. This is useful only in the
upcoming versions of Win10 but might be something useful and cheaper to use
in the future.
Gabriel
www.kasardia.com
On Mon, Mar 26, 2018, 17:54 xxxxx@jazznetworks.com
wrote:
> FltQueryInformationFile (querying for whether file was deleted)
> FltSendMessage (“push” auditing of sorts. Refactoring to a pull mechanism
> would not be straightforward but might have to be the answer. Doesn’t avoid
> need for FltQueryInformationFile though afaics)
> FltGetDestinationFileNameInformation
> RtlStringCchCopyUnicodeString (possibly avoidable by having fixed sized
> strings in contexts but would rather not)
>
> Possibly a few other <= APC_LEVEL calls
>
> —
> NTFSD is sponsored by OSR
>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
></http:>
I would ditch work items and just do FLT_PREOP_SYNCHRONIZE. IRP_MJ_CLEANUP
should* be called at IRQL PASSIVE_LEVEL and is synchronous anyway, so you’re
just complicating things by trying to post.
But, there’s still a problem. Even if you just FLT_PREOP_SYNCHRONIZE, the
dangling TopLevelIrp field is going to continue to haunt you. For example,
you won’t be able to call FltGetDestinationFileNameInformation as it will
just immediately fail.
I’d be interested to track down exactly who is leaving TopLevelIrp set. Is
the TopLevelIrp field a pointer? It’s possible NTFS is not clearing it out
even though it probably should.
In the meantime, you need some way to work around this issue. I only see two
possible solutions:
-
If TopLevelIrp is not set in PreCleanup but *is* set in PostCleanup, just
clear it, do your work, and then restore whatever it was. We can say pretty
definitively that TopLevelIrp is just leftover garbage in this case, so I
wouldn’t feel guilty about clearing it. Even having to restore it is a
stretch, but it feels tidier.
-
If you really hate that you could queue up a generic work item in
PostCleanup. Your work item could then do whatever processing you need in a
fresh context. It would be a design choice then to either synchronously
block the PostCleanup waiting for your work item to complete or just letting
it complete back to the user (the file system is done with the request at
that point anyway).
*I say “should” because other filters can do terrible things…
-scott
OSR
@OSRDrivers