I have a minifilter driver. I’d like to be able to fetch pieces of data “on demand”. Thus, when an IRP_MJ_READ request comes in for a particular file, I want to signal a user mode app to actually go and fetch the contents and write them to the file before the read is actually completed. When I try to use FltQueueDeferredIoWorkItem to pend all of the IRP_MJ_READ requests, they all fail with STATUS_FLT_NOT_SAFE_TO_POST_OPERATION. I’m not sure which of the criteria that I’m failing. I’ve simply opened a file with notepad, and am trying to pend the read operations performed by notepad to the work queue. Is there any obvious reason why this wouldn’t work?
You must be trying to pend a paging io request.
On 29-May-2014, at 11:53 pm, xxxxx@comcast.net wrote:
I have a minifilter driver. I’d like to be able to fetch pieces of data “on demand”. Thus, when an IRP_MJ_READ request comes in for a particular file, I want to signal a user mode app to actually go and fetch the contents and write them to the file before the read is actually completed. When I try to use FltQueueDeferredIoWorkItem to pend all of the IRP_MJ_READ requests, they all fail with STATUS_FLT_NOT_SAFE_TO_POST_OPERATION. I’m not sure which of the criteria that I’m failing. I’ve simply opened a file with notepad, and am trying to pend the read operations performed by notepad to the work queue. Is there any obvious reason why this wouldn’t work?
NTFSD is sponsored by OSR
OSR is hiring!! Info at http://www.osr.com/careers
For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
When they say a “paging IO request” I assume that we’re talking about the OS reading or writing to pagefile.sys ?
First, I match the file name - so I know that the read is particularly for myfile.txt, and these reads all occur immediately after I open the file in notepad. Each and every one of them fails with that error.
It’s either paging I/O (IRP_PAGING_IO bit is set) or the top level IRP field (IoGetTopLevelIrp()) is not NULL.
If you wish to pend such operations: (1) you need to understand the locking restrictions inherent in these states; (2) you need to use your own queue.
In your case, you should signal the user mode application and then pend the current thread until user mode has signaled that the data is present (ergo, use a KEVENT and call KeSetEvent when the user mode stuff is done).
In fact, there’s no benefit in almost any read case for posting it - the caller is 99% likely to want synchronous I/O anyway and when they get back STATUS_PENDING the next thing they do is call KeWaitForSingleObject on an event (e.g., Irp->UserEvent). That’s definitely the case for paging read operations (they are *always* synchronous).
Fair warning: locking can be complicated here, so you will need to ensure your user mode application can write into that region of the file, even though this other thread is likely holding lock state.
Tony
OSR
Hi Tony - I posted at the same time you did… But your great reply leads me to another question:
When I tried to block the thread upon receipt of the IRP_MJ_READ for my .txt file (waiting for a reply from FltSendMessage), the application was unable to do any WriteFiles. After observing that, I made the assumption that ‘pausing’ the IRP_MJ_READ was effectively halting the IO, and that write requests wouldn’t go through (or any other IO requests) while the filter was blocked in the pre-callback. Is this assumption correct? Or could I have been doing something else wrong?
Or could it be that blocking in this way, (waiting for a reply for FltSendMessage) is the thing that is preventing other IO from being performed?
Executables are always loaded (read) via paging I/O against the image section object. They may be read/written as data (like by the linker) via any sort of I/O (read/write or memory mapped and thus paging I/O) against the data section object.
Notepad is well known for its use of memory mapping. This is causing your page faults, which translate to IRP_MJ_READ with the IRP_PAGING_IO bit set.
Is your application doing non-cached I/O? I’m going to hazard a guess here: “no, it’s using CACHED I/O”.
So the call gets to NTFS… and NTFS calls CcCopyWrite to stick the data in the cache. Since the pages aren’t currently mapped into the cache (because there’s a page fault against them) the memory manager blocks the write operation so the paging read can finish and the pages become available for use.
Hence my observation you really need to understand the locking in the system.
To be honest, I’m not sure that non-cached write operations will be enough because NTFS will try to purge (CcCoherencyFlushAndPurgeCache). It is distinctly possible that this will ALSO block and wait, but you will need to try it and see.
Assuming that is the case, you’d need to change your implementation model: you’d need to take that write operation from the application and satisfy the read operation so those pages get unblocked. Then you can (asynchronously) write the data content back to the file.
We’ve done this sort of thing before, but we have a framework we use for doing it that we call an “isolation filter” - it avoids this sort of problem by managing the cache in the filter so that the write from the application doesn’t collide against the read page fault.
Welcome to building filters: where it looks easy when you start and turns out to be more work and more difficult than building file systems.
Tony
OSR