ccFlushCache faults on a memory mapped file

NTFSD
1

I am trying to flush memory mapped file in my FSD. I see page faults when ccFlushCache is executed on the file. I have made sure that all the IRPs are with IRP_NOCACHE. Also, CreateFileMapping is using SEC_NOCACHE flag. I unable imagine the scenario why the call would fault even if PDE/PTE are invalid (i.e. data is already on the Disk).

Thanks.

2

You are saying that you see page read requests for the same file when you are flushing it? For what region of the file do you observe said page faults? Is there anything special about this file, for example, is it compressed?

While it does seem unusual, it certainly isn’t incorrect.

Tony
OSR

3

And regardless of everything else, you definitely don’t want to use SEC_NOCACHE. This flag disables processor caching by setting a bit in the PTE when the file is mapped. This has nothing to do with the OS file cache, and will just make memory mapped accesses really slow.

From: Tony Masonmailto:xxxxx
Sent: ?1/?24/?2015 4:18 PM
To: Windows File Systems Devs Interest Listmailto:xxxxx
Subject: RE: [ntfsd] ccFlushCache faults on a memory mapped file



You are saying that you see page read requests for the same file when you are flushing it? For what region of the file do you observe said page faults? Is there anything special about this file, for example, is it compressed?

While it does seem unusual, it certainly isn’t incorrect.

Tony
OSR

—
NTFSD is sponsored by OSR

OSR is hiring!! Info at http://www.osr.com/careers

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

4

Thanks Pavel and Tony.

I figured that the faulting I/O requests were write requests. As these requests asserted under mmIsRecursiveIOfault, I got confused. The request flow is FSD (ccflush) -> CM -> FSD which is recursive.

5

Tony, I read your post somewhere mentioning that the memory mapped files are written back by only Mapped Page Writer thread when ccFlushCache is executed to which I agree. But, I see another thread, the one which called ccFlushCache, doing write backs and asserting under mmIsRecursiveIOfault. Can you explain this behavior?

6

Flushes initiated by the Memory Manager are done by the Mapped Page Writer (versus the Modified Page Writer). But that’s not to say that flushes initiated elsewhere will do the same thing. The reason the Memory Manager does this is to ensure that it doesn’t deadlock (originally, Mm only had the Modified Page Writer thread. They introduced the Mapped Page Writer to offload the Modified Page Writer and resolve some deadlock situation).

So a thread that calls CcFlushCache (or MmFlushVirtualMemory) would do the write back operations in thread context, not in the context of the Modified Page Writer.

I’m not sure what you mean by “asserting under MmIsRecursiveIoFault”, because as far as I know, there is no assert in this function - it just looks to see if the Cache Manager has indicated it is manipulating the file or page fault clustering has been disabled.

kd> uf nt!MmIsRecursiveIoFault
nt!MmIsRecursiveIoFault:
fffff803947f9714 65488b0c2588010000 mov rcx,qword ptr gs:[188h] ;; get current thread pointer fffff803947f971d 0fb681c1060000 movzx eax,byte ptr [rcx+6C1h] ;; Disable Page Fault Clustering
fffff803947f9724 0fb689c0060000 movzx ecx,byte ptr [rcx+6C0h] ;; Cache Manager Active fffff803947f972b 0bc1 or eax,ecx ;; or the two fields as the return value (EAX)
fffff803`947f972d c3 ret

kd> dt nt!_ETHREAD
+0x6c0 CacheManagerActive : UChar
+0x6c1 DisablePageFaultClustering : UChar

If you’re calling CcFlushCache, this condition will definitely be true for any paging I/O that you see as a result.

Tony
OSR

7

So a thread that calls CcFlushCache (or MmFlushVirtualMemory) would do the write
back operations in thread context, not in the context of the Modified Page
Writer

> But, If I call ccFlushCache on a memory mapped file with an exclusive acquire on the FCB -> Resource and also try to acquire the same FCB->Resource exclusively for the modified page writer (Locks are acquired using callbacks) then deadlock happens. Doesn’t this mean that ccFlushCache is invoking mapped page writer and waiting for it?

If you’re calling CcFlushCache, this condition will definitely be true for any
paging I/O that you see as a result.

> I figured that. Thanks Tony.