wiping memory mapped files on close

NTFSD
1

Hello

I have minifilter that wipes files set for delete during cleanup. While it
works ok some scenarios are not supported.

For example:

  1. create file with delete_on_close flag.
  2. create section.
  3. map view.
  4. CloseHandle <- got MJ_CLEANUP but file still in use.

To support mm files i reworked my driver to operate like this:

  1. pre-cleanup - query location of file clusters
  2. last pre-close for stream :
    insert request in queue.
    return FLT_PREOP_PENDING.
  3. my worker thread:
    pops request from queue
    synchronously write to clusters (IoBuildSynchronousFsdRequest +
    IoCallDriver)
    resumes close dispatching with
    FltCompletePendedPreOperation(FLT_PREOP_SUCCESS_NO_CALLBACK)

It works fine most of the times but in some cases i have problems.
Sometimes errors for vcredist installations, sometimes i am unable to open
Start menu.
For now i did some testing on windows 10 x64 only.

Before i start with lot of tracing and debugging i would like to ask if
anyone sees some obvious fundamental flaw in my design and task is just not
feasible.

2

One thing you should check is that you would get writes after you have
cleared the extents by directly writing to them in your worker thread.
How are you ensuring that MM or CM doesn’t still have a reference on the
file and therefore the CLOSE you are doing your work from, in pre-close,
is the LAST close for the file object? If it isn’t, you could clear some
of the file then a flush would write out a different portion, resulting
in a corrupt file.

Pete

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

------ Original Message ------
From: “Sergey Pisarev”
To: “Windows File Systems Devs Interest List”
Sent: 2/10/2016 9:54:18 AM
Subject: [ntfsd] wiping memory mapped files on close

>Hello
>
>I have minifilter that wipes files set for delete during cleanup. While
>it works ok some scenarios are not supported.
>
>For example:
>1. create file with delete_on_close flag.
>2. create section.
>3. map view.
>4. CloseHandle <- got MJ_CLEANUP but file still in use.
>
>To support mm files i reworked my driver to operate like this:
>1. pre-cleanup - query location of file clusters
>2. last pre-close for stream :
> insert request in queue.
> return FLT_PREOP_PENDING.
>3. my worker thread:
> pops request from queue
> synchronously write to clusters (IoBuildSynchronousFsdRequest +
>IoCallDriver)
> resumes close dispatching with
>FltCompletePendedPreOperation(FLT_PREOP_SUCCESS_NO_CALLBACK)
>
>It works fine most of the times but in some cases i have problems.
>Sometimes errors for vcredist installations, sometimes i am unable to
>open Start menu.
>For now i did some testing on windows 10 x64 only.
>
>Before i start with lot of tracing and debugging i would like to ask if
>anyone sees some obvious fundamental flaw in my design and task is just
>not feasible.
>— NTFSD is sponsored by OSR MONTHLY seminars on crash dump analysis,
>WDF, Windows internals and software drivers! Details at To unsubscribe,
>visit the List Server section of OSR Online at

3

Thank you Peter!

will check this first thing tomorrow.

i have my own refcount in stream context.
increment it after each successful create (except reparse),
decrement at each close without FO_STREAM_FILE flag set.

after what you said i think that it is actually quite possible that there
are still stream FILE_OBJECTs even when my refcount reaches zero. i don’t
know why i thought that normal(non-stream) FILE_OBJECT is always closed
last.

Thank you again Peter! Invaluable advice as always.

On Wed, Feb 10, 2016 at 8:29 PM, PScott wrote:

> One thing you should check is that you would get writes after you have
> cleared the extents by directly writing to them in your worker thread. How
> are you ensuring that MM or CM doesn’t still have a reference on the file
> and therefore the CLOSE you are doing your work from, in pre-close, is the
> LAST close for the file object? If it isn’t, you could clear some of the
> file then a flush would write out a different portion, resulting in a
> corrupt file.
>
> Pete
>
> –
> Kernel Drivers
> Windows File System and Device Driver Consulting
> www.KernelDrivers.com http:</http:>
> 866.263.9295
>
>
>
> ------ Original Message ------
> From: “Sergey Pisarev”
> To: “Windows File Systems Devs Interest List”
> Sent: 2/10/2016 9:54:18 AM
> Subject: [ntfsd] wiping memory mapped files on close
>
>
> Hello
>
> I have minifilter that wipes files set for delete during cleanup. While it
> works ok some scenarios are not supported.
>
> For example:
> 1. create file with delete_on_close flag.
> 2. create section.
> 3. map view.
> 4. CloseHandle <- got MJ_CLEANUP but file still in use.
>
> To support mm files i reworked my driver to operate like this:
> 1. pre-cleanup - query location of file clusters
> 2. last pre-close for stream :
> insert request in queue.
> return FLT_PREOP_PENDING.
> 3. my worker thread:
> pops request from queue
> synchronously write to clusters (IoBuildSynchronousFsdRequest +
> IoCallDriver)
> resumes close dispatching with
> FltCompletePendedPreOperation(FLT_PREOP_SUCCESS_NO_CALLBACK)
>
> It works fine most of the times but in some cases i have problems.
> Sometimes errors for vcredist installations, sometimes i am unable to open
> Start menu.
> For now i did some testing on windows 10 x64 only.
>
> Before i start with lot of tracing and debugging i would like to ask if
> anyone sees some obvious fundamental flaw in my design and task is just not
> feasible.
> — NTFSD is sponsored by OSR MONTHLY seminars on crash dump analysis,
> WDF, Windows internals and software drivers! Details at To unsubscribe,
> visit the List Server section of OSR Online at
>
>
> —
> NTFSD is sponsored by OSR
>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

4

>even when my refcount reaches zero. i don’t know why i thought that normal(non-stream)

FILE_OBJECT is always closed last.

I remember that in some Windows versions NTFS was using stream file objects for Cc/Mm use.

Then this one will be the last closed one.

–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

5

>1. pre-cleanup - query location of file clusters

Disaster waiting to happen.

For delete-on-close case, cleanup will truncate the file to zero.

After this, the clusters are free for reuse by other files.

So, when your work item starts, then it can overwrite other files.

You should only wipe within the file’s allocated size while it is still allocated, before the file is truncated. Also note that you must wipe not only on delete, but on usual truncates too.

So you cannot use work items, you can only do this synchronously. Otherwise, the truncate path (called from cleanup-on-delete-on-close or from SetInformationFile) will continue after queuing your work item, and continuation means - freeing the disk blocks for reuse.

My suggestion:

  1. review all FSD paths, which of them can reduce the file size, thus freeing the disk space. Note that this includes FSCTL_SET_SPARSE :slight_smile:
  2. look at FASTFAT on what Mm/Cc interaction is there in these paths. At least MmCanFileBeTruncated.
  3. in Pre-these paths, do your own Mm/Cc interaction (MmCanFileBeTruncated), and then wipe if Mm/Cc do allow the truncation. Do the noncached write - if there is a cache on the FCB, then the FSB will execute it as cached write+immediate flush anyway.

–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

6

Thank you Max !

That is a lots of things to think about and sources to check.
Thank you for very detailed advice.

On Thu, Feb 11, 2016 at 8:51 AM, Maxim S. Shatskih
wrote:

> >1. pre-cleanup - query location of file clusters
>
> Disaster waiting to happen.
>
> For delete-on-close case, cleanup will truncate the file to zero.
>
> After this, the clusters are free for reuse by other files.
>
> So, when your work item starts, then it can overwrite other files.
>
> You should only wipe within the file’s allocated size while it is still
> allocated, before the file is truncated. Also note that you must wipe not
> only on delete, but on usual truncates too.
>
> So you cannot use work items, you can only do this synchronously.
> Otherwise, the truncate path (called from cleanup-on-delete-on-close or
> from SetInformationFile) will continue after queuing your work item, and
> continuation means - freeing the disk blocks for reuse.
>
> My suggestion:
>
> 1) review all FSD paths, which of them can reduce the file size, thus
> freeing the disk space. Note that this includes FSCTL_SET_SPARSE :slight_smile:
> 2) look at FASTFAT on what Mm/Cc interaction is there in these paths. At
> least MmCanFileBeTruncated.
> 3) in Pre-these paths, do your own Mm/Cc interaction
> (MmCanFileBeTruncated), and then wipe if Mm/Cc do allow the truncation. Do
> the noncached write - if there is a cache on the FCB, then the FSB will
> execute it as cached write+immediate flush anyway.
>
> –
> Maxim S. Shatskih
> Microsoft MVP on File System And Storage
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> —
> NTFSD is sponsored by OSR
>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>