Mini filter to redirect changes on free space on disk

Hello Experts,

My goal is to have a mini filter that undo changes made to files. The
changes are reverted when the system reboots, with the possibility of allowing
certain processes to escape the redirection and their processed data being
incorporated into the old baseline that gets updated]

After following many advice here, I came with following design principles.

  1. When processing the post create, I identify the object of interest and
    allocate a context.
  2. In Pre Read/Write get callback data and check if it is an object
    of interest then queue it using cancel safe queue.
  3. In my processingIo routine I retrieve the offset and length in order to
    obtain LBA on disk.
  4. if 3 works fine create an IRP initialized with the data and offset from the callback data then send the
    newly build IRP to the device object of the volume of interest where i query the
    bitmap structure and write to the free sectors on that volume, then complete the initial
    request.

1 and 2 seems to work but I am having trouble performing the I/O association
from above the file system to the volume level.

FSCTL_GET_RETRIEVAL_POINTERS succeed but from time to time fails with status
STATUS_END_OF_FILE(c0000011).
I made some research and found out that the problem can arise when the file is
small(< 900B) thus embedded in the MFT, and also when the Extents have not been
allocated yet.

If my design seems exact, how do I solve these problems?
Also for those who have been down this path, are there some considerations that
I might no be taking in consideration so I ca improve my design?

Thank you.

The problem of detecting files that are embedded within the MFT can be
very problematic, to say the least. You could do tricks such as
inflating the file size to have it non-resident then write out the real
content, truncating it back (I have not seen where a file will go from
non-resident to resident but this could change or I may have not
triggered it) but this can get really ugly, fast. Also, when are you
checking for the extents? They are not necessarily going to be allocated
in the pre-operation of cached IOs or in non-cached, non-paging
requests.

In general, trying to associate high level IO to volume level activity
is going to be very hard, 100% of the time. There are going to be edge
cases, as you’ve found, that are nearly impossible to capture. The case
of resident files … having them resident breaks your design from the
start since within a given MFT entry, it’s not only the data you want to
ensure gets written out but the entire MFT entry content. Which brings
up my next point in that for non-resident files, you want to ensure any
meta-data updates for that file, which are contained within the MFT
entry, are also written to disk, not just the file data. Since the file
could be extended and the file size information as well as the new
extent information, which are stored in the MFT entry, must be preserved
on disk.

Pete


Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com http:</http:>
866.263.9295

------ Original Message ------
From: xxxxx@yahoo.fr
To: “Windows File Systems Devs Interest List”
Sent: 11/29/2015 11:27:43 PM
Subject: [ntfsd] Mini filter to redirect changes on free space on disk

>Hello Experts,
>
>My goal is to have a mini filter that undo changes made to files. The
>changes are reverted when the system reboots, with the possibility of
>allowing
>certain processes to escape the redirection and their processed data
>being
>incorporated into the old baseline that gets updated]
>
>After following many advice here, I came with following design
>principles.
>
>1) When processing the post create, I identify the object of interest
>and
>allocate a context.
>2) In Pre Read/Write get callback data and check if it is an object
>of interest then queue it using cancel safe queue.
>3) In my processingIo routine I retrieve the offset and length in order
>to
>obtain LBA on disk.
>4) if 3 works fine create an IRP initialized with the data and offset
>from the callback data then send the
>newly build IRP to the device object of the volume of interest where i
>query the
>bitmap structure and write to the free sectors on that volume, then
>complete the initial
>request.
>
>1 and 2 seems to work but I am having trouble performing the I/O
>association
>from above the file system to the volume level.
>
>FSCTL_GET_RETRIEVAL_POINTERS succeed but from time to time fails with
>status
>STATUS_END_OF_FILE(c0000011).
>I made some research and found out that the problem can arise when the
>file is
>small(< 900B) thus embedded in the MFT, and also when the Extents have
>not been
>allocated yet.
>
>If my design seems exact, how do I solve these problems?
>Also for those who have been down this path, are there some
>considerations that
>I might no be taking in consideration so I ca improve my design?
>
>Thank you.
>
>—
>NTFSD is sponsored by OSR
>
>OSR is hiring!! Info at http://www.osr.com/careers
>
>For our schedule of debugging and file system seminars visit:
>http://www.osr.com/seminars
>
>To unsubscribe, visit the List Server section of OSR Online at
>http://www.osronline.com/page.cfm?name=ListServer

Thank you for replying.
I am currently checking the extents In a working thread routine when processing the queued callback data received from Pre Read/Write routine. I understand you reply and see how my design can be broken. Taking into consideration your analysis I’ll keep looking for a way to approach that problem.

“They are not necessarily going to be allocated in the pre-operation of cached IOs or in non-cached, non-paging requests”

Maybe you can also clear my confusion? while processing the pre-R/W request I do not process data with IRP_PAGING_IO flag set nor IPR_SYNCHRONOUS_PAGING_IO, as I recall I cannot queue any synchronous R/W operation. I which to avoid request targeting the pagefile doing so, but don’t really understand why the file extents wouldn’t be ready at this level.

Thank you.