Refcounting in file system minifilter driver

Anand_A · May 16, 2017, 2:14pm

Hi All,

I am developing a file system filter minifilter driver to track changes to specific files or files of specific type. I will attach a file context in PostOperationCallback() for IRP_MJ_CREATE and use this in PreOperationCallback() of IRP_MJ_WRITE and IRP_MJ_CLOSE.

If the driver has been loaded and attached after some IRP_MJ_CREATE calls have already gone through, is there a way to find the current refcount (through ObjectManager?) so that the same could be taken into consideration to determine when the file context attached by the driver needs to be cleared ?

I have read this article : http://www.osronline.com/article.cfm?id=102

It states the same issue with the proposed algo : “Do note that this algorithm only works if your filter observes all operations against this file - if you dynamically load your filter driver and the file has already been opened prior to that, this algorithm will not suffice.”

I am targeting Windows Server 2012 [R2]. I cannot rely on the IRP_MJ_CLEANUP callback because the environment where I am running (HyperV on CSV) could have a case of VM failover - failback and hence the file could be closed on host1, opened, written to and closed from host 2 and then re-opened from host1 and if all this happens in a short duration before cache manager removes the file context then the correctness of change tracking will be lost.

Thanks,
Anand.

Peter_Scott · May 16, 2017, 6:40pm

There is no documented, or consistent and reliable undocumented way to
do this. The only components that would know, across all open instances
of a file, what the ref count is on that file would be the file system
or an fs filter that has processed all open requests.

Why can’t you load your filter at boot time, if this is a requirement?

Pete

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

------ Original Message ------
From: xxxxx@gmail.com
To: “Windows File Systems Devs Interest List”
Sent: 5/16/2017 12:14:35 PM
Subject: [ntfsd] Refcounting in file system minifilter driver

>Hi All,
>
>I am developing a file system filter minifilter driver to track changes
>to specific files or files of specific type. I will attach a file
>context in PostOperationCallback() for IRP_MJ_CREATE and use this in
>PreOperationCallback() of IRP_MJ_WRITE and IRP_MJ_CLOSE.
>
>If the driver has been loaded and attached after some IRP_MJ_CREATE
>calls have already gone through, is there a way to find the current
>refcount (through ObjectManager?) so that the same could be taken into
>consideration to determine when the file context attached by the driver
>needs to be cleared ?
>
>I have read this article : http://www.osronline.com/article.cfm?id=102
>
>It states the same issue with the proposed algo : “Do note that this
>algorithm only works if your filter observes all operations against
>this file - if you dynamically load your filter driver and the file has
>already been opened prior to that, this algorithm will not suffice.”
>
>I am targeting Windows Server 2012 [R2]. I cannot rely on the
>IRP_MJ_CLEANUP callback because the environment where I am running
>(HyperV on CSV) could have a case of VM failover - failback and hence
>the file could be closed on host1, opened, written to and closed from
>host 2 and then re-opened from host1 and if all this happens in a short
>duration before cache manager removes the file context then the
>correctness of change tracking will be lost.
>
>Thanks,
>Anand.
>
>—
>NTFSD is sponsored by OSR
>
>
>MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>software drivers!
>Details at http:
>
>To unsubscribe, visit the List Server section of OSR Online at
>http:</http:></http:>

Anand_A · May 17, 2017, 12:16am

I am trying to solve the problem for already running VM scenario when the driver is installed for the first time for which incremental change tracking is desirable after the first full backup. Subsequent to reboot of host or power off/on of VM, it will work fine.

Scott_Noone_OSR · May 17, 2017, 9:47am

All this reference counting nonsense only related to legacy filters. These
days Filter Manager deals with all of that, you just need to worry about
using the FltXxx APIs properly. See the Context sample:

https://github.com/Microsoft/Windows-driver-samples/tree/master/filesys/miniFilter/ctx

-scott
OSR
@OSRDrivers

wrote in message news:xxxxx@ntfsd…

Hi All,

I am developing a file system filter minifilter driver to track changes to
specific files or files of specific type. I will attach a file context in
PostOperationCallback() for IRP_MJ_CREATE and use this in
PreOperationCallback() of IRP_MJ_WRITE and IRP_MJ_CLOSE.

If the driver has been loaded and attached after some IRP_MJ_CREATE calls
have already gone through, is there a way to find the current refcount
(through ObjectManager?) so that the same could be taken into consideration
to determine when the file context attached by the driver needs to be
cleared ?

I have read this article : http://www.osronline.com/article.cfm?id=102

It states the same issue with the proposed algo : “Do note that this
algorithm only works if your filter observes all operations against this
file - if you dynamically load your filter driver and the file has already
been opened prior to that, this algorithm will not suffice.”

I am targeting Windows Server 2012 [R2]. I cannot rely on the IRP_MJ_CLEANUP
callback because the environment where I am running (HyperV on CSV) could
have a case of VM failover - failback and hence the file could be closed on
host1, opened, written to and closed from host 2 and then re-opened from
host1 and if all this happens in a short duration before cache manager
removes the file context then the correctness of change tracking will be
lost.

Thanks,
Anand.

Anand_A · May 18, 2017, 2:07am

Thanks Scott.

I am using per file context (FltSetFileContext()). So, IIUC from the sample and reading (https://docs.microsoft.com/en-us/windows-hardware/drivers/ifs/releasing-contexts), the FltReleaseContext() call corresponding to a successful call of FltSetFileContext() need not be made (by the minifilter driver).

Now, I still have a few questions :

How can my driver get to know when this reference is released? I understand that this could be well after the last handle to the file is closed. Is there a callback I can register for ?
In the CSV environment (with NTFS) is it so that the context gets released as soon as last handle to the file is closed ? That seems to be the case based on my experimentation (on CSV owner node, experimenting further) but I do not want to make any assumptions unless there is some documentation about it or others have made similar observations. I have not yet set any block cache for CSV, will check with that too but wondering if that should have any impact

Anand_A · May 18, 2017, 3:11am

Sorry, please ignore question #1 in the above post. The answer to that is known : https://msdn.microsoft.com/library/windows/hardware/ff551078
I was thinking that there will be a way to specify the cleanup callback at the time of setting the context.

Scott_Noone_OSR · May 18, 2017, 3:01pm

The file context only goes away when the last reference (not HANDLE) to the
last file object goes away. This may effectively never happen due to the
wonders of caching.

-scott
OSR
@OSRDrivers

wrote in message news:xxxxx@ntfsd…

Sorry, please ignore question #1 in the above post. The answer to that is
known : https://msdn.microsoft.com/library/windows/hardware/ff551078
I was thinking that there will be a way to specify the cleanup callback at
the time of setting the context.