Driver Crash at SendNetBufferListsComplete

I have filter driver who does initiate the packets and then deallocate the memory of them in SendNetBufferListsComplete, but I see driver crashes at random. with the help of windbg I see that it crashes while deallocating memory, some times for buffer, some times for MDL deallocation,

Here is the code I have in SendNetbufferlistcomplete to handle the deallocation.

currNB = NET_BUFFER_LIST_FIRST_NB(CurrNbl);
while (currNB != NULL)
{
currentMBL = NET_BUFFER_CURRENT_MDL(currNB);
while (currentMBL != NULL)
{
NdisQueryMdl(currentMBL,
&bufferAddress,
&bufferLength,
HighPagePriority);

if (bufferAddress != NULL)
{
NdisFreeMemory(bufferAddress, 0, 0);
}
NdisGetNextMdl(currentMBL,
&nextMBL);
NdisFreeMdl(currentMBL);
currentMBL = nextMBL;
}
nextNB = NET_BUFFER_NEXT_NB(currNB);
NdisFreeNetBuffer(currNB);
currNB = nextNB;
}
next_Nbl = NET_BUFFER_LIST_NEXT_NBL(CurrNbl);
NdisFreeNetBufferList(CurrNbl);
CurrNbl = next_Nbl;

Here is the Crash data
POOL_ADDRESS: GetPointerFromAddress: unable to read from 82dc084c
Unable to read MiSystemVaType memory at 82d9f780
8813c5d0

BUGCHECK_STR: 0xc2_7

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: WIN7_DRIVER_FAULT

CURRENT_IRQL: 0

ANALYSIS_VERSION: 6.3.9600.17336 (debuggers(dbg).150226-1500) amd64fre

LAST_CONTROL_TRANSFER: from 89029135 to 82d76c6b
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

BAD_POOL_CALLER (c2)
The current thread is making a bad pool request. Typically this is at a bad IRQL level or double freeing the same allocation, etc.
Arguments:
Arg1: 00000007, Attempt to free pool which was already freed
Arg2: 0000109b, (reserved)
Arg3: 08110012, Memory contents of the pool block
Arg4: 8813c5d0, Address of the block of pool being deallocated

Debugging Details:

GetPointerFromAddress: unable to read from 82dc084c
Unable to read MiSystemVaType memory at 82d9f780

STACK_TEXT:
8af56e08 89029135 8813c5d0 00000000 8af56e70 nt!ExFreePoolWithTag+0x1b1
8af56e18 b4002cf2 8813c5d0 00000000 00000000 ndis!NdisFreeMemory+0x16
8af56e70 89084f0a 879400b0 88017808 00000001 Afdx_Filter!FilterSendNetBufferListsComplete+0x1c2 [d:\subversion\ims\drivers\afdx_lwf\afdxfilter.c @ 1504]
8af56e94 8f5c32cb 86c040e0 88017808 00000001 ndis!NdisMSendNetBufferListsComplete+0xa4
WARNING: Stack unwind information not available. Following frames may be wrong.
8af56ea8 8f5c36cb 00000001 10000000 873f9044 ksz8841_pci+0x42cb
8af56ecc 8f5c22a8 00000044 00000000 86c040e0 ksz8841_pci+0x46cb
8af56ee4 8907289a 00000040 00000000 8af56f10 ksz8841_pci+0x32a8
8af56f20 8901da1f 87495b74 00495b60 00000000 ndis!ndisMiniportDpc+0xe2
8af56f48 82ccc015 87495b74 87495b60 00000000 ndis!ndisInterruptDpc+0xaf
8af56fa4 82ccbe78 8af36120 8864c030 00000000 nt!KiExecuteAllDpcs+0xf9
8af56ff4 82ccb63c b1ce7ce4 00000000 00000000 nt!KiRetireDpcList+0xd5
8af56ff8 b1ce7ce4 00000000 00000000 00000000 nt!KiDispatchInterrupt+0x2c
82ccb63c 00000000 0000001a 00d6850f bb830000 0xb1ce7ce4

I have gone through the code and could not get where it was already been deallocated.

Could it be that the NdisQueryMdl macro may not alter the bufferAddress parameter ?

If that could happen and because you do not set this parameter to NULL before calling NdisQueryMdl, you may use the value that was set in the preceding sequence of the while loop.

// Clear the values that may have been set in the previous loop sequence
bufferAddress = NULL;
bufferLength = 0;

NdisQueryMdl(
currentMBL,
&bufferAddress,
&bufferLength,
HighPagePriority);

if (bufferAddress != NULL) {
NdisFreeMemory(bufferAddress, 0, 0);
}

Also when you walk a buffer list or MDL chain, grab the next item as soon as you enter the while block:

while (currNB != NULL) {
nextNB = NET_BUFFER_NEXT_NB(currNB);
.
.
.
currNB = nextNB;
}

I would look for buffer overwrite or underwrite. Did you try with DriverVerifier and Special Pools?

This should not be the case here as MDL must have been initialized with NdisAllocateMdl so the buffer was from NonPageedPool and MDL has been set by MmBuildMdlForNonPagedPool inside NdisAllocateMdl . Right?

Anyway, for the future - you should unlock pages before freeing memory.
That is

MmUnlockPages( Mdl );
ExFreePool( MmGetMdlVirtualAddress(Mdl) );
IoFreeMdl( Mdl );

The reason is ExFreePool() might try to release page to the free list if the last allocation has gone( i.e. return a page back to the Memory Manager ). If the lock count of a page which is being moved to the free list shows that the page is locked this is a critical bug condition and KeBugCheck is called.

The NdisQueryMdl macro sets the value of bufferAddress to the value returned by MmGetSystemAddressForMdlSafe.

*(PVOID *)(_VirtualAddress) = MmGetSystemAddressForMdlSafe(_Mdl, _Priority);

So you do not need to set it to NULL before invoking the macro.


If the filter driver originated the send request, FilterSendNetBufferListsComplete can either release the NET_BUFFER_LIST structures and associated data or prepare them for reuse in a subsequent call to NdisFSendNetBufferLists.

Note : A filter driver should keep track of send requests that it initiates and make sure that it does not call NdisFSendNetBufferListsComplete when NDIS calls FilterSendNetBufferListsComplete for such requests.

Are you sure you are releasing resources your filter driver owns ?

How do you keep track of NET_BUFFER_LISTs for send requests your filter driver initiated ? I mean, are you using a list of sent requests or something like that ?

Yes, I keep track of the net buffer list, that my filter initiated and then once we get to Complete handler, I compare the NBL in sendcomplete handler to the one I sent, then only I am deallocating memory for the the NBL.

I have modified the code as below.

// Control comes here means we found the right entry of CurrNBL in the SentQueue
// Get the Buffer and NBL and deallocate the memory of them.
currNB = NET_BUFFER_LIST_FIRST_NB(CurrNbl);
while (currNB != NULL)
{
nextNB = NET_BUFFER_NEXT_NB(currNB);
currentMBL = NET_BUFFER_CURRENT_MDL(currNB);
while (currentMBL != NULL)
{
NdisGetNextMdl(currentMBL,
&nextMBL);

// Clear the previous bufferAddress and bufferLength values
bufferAddress = NULL;
bufferLength = 0;

NdisQueryMdl(currentMBL,
&bufferAddress,
&bufferLength,
HighPagePriority);

if (bufferAddress != NULL && bufferLength > 0)
{
NdisFreeMemory(bufferAddress, 0, 0);
}
NdisFreeMdl(currentMBL);
currentMBL = nextMBL;
}
NdisFreeNetBuffer(currNB);
currNB = nextNB;
}
next_Nbl = NET_BUFFER_LIST_NEXT_NBL(CurrNbl);
NdisFreeNetBufferList(CurrNbl);
CurrNbl = next_Nbl;