I am using a spinlock to serialize operations in both threads running at <= DISPATCH_LEVEL and a IMSR running at the IRQL lever of 7 which could cause Priority-Inversion/deadlock.
One easy way to solve the problem is to lower the IRQL of my IMSR to DISPATCH_LEVEL so that the IMSR will not be able to get processor time when other thread is holding the spinlock at DISPATCH_LEVEL. Is there anyway to lower the IRQL of an IMSR?
Thanks,
Ying
No, there is no way to do this, if you do you cause priority problems.
Instead use the interrupt spinlock in the non-ISR routines to control
access.
Don Burn
Windows Driver Consulting
Website: http://www.windrvr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.ca
Sent: Thursday, September 03, 2015 12:08 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Can I lower the IRQL of an IMSR
I am using a spinlock to serialize operations in both threads running at <=
DISPATCH_LEVEL and a IMSR running at the IRQL lever of 7 which could cause
Priority-Inversion/deadlock.
One easy way to solve the problem is to lower the IRQL of my IMSR to
DISPATCH_LEVEL so that the IMSR will not be able to get processor time when
other thread is holding the spinlock at DISPATCH_LEVEL. Is there anyway to
lower the IRQL of an IMSR?
Thanks,
Ying
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
No, absolutely impossible.
If you need a spinlock in the ISR - then it MUST be the interrupt spinlock.
–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com
wrote in message news:xxxxx@ntdev…
>I am using a spinlock to serialize operations in both threads running at <= DISPATCH_LEVEL and a IMSR running at the IRQL lever of 7 which could cause Priority-Inversion/deadlock.
>
> One easy way to solve the problem is to lower the IRQL of my IMSR to DISPATCH_LEVEL so that the IMSR will not be able to get processor time when other thread is holding the spinlock at DISPATCH_LEVEL. Is there anyway to lower the IRQL of an IMSR?
>
> Thanks,
>
> Ying
>
Hi Don and Maxim,
Thanks for your quick response. What is Interrupt Spinlock? is it the myIMRLock specified in the following code?
IO_CONNECT_INTERRUPT_PARAMETERS params;
…
params.MessageBased.SpinLock = &myIMSRLock;
…
status = IoConnectInterruptEx(¶ms);
Please confirm.
Ying
My efforts to use myIMRlock in the none-ISR routine has just failed with the following error:
IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 0000000000000001, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000001, bitfield :
bit 0 : value 0 = read operation, 1 = write operation
bit 3 : value 0 = not an execute operation, 1 = execute operation (only on chips which support this level of status)
Arg4: fffff803176c0e52, address which referenced memory
WORKER_ROUTINE:
+295d380
00000000`00000001 ?? ???
WORK_ITEM: fffff803176c0e52
CURRENT_IRQL: 2
DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT
BUGCHECK_STR: AV
PROCESS_NAME: send
ANALYSIS_VERSION: 6.3.9600.17237 (debuggers(dbg).140716-0327) amd64fre
TRAP_FRAME: fffff8800a076460 – (.trap 0xfffff8800a076460)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000000 rbx=0000000000000000 rcx=fffff8800a076690
rdx=0000000000000001 rsi=0000000000000000 rdi=0000000000000000
rip=fffff803176c0e52 rsp=fffff8800a0765f0 rbp=fffff8800a076cc0
r8=fffff8800a076690 r9=0000000000000000 r10=fffff880009c7a80
r11=fffffa800b97f1e0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na po nc
nt!KxWaitForLockOwnerShip+0x12:
fffff803176c0e52 48890a mov qword ptr [rdx],rcx ds:0000000000000001=???
Resetting default scope
LAST_CONTROL_TRANSFER: from fffff8031779aaba to fffff80317671cb0
STACK_TEXT:
fffff8800a075b58 fffff8031779aaba : fffff6fc018bed48 fffff6fb7e00c5f0 fffff6fb7dbf0060 fffff6fb7dbedf80 : nt!DbgBreakPointWithStatus
fffff8800a075b60 fffff8031779a44a : fffff80300000004 fffff803178f0ea0 0000000000000000 0000000000000001 : nt!KiBugCheckDebugBreak+0x12
fffff8800a075bc0 fffff80317677544 : 0000000000000000 fffff8800a076400 0000000000000000 fffff80317671086 : nt!KeBugCheck2+0xadb
fffff8800a0762e0 fffff80317676769 : 000000000000000a 0000000000000001 0000000000000002 0000000000000001 : nt!KeBugCheckEx+0x104
fffff8800a076320 fffff80317674fe0 : 0000000000000001 fffffa800b3b2750 fffffa800b773200 fffff8800a076460 : nt!KiBugCheckDispatch+0x69
fffff8800a076460 fffff803176c0e52 : 0000000000000000 fffffa8009e9df20 fffffa8009e9df20 fffffa800b97f298 : nt!KiPageFault+0x260
fffff8800a0765f0 fffff88009a11b91 : fffffa800b3b2750 fffff88009a1159c fffffa800b97f1e0 0000000000000000 : nt!KxWaitForLockOwnerShip+0x12
fffff8800a076620 fffff88009a085ce : fffffa800c74c108 fffff8800a076690 0000000000000000 fffff8800a076688 : mydrv!os_spin_lock_irqsave+0x21 [c:\sboxes\sv\542dhcert\drivers\sv\driver\os_win.c @ 779]
fffff8800a076650 fffff88009a07bc0 : fffffa800c74c128 fffffa800b97f1e0 fffff88000000001 fffffa8009ed8080 : mydrv!sv_decrement_active_count+0x2e [c:\sboxes\sv\542dhcert\drivers\sv\driver\main.c @ 927]
fffff8800a0766c0 fffff88009a08d14 : fffffa800c74c000 fffff8800a0768c8 fffffa8000000000 fffffa800bc513c0 : mydrv!sv_attempt_dma_ioctl+0x420 [c:\sboxes\sv\542dhcert\drivers\sv\driver\main.c @ 1228]
fffff8800a076780 fffff88009a0909a : fffffa800c74c000 fffff8800a0768c8 0000000000000020 fffffa800b3b2750 : mydrv!sv_do_dma_ioctl+0x24 [c:\sboxes\sv\542dhcert\drivers\sv\driver\main.c @ 1241]
fffff8800a0767c0 fffff88009a0f293 : fffffa800c74c000 000000009c410044 000000000040f788 0000000020206f49 : mydrv!sv_do_ioctl+0xca [c:\sboxes\sv\542dhcert\drivers\sv\driver\main.c @ 1957]
fffff8800a076900 fffff88009a0dec7 : fffffa800c74c000 000000009c410044 000000000040f788 0000000000000001 : mydrv!sv_do_ioctl_win+0x43 [c:\sboxes\sv\542dhcert\drivers\sv\driver\os_win.c @ 3073]
fffff8800a076940 fffff80317a51457 : fffffa800b243060 fffffa800b3b2750 fffffa800b3b28b0 fffff88000000000 : mydrv!sv_dispatch+0x3a7 [c:\sboxes\sv\542dhcert\drivers\sv\driver\os_win.c @ 3144]
fffff8800a0769d0 fffff80317a66646 : fffff8a003165120 000000000003fffa 0000000000000000 0000000000000000 : nt!IopXxxControlFile+0x7e5
fffff8800a076b60 fffff80317676453 : 0000000000000000 0000000000000000 fffff6fb7dbed000 fffff6fb7da00008 : nt!NtDeviceIoControlFile+0x56
fffff8800a076bd0 00000000774f2ad2 : 00000000774f2717 000000237758da4c 0000000000000023 000000007f034000 : nt!KiSystemServiceCopyEnd+0x13
000000000030eb28 00000000774f2717 : 000000237758da4c 0000000000000023 000000007f034000 000000000040f88c : wow64cpu!CpupSyscallStub+0x2
000000000030eb30 00000000774ac4f6 : 00343fff00000001 00000000774f182c 0000000000010000 000007fd59b6dca6 : wow64cpu!DeviceIoctlFileFault+0x31
000000000030ebe0 00000000774ab8f5 : 000000000030f560 000000000030f560 00000000774a3e08 000000007ffe0030 : wow64!RunCpuSimulation+0xa
000000000030ec30 000007fd59b9a1c9 : 0000000037fdef28 0000000000000000 0000000000000009 000000007f035000 : wow64!Wow64LdrpInitialize+0x435
000000000030f170 000007fd59be99be : 000000042e233c52 000007fd59c7c020 0000000000000000 0000000000000001 : ntdll!LdrpInitializeProcess+0x1576
000000000030f480 000007fd59b70066 : 000000000030f560 0000000000000000 000000007f035000 0000000000000000 : ntdll!LdrpInitialize+0xa0
000000000030f510 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : ntdll!LdrInitializeThunk+0xe
Any comments why?
Another approach is to disable the ISR before acquiring the spinlock. Is that possible to do this in Windows?
Thanks,
Ying
In the document title as “Interrupt Architecture Enhancements in Microsoft Windows Vista”, there is a section called Interrupt Prioritization. In that section, it is suggested that
\Interrupt Management
\Affinity Policy
DevicePriority: REG_DWORD: InterruptPriorityPolicyValue
in the registry can be used to specify the level of urgency of its interrupt relative to those of other device in the system. Can I use it to adjust the IRQL of my IMSR?
Thanks,
Ying
Yes it is that lock, use KeAcquireInterruptSpinlock to acquire it.
Don Burn
Windows Driver Consulting
Website: http://www.windrvr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.ca
Sent: Thursday, September 03, 2015 1:35 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Can I lower the IRQL of an IMSR
Hi Don and Maxim,
Thanks for your quick response. What is Interrupt Spinlock? is it the
myIMRLock specified in the following code?
IO_CONNECT_INTERRUPT_PARAMETERS params;
…
params.MessageBased.SpinLock = &myIMSRLock;
…
status = IoConnectInterruptEx(¶ms);
Please confirm.
Ying
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
xxxxx@yahoo.ca wrote:
My efforts to use myIMRlock in the none-ISR routine has just failed with the following error:
…
Arguments:
Arg1: 0000000000000001, memory referenced
Null pointer. Is it possible for your ioctl handler to run before you
have initialized the spin lock? How did you attempt to acquire the
lock? I don’t see the call in the stack trace.
Another approach is to disable the ISR before acquiring the spinlock. Is that possible to do this in Windows?
That is exactly what “acquiring the interrupt spin lock” will do,
assuming you do it the right way.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Hi Tim,
Thanks pointing out.
I know why I have the error as I am using KeAcquireInStackQueuedSpinLock() to acquire the spinlock. I will try to use KeAcquireInterruptSpinLock() as what Don suggested.
What is the input argument “Interrupt” for KeAcquireInterruptSpinLock()? Is it comes from InterruptObject field of the IO_INTERRUPT_MESSAGE_INFO_ENTRY data structure or ConnectionContext of the data structure MessageBased after IoConnectInterruptEx() is successfully called?
typedef struct _IO_CONNECT_INTERRUPT_PARAMETERS {
ULONG Version;
union {
…
} FullySpecified;
struct {
…
} LineBased;
struct {
PDEVICE_OBJECT PhysicalDeviceObject;
union {
PVOID *Generic;
PIO_INTERRUPT_MESSAGE_INFO *InterruptMessageTable;
PKINTERRUPT *InterruptObject;
} ConnectionContext;
PKMESSAGE_SERVICE_ROUTINE MessageServiceRoutine;
PVOID ServiceContext;
PKSPIN_LOCK SpinLock;
KIRQL SynchronizeIrql;
BOOLEAN FloatingSave;
PKSERVICE_ROUTINE FallBackServiceRoutine;
} MessageBased;
};
} IO_CONNECT_INTERRUPT_PARAMETERS, *PIO_CONNECT_INTERRUPT_PARAMETERS;
typedef struct _IO_INTERRUPT_MESSAGE_INFO {
KIRQL UnifiedIrql;
ULONG MessageCount;
IO_INTERRUPT_MESSAGE_INFO_ENTRY MessageInfo[1];
} IO_INTERRUPT_MESSAGE_INFO, *PIO_INTERRUPT_MESSAGE_INFO;
typedef struct _IO_INTERRUPT_MESSAGE_INFO_ENTRY {
PHYSICAL_ADDRESS MessageAddress;
KAFFINITY TargetProcessorSet;
PKINTERRUPT InterruptObject;
ULONG MessageData;
ULONG Vector;
KIRQL Irql;
KINTERRUPT_MODE Mode;
KINTERRUPT_POLARITY Polarity;
} IO_INTERRUPT_MESSAGE_INFO_ENTRY, *PIO_INTERRUPT_MESSAGE_INFO_ENTRY;
Please confirm,
Thanks,
Ying
I have used KeAcquireInterruptSpinLock(), I can run two threads without any problem. However after I increase to 4 threads, I get the following error:
Unknown bugcheck code (0)
Unknown bugcheck description
Arguments:
Arg1: 0000000000000000
Arg2: 0000000000000000
Arg3: 0000000000000000
Arg4: 0000000000000000
PROCESS_NAME: System
DPC_TIMEOUT_TYPE: SINGLE_DPC_TIMEOUT_EXCEEDED
DPC_RUNTIME: 504
DPC_TIME_LIMIT: 503
FAULTING_IP:
nt!KeAccumulateTicks+57e
fffff800`59d19f4e cd2c int 2Ch
ERROR_CODE: (NTSTATUS) 0xc0000420 - An assertion failure has occurred.
EXCEPTION_CODE: (NTSTATUS) 0xc0000420 - An assertion failure has occurred.
DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT
BUGCHECK_STR: 0x133
CURRENT_IRQL: d
ANALYSIS_VERSION: 6.3.9600.17237 (debuggers(dbg).140716-0327) amd64fre
DPC_STACK_BASE: FFFFF88000A94FB0
LAST_CONTROL_TRANSFER: from fffff80059d1bea1 to fffff80059d19f4e
STACK_TEXT:
fffff88000a8d650 fffff80059d1bea1 : 0000000000000000 fffff88000a64180 fffff88000a8d7b0 fffff78000000320 : nt!KeAccumulateTicks+0x57e
fffff88000a8d6d0 fffff80059c33e94 : ffffffffffd104c8 fffffa800a000202 fffffa800b72d700 fffff88000ad6180 : nt!KeUpdateRunTime+0x51
fffff88000a8d700 fffff80059cd06ba : fffffa800ac08530 fffffa800b72d700 fffffa800a000210 fffffa800bdccb60 : hal!HalpTimerClockInterrupt+0x50
fffff88000a8d730 fffff80059d07f55 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiInterruptDispatchNoLockNoEtw+0x1aa
fffff88000a8d8c0 fffff80059d0071f : fffff88000a64180 0000000000000100 fffff88000a8dc00 fffff88000a60000 : nt!KiDeferredReadyThread+0xbce
fffff88000a8d9b0 fffff80059cfec25 : fffff88000a66f10 fffff80059d002cf fffff88000a8dbf0 fffff88000a674c0 : nt!KiProcessExpiredTimerList+0x18f
fffff88000a8dae0 fffff80059d00c38 : fffff88000a64180 fffff88000a66f80 fffff88000000002 fffffa8000005b02 : nt!KiExpireTimerTable+0xa9
fffff88000a8db80 fffff80059cfffd6 : fffffa8000000000 00001f8000a60080 0000000000000000 0000000000000002 : nt!KiTimerExpiration+0xc8
fffff88000a8dc30 fffff80059d00f8a : fffff88000a64180 fffff88000a64180 0000000000000000 fffff88000a70540 : nt!KiRetireDpcList+0x1f6
fffff88000a8dda0 0000000000000000 : fffff88000a8e000 fffff88000a88000 0000000000000000 0000000000000000 : nt!KiIdleLoop+0x5a
STACK_COMMAND: kb
FOLLOWUP_IP:
nt!KeAccumulateTicks+57e
fffff800`59d19f4e cd2c int 2Ch
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: nt!KeAccumulateTicks+57e
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: nt
IMAGE_NAME: ntkrnlmp.exe
DEBUG_FLR_IMAGE_TIMESTAMP: 5507a86c
IMAGE_VERSION: 6.2.9200.17313
BUCKET_ID_FUNC_OFFSET: 57e
FAILURE_BUCKET_ID: 0x133_nt!KeAccumulateTicks
BUCKET_ID: 0x133_nt!KeAccumulateTicks
ANALYSIS_SOURCE: KM
FAILURE_ID_HASH_STRING: km:0x133_nt!keaccumulateticks
FAILURE_ID_HASH: {77140953-e8fc-eb2e-c8a7-c5f30a634add}
Followup: MachineOwner
Any idea what I should start look?
Thanks,
Ying
xxxxx@yahoo.ca wrote:
What is the input argument “Interrupt” for KeAcquireInterruptSpinLock()? Is it comes from InterruptObject field of the IO_INTERRUPT_MESSAGE_INFO_ENTRY data structure or ConnectionContext of the data structure MessageBased after IoConnectInterruptEx() is successfully called?
You have to look at the Version field to know which one of those unions
to use. The MSDN documentation describes this.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
xxxxx@yahoo.ca wrote:
I have used KeAcquireInterruptSpinLock(), I can run two threads without any problem. However after I increase to 4 threads, I get the following error:
…
DPC_TIMEOUT_TYPE: SINGLE_DPC_TIMEOUT_EXCEEDED
That seems pretty clear. On Windows 10, your DPC must complete within
64µs. Remember that, while you hold your interrupt spinlock, you are
disabling interrupts for all devices with a lower IRQL than yours.
That’s unfriendly. How much processing are you doing with the spinlock?
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Looks like that I have two options. 1. Do not use KeAcquireInterruptSpinLock() and offload the work of my IMSR into a lower IRQL thread or DPC so that I can use KeAcquireSpinLock at DISPATCH_LEVEL. 2. Cut the work in my existing DPC so that the processing is within 64?s while it hold my interrupt spinlock.