IRQL for RtlStringxxx functions

I have a question about kernl run-time functions. There are many Rlt**
functions for kernel such as RtlCchPrintf/RtlCchCat/RtlCopyString and so
on. I’m very confused about their declaration of IRQL level that most of
them can only be called in PASSIVE_LEVEL. See below:

  1. RtlCopyString: Any Level (if both source and destination are resident)
  2. RtlAppendStringToString: <= APC_LEVEL
  3. RtlCchPrintf/RtlCchCat: Passive_level

I can understand about any level and <=APC_LEVEL. If both source and
destination buffers are resident, such functions can be called in any level
else lower than DISPATCH_LEVEL. But why PASSIVE_LEVEL(3)? If such functions
will allocate memory from paged pool, this won’t prevent them being called
in APC_LEVE, right?

By de-assembling RtlCchPrintf, it will at last call into nt!_vsnwprintf. Is
vsnwprintf the reason for PASSIVE_LEVEL? Anyone can show me some light?
Thanks.

lkd> uf nt!RtlStringCchPrintfW
nt!RtlStringCchPrintfW:
82267453 8bff mov edi,edi
82267455 55 push ebp
82267456 8bec mov ebp,esp
82267458 33c0 xor eax,eax
8226745a 85c9 test ecx,ecx
8226745c 7408 je nt!RtlStringCchPrintfW+0x13 (82267466)

nt!RtlStringCchPrintfW+0xb:
8226745e 81f9ffffff7f cmp ecx,7FFFFFFFh
82267464 7605 jbe nt!RtlStringCchPrintfW+0x18 (8226746b)

nt!RtlStringCchPrintfW+0x13:
82267466 b80d0000c0 mov eax,0C000000Dh

nt!RtlStringCchPrintfW+0x18:
8226746b 85c0 test eax,eax
8226746d 7c35 jl nt!RtlStringCchPrintfW+0x4f (822674a4)

nt!RtlStringCchPrintfW+0x1c:
8226746f 53 push ebx
82267470 56 push esi
82267471 8d450c lea eax,[ebp+0Ch]
82267474 50 push eax
82267475 ff7508 push dword ptr [ebp+8]
82267478 8d71ff lea esi,[ecx-1]
8226747b 56 push esi
8226747c 57 push edi
8226747d 33db xor ebx,ebx
8226747f e8c6c80800 call nt!_vsnwprintf (822f3d4a)
82267484 83c410 add esp,10h
82267487 85c0 test eax,eax
82267489 7c0a jl nt!RtlStringCchPrintfW+0x40 (82267495)

nt!RtlStringCchPrintfW+0x38:
8226748b 3bc6 cmp eax,esi
8226748d 7706 ja nt!RtlStringCchPrintfW+0x40 (82267495)

nt!RtlStringCchPrintfW+0x3c:
8226748f 7409 je nt!RtlStringCchPrintfW+0x45 (8226749a)

nt!RtlStringCchPrintfW+0x3e:
82267491 eb0d jmp nt!RtlStringCchPrintfW+0x4b (822674a0)

nt!RtlStringCchPrintfW+0x40:
82267495 bb05000080 mov ebx,80000005h

nt!RtlStringCchPrintfW+0x45:
8226749a 33c0 xor eax,eax
8226749c 66890477 mov word ptr [edi+esi*2],ax

nt!RtlStringCchPrintfW+0x4b:
822674a0 5e pop esi
822674a1 8bc3 mov eax,ebx
822674a3 5b pop ebx

nt!RtlStringCchPrintfW+0x4f:
822674a4 5d pop ebp
822674a5 c3 ret

=================================
Best Regards!
Moore.Zhang (Zhang Pei)

I hope you realize that RtlCchPrintf can be an inline function and thus you have complete source in ntstrsafe.h, no need to disassemble. The irql requirement is because the NLS tables used for string conversion are pagable.

d

debt from my phone


From: Moore Zhang
Sent: 12/14/2011 9:15 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] IRQL for RtlStringxxx functions

I have a question about kernl run-time functions. There are many Rlt** functions for kernel such as RtlCchPrintf/RtlCchCat/RtlCopyString and so on. I’m very confused about their declaration of IRQL level that most of them can only be called in PASSIVE_LEVEL. See below:

  1. RtlCopyString: Any Level (if both source and destination are resident)
  2. RtlAppendStringToString: <= APC_LEVEL
  3. RtlCchPrintf/RtlCchCat: Passive_level

I can understand about any level and <=APC_LEVEL. If both source and destination buffers are resident, such functions can be called in any level else lower than DISPATCH_LEVEL. But why PASSIVE_LEVEL(3)? If such functions will allocate memory from paged pool, this won’t prevent them being called in APC_LEVE, right?

By de-assembling RtlCchPrintf, it will at last call into nt!_vsnwprintf. Is vsnwprintf the reason for PASSIVE_LEVEL? Anyone can show me some light? Thanks.

lkd> uf nt!RtlStringCchPrintfW
nt!RtlStringCchPrintfW:
82267453 8bff mov edi,edi
82267455 55 push ebp
82267456 8bec mov ebp,esp
82267458 33c0 xor eax,eax
8226745a 85c9 test ecx,ecx
8226745c 7408 je nt!RtlStringCchPrintfW+0x13 (82267466)

nt!RtlStringCchPrintfW+0xb:
8226745e 81f9ffffff7f cmp ecx,7FFFFFFFh
82267464 7605 jbe nt!RtlStringCchPrintfW+0x18 (8226746b)

nt!RtlStringCchPrintfW+0x13:
82267466 b80d0000c0 mov eax,0C000000Dh

nt!RtlStringCchPrintfW+0x18:
8226746b 85c0 test eax,eax
8226746d 7c35 jl nt!RtlStringCchPrintfW+0x4f (822674a4)

nt!RtlStringCchPrintfW+0x1c:
8226746f 53 push ebx
82267470 56 push esi
82267471 8d450c lea eax,[ebp+0Ch]
82267474 50 push eax
82267475 ff7508 push dword ptr [ebp+8]
82267478 8d71ff lea esi,[ecx-1]
8226747b 56 push esi
8226747c 57 push edi
8226747d 33db xor ebx,ebx
8226747f e8c6c80800 call nt!_vsnwprintf (822f3d4a)
82267484 83c410 add esp,10h
82267487 85c0 test eax,eax
82267489 7c0a jl nt!RtlStringCchPrintfW+0x40 (82267495)

nt!RtlStringCchPrintfW+0x38:
8226748b 3bc6 cmp eax,esi
8226748d 7706 ja nt!RtlStringCchPrintfW+0x40 (82267495)

nt!RtlStringCchPrintfW+0x3c:
8226748f 7409 je nt!RtlStringCchPrintfW+0x45 (8226749a)

nt!RtlStringCchPrintfW+0x3e:
82267491 eb0d jmp nt!RtlStringCchPrintfW+0x4b (822674a0)

nt!RtlStringCchPrintfW+0x40:
82267495 bb05000080 mov ebx,80000005h

nt!RtlStringCchPrintfW+0x45:
8226749a 33c0 xor eax,eax
8226749c 66890477 mov word ptr [edi+esi*2],ax

nt!RtlStringCchPrintfW+0x4b:
822674a0 5e pop esi
822674a1 8bc3 mov eax,ebx
822674a3 5b pop ebx

nt!RtlStringCchPrintfW+0x4f:
822674a4 5d pop ebp
822674a5 c3 ret

=================================
Best Regards!
Moore.Zhang (Zhang Pei)
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Cool! Always thanks to you Holan!

?? 2011??12??15?? ???1:27??Doron Holan д???

> I hope you realize that RtlCchPrintf can be an inline function and thus
> you have complete source in ntstrsafe.h, no need to disassemble. The irql
> requirement is because the NLS tables used for string conversion are
> pagable.
>
> d
>
> debt from my phone
> ------------------------------
> From: Moore Zhang
> Sent: 12/14/2011 9:15 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] IRQL for RtlStringxxx functions
>
> I have a question about kernl run-time functions. There are many Rlt**
> functions for kernel such as RtlCchPrintf/RtlCchCat/RtlCopyString and so
> on. I’m very confused about their declaration of IRQL level that most of
> them can only be called in PASSIVE_LEVEL. See below:
>
> 1. RtlCopyString: Any Level (if both source and destination are resident)
> 2. RtlAppendStringToString: <= APC_LEVEL
> 3. RtlCchPrintf/RtlCchCat: Passive_level
>
> I can understand about any level and <=APC_LEVEL. If both source and
> destination buffers are resident, such functions can be called in any level
> else lower than DISPATCH_LEVEL. But why PASSIVE_LEVEL(3)? If such functions
> will allocate memory from paged pool, this won’t prevent them being called
> in APC_LEVE, right?
>
> By de-assembling RtlCchPrintf, it will at last call into nt!_vsnwprintf.
> Is vsnwprintf the reason for PASSIVE_LEVEL? Anyone can show me some light?
> Thanks.
>
> lkd> uf nt!RtlStringCchPrintfW
> nt!RtlStringCchPrintfW:
> 82267453 8bff mov edi,edi
> 82267455 55 push ebp
> 82267456 8bec mov ebp,esp
> 82267458 33c0 xor eax,eax
> 8226745a 85c9 test ecx,ecx
> 8226745c 7408 je nt!RtlStringCchPrintfW+0x13 (82267466)
>
> nt!RtlStringCchPrintfW+0xb:
> 8226745e 81f9ffffff7f cmp ecx,7FFFFFFFh
> 82267464 7605 jbe nt!RtlStringCchPrintfW+0x18 (8226746b)
>
> nt!RtlStringCchPrintfW+0x13:
> 82267466 b80d0000c0 mov eax,0C000000Dh
>
> nt!RtlStringCchPrintfW+0x18:
> 8226746b 85c0 test eax,eax
> 8226746d 7c35 jl nt!RtlStringCchPrintfW+0x4f (822674a4)
>
> nt!RtlStringCchPrintfW+0x1c:
> 8226746f 53 push ebx
> 82267470 56 push esi
> 82267471 8d450c lea eax,[ebp+0Ch]
> 82267474 50 push eax
> 82267475 ff7508 push dword ptr [ebp+8]
> 82267478 8d71ff lea esi,[ecx-1]
> 8226747b 56 push esi
> 8226747c 57 push edi
> 8226747d 33db xor ebx,ebx
> 8226747f e8c6c80800 call nt!_vsnwprintf (822f3d4a)
> 82267484 83c410 add esp,10h
> 82267487 85c0 test eax,eax
> 82267489 7c0a jl nt!RtlStringCchPrintfW+0x40 (82267495)
>
> nt!RtlStringCchPrintfW+0x38:
> 8226748b 3bc6 cmp eax,esi
> 8226748d 7706 ja nt!RtlStringCchPrintfW+0x40 (82267495)
>
> nt!RtlStringCchPrintfW+0x3c:
> 8226748f 7409 je nt!RtlStringCchPrintfW+0x45 (8226749a)
>
> nt!RtlStringCchPrintfW+0x3e:
> 82267491 eb0d jmp nt!RtlStringCchPrintfW+0x4b (822674a0)
>
> nt!RtlStringCchPrintfW+0x40:
> 82267495 bb05000080 mov ebx,80000005h
>
> nt!RtlStringCchPrintfW+0x45:
> 8226749a 33c0 xor eax,eax
> 8226749c 66890477 mov word ptr [edi+esi*2],ax
>
> nt!RtlStringCchPrintfW+0x4b:
> 822674a0 5e pop esi
> 822674a1 8bc3 mov eax,ebx
> 822674a3 5b pop ebx
>
> nt!RtlStringCchPrintfW+0x4f:
> 822674a4 5d pop ebp
> 822674a5 c3 ret
> –
> =================================
> Best Regards!
> Moore.Zhang (Zhang Pei)
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>


=================================
Best Regards!
Moore.Zhang (Zhang Pei)

Hloan, I’m still looking NLS table. One more question: If NLS is in pagable
area, it can still be used in APC_LEVEL, right? Why PASSIVE_LEVEL is
declared?

?? 2011??12??15?? ???1:41??Moore Zhang д???

> Cool! Always thanks to you Holan!
>
>
> ?? 2011??12??15?? ???1:27??Doron Holan д???
>
> I hope you realize that RtlCchPrintf can be an inline function and thus
>> you have complete source in ntstrsafe.h, no need to disassemble. The irql
>> requirement is because the NLS tables used for string conversion are
>> pagable.
>>
>> d
>>
>> debt from my phone
>> ------------------------------
>> From: Moore Zhang
>> Sent: 12/14/2011 9:15 PM
>> To: Windows System Software Devs Interest List
>> Subject: [ntdev] IRQL for RtlStringxxx functions
>>
>> I have a question about kernl run-time functions. There are many Rlt**
>> functions for kernel such as RtlCchPrintf/RtlCchCat/RtlCopyString and so
>> on. I’m very confused about their declaration of IRQL level that most of
>> them can only be called in PASSIVE_LEVEL. See below:
>>
>> 1. RtlCopyString: Any Level (if both source and destination are resident)
>> 2. RtlAppendStringToString: <= APC_LEVEL
>> 3. RtlCchPrintf/RtlCchCat: Passive_level
>>
>> I can understand about any level and <=APC_LEVEL. If both source and
>> destination buffers are resident, such functions can be called in any level
>> else lower than DISPATCH_LEVEL. But why PASSIVE_LEVEL(3)? If such functions
>> will allocate memory from paged pool, this won’t prevent them being called
>> in APC_LEVE, right?
>>
>> By de-assembling RtlCchPrintf, it will at last call into nt!_vsnwprintf.
>> Is vsnwprintf the reason for PASSIVE_LEVEL? Anyone can show me some light?
>> Thanks.
>>
>> lkd> uf nt!RtlStringCchPrintfW
>> nt!RtlStringCchPrintfW:
>> 82267453 8bff mov edi,edi
>> 82267455 55 push ebp
>> 82267456 8bec mov ebp,esp
>> 82267458 33c0 xor eax,eax
>> 8226745a 85c9 test ecx,ecx
>> 8226745c 7408 je nt!RtlStringCchPrintfW+0x13 (82267466)
>>
>> nt!RtlStringCchPrintfW+0xb:
>> 8226745e 81f9ffffff7f cmp ecx,7FFFFFFFh
>> 82267464 7605 jbe nt!RtlStringCchPrintfW+0x18 (8226746b)
>>
>> nt!RtlStringCchPrintfW+0x13:
>> 82267466 b80d0000c0 mov eax,0C000000Dh
>>
>> nt!RtlStringCchPrintfW+0x18:
>> 8226746b 85c0 test eax,eax
>> 8226746d 7c35 jl nt!RtlStringCchPrintfW+0x4f (822674a4)
>>
>> nt!RtlStringCchPrintfW+0x1c:
>> 8226746f 53 push ebx
>> 82267470 56 push esi
>> 82267471 8d450c lea eax,[ebp+0Ch]
>> 82267474 50 push eax
>> 82267475 ff7508 push dword ptr [ebp+8]
>> 82267478 8d71ff lea esi,[ecx-1]
>> 8226747b 56 push esi
>> 8226747c 57 push edi
>> 8226747d 33db xor ebx,ebx
>> 8226747f e8c6c80800 call nt!_vsnwprintf (822f3d4a)
>> 82267484 83c410 add esp,10h
>> 82267487 85c0 test eax,eax
>> 82267489 7c0a jl nt!RtlStringCchPrintfW+0x40 (82267495)
>>
>> nt!RtlStringCchPrintfW+0x38:
>> 8226748b 3bc6 cmp eax,esi
>> 8226748d 7706 ja nt!RtlStringCchPrintfW+0x40 (82267495)
>>
>> nt!RtlStringCchPrintfW+0x3c:
>> 8226748f 7409 je nt!RtlStringCchPrintfW+0x45 (8226749a)
>>
>> nt!RtlStringCchPrintfW+0x3e:
>> 82267491 eb0d jmp nt!RtlStringCchPrintfW+0x4b (822674a0)
>>
>> nt!RtlStringCchPrintfW+0x40:
>> 82267495 bb05000080 mov ebx,80000005h
>>
>> nt!RtlStringCchPrintfW+0x45:
>> 8226749a 33c0 xor eax,eax
>> 8226749c 66890477 mov word ptr [edi+esi*2],ax
>>
>> nt!RtlStringCchPrintfW+0x4b:
>> 822674a0 5e pop esi
>> 822674a1 8bc3 mov eax,ebx
>> 822674a3 5b pop ebx
>>
>> nt!RtlStringCchPrintfW+0x4f:
>> 822674a4 5d pop ebp
>> 822674a5 c3 ret
>> –
>> =================================
>> Best Regards!
>> Moore.Zhang (Zhang Pei)
>> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
>> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
>> the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>
>
>
> –
> =================================
> Best Regards!
> Moore.Zhang (Zhang Pei)
>


=================================
Best Regards!
Moore.Zhang (Zhang Pei)

Moore Zhang wrote:

Hloan, I’m still looking NLS table. One more question: If NLS is in
pagable area, it can still be used in APC_LEVEL, right? Why
PASSIVE_LEVEL is declared?

The “NLS tables” refer to the tables that convert between Unicode and
the various 8-bit character sets. They are pageable because they are
large, and because the vast majority of the tables are never used.

The interaction of paging I/O with APC_LEVEL is complicated. Even as a
driver guy, there are many aspects of that I do not understand. Rather
than try to enumerate a long list of restrictions that say “usable at
APC_LEVEL with these exceptions”, it’s safer just to say PASSIVE_LEVEL.

The reality is that the typical driver doesn’t do that much text
manipulation, so the PASSIVE_LEVEL thing is rarely a problem.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

> I can understand about any level and <=APC_LEVEL. If both source and destination buffers

are resident, such functions can be called in any level else lower than DISPATCH_LEVEL.
But why PASSIVE_LEVEL(3)? If such functions will allocate memory from paged pool,
this won’t prevent them being called in APC_LEVE, right?

Let’s try to put it simply, without going into any details.

DISPATCH_LEVEL is simply a Windows equivalent of something known as “cannot block” notion under the other OSes. Once retrieving a page from the disk invariably involves blocking a failing thread you cannot incur page faults at elevated IRQL. Simple, ugh…

APC_LEVEL is an indication that you may be in the paging IO path to the pagefile. You can go blocking without a slightest problem, but consider what happens if you incur a page fault - in order to keep on going you have to retrieve a page from the pagefile first, but, by doing so, you incur yet another page fault and so on and so forth up to infinity. This is what APC_LEVEL constraint is for. In “regular” drivers that are not in file system stack running at this IRQL is pretty uncommon scenario, and normally is the result of using FAST_MUTEX synchronization construct…

Anton Bassov

This sure is a high frequency topic. Many people need to do string work at raised irql. It would be great if there was an API to lock/unlock the NLS table to make this possible. Or if practical future windows the table can be nonpaged, relaxing the irql requirements meanwhile retaining compatibility with existing drivers. How big is the nls table?

Windows is striving at every release to reduce memory usage, moving nls from paged to non paged would be highly difficult wrt that goal

d

debt from my phone


From: xxxxx@gmail.com
Sent: 12/15/2011 5:23 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] IRQL for RtlStringxxx functions

This sure is a high frequency topic. Many people need to do string work at raised irql. It would be great if there was an API to lock/unlock the NLS table to make this possible. Or if practical future windows the table can be nonpaged, relaxing the irql requirements meanwhile retaining compatibility with existing drivers. How big is the nls table?


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Wow, I got so much from you guys! If we want to use string functions in
raised level, unicode string functions can be used such as:
RtlAppendUnicodeToString/RtlCopyUnicodeString

2011/12/16 AM 9:34??Doron Holan Wrote??

> Windows is striving at every release to reduce memory usage, moving nls
> from paged to non paged would be highly difficult wrt that goal
>
> d
>
> debt from my phone
> ------------------------------
> From: xxxxx@gmail.com
> Sent: 12/15/2011 5:23 PM
> To: Windows System Software Devs Interest List
> Subject: RE:[ntdev] IRQL for RtlStringxxx functions
>
> This sure is a high frequency topic. Many people need to do string work
> at raised irql. It would be great if there was an API to lock/unlock the
> NLS table to make this possible. Or if practical future windows the table
> can be nonpaged, relaxing the irql requirements meanwhile retaining
> compatibility with existing drivers. How big is the nls table?
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>


=================================
Best Regards!
Moore.Zhang (Zhang Pei)

> This sure is a high frequency topic. Many people need to do string work at raised irql.

Only ANSI<->Unicode conversions require passive, which is done extremely rarely since all Windows kernel is Unicode.

Also, you can make your own code of “poor man’s Unicode” which just adds senior bytes of zeroes.


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

most RtlXxxString routines cannot be called at IRQL > PASSIVE_LEVEL

xxxxx@gmail.com wrote:

This sure is a high frequency topic. Many people need to do string work at raised irql. It would be great if there was an API to lock/unlock the NLS table to make this possible. Or if practical future windows the table can be nonpaged, relaxing the irql requirements meanwhile retaining compatibility with existing drivers. How big is the nls table?

You can come up with an estimate. Single-byte character set code pages
require about 70kB for translation tables. Double-byte character set
code pages might require twice that. How many code pages are there?
30? 50? So, we’re talking about roughly 10 megabytes, within an order
of magnitude. The issue is that, for the vast majority of users, only
one code page will ever be referenced. The rest is just wasted memory.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

These NLS tables seem to be in /NLS/NlsSectionxxx
If so, their size is visible in ObjDir. Is it in bytes or pages?

It should be possible to lock down all these sections temporarily.

– pa

Mostly you don’t even need these functions unless you need to do a
conversion. It is easy to rewrite them to work at any IRQL. Instead of copy
string, use copy memory. The same for the append functions. For non NULL
terminated strings, lengths are already known otherwise it requires scanning
for the NULL character which is also not that hard.

//Daniel

“Pavel A.” wrote in message news:xxxxx@ntdev…
> These NLS tables seem to be in /NLS/NlsSectionxxx If so, their size is
> visible in ObjDir. Is it in bytes or pages?
>
> It should be possible to lock down all these sections temporarily.
>
> – pa