Extension for CPU time consumed?

WINDBG
1

Hi,

I would like to solve a problem of a PC whose CPU is utilized to 100%. I tried procmon and procexp, and it looks like one of the svchosts is the culprit. Now I would like to know which thread utilizes most CPU time.

I can do that using !process XXXXXXXX 0, but it would be nice to have a tool that dumps all threads sorted by CPU time used (KTHREAD::UserTime+KTHREAD::kernelTime). I plan to write such an extension, but to prevent reinventing the wheel, isn’t there an existing one already?

Thanks in advance,

L.

2

Have you tried !runaway (after snapping a user mode dump of the process in question) ?

Of course, if there is always one CPU spinning, you could break in with the kernel debugger and see what’s running on all processors (!running -it); one is likely your culprit.

  • S (Msft)

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@volny.cz
Sent: Tuesday, November 19, 2013 11:28 PM
To: Kernel Debugging Interest List
Subject: [windbg] Extension for CPU time consumed?

Hi,

I would like to solve a problem of a PC whose CPU is utilized to 100%. I tried procmon and procexp, and it looks like one of the svchosts is the culprit. Now I would like to know which thread utilizes most CPU time.

I can do that using !process XXXXXXXX 0, but it would be nice to have a tool that dumps all threads sorted by CPU time used (KTHREAD::UserTime+KTHREAD::kernelTime). I plan to write such an extension, but to prevent reinventing the wheel, isn’t there an existing one already?

Thanks in advance,

L.

WINDBG is sponsored by OSR

OSR is hiring!! Info at http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

> Have you tried !runaway (after snapping a user mode dump of the process in
question) ?

Nope, because I didn’t know it exists :slight_smile:
I have kernel dump from the machine (it’s been reinstalled already),
so !runaway does not work. !running only shows the thread for “bang.sys”
that was used to create the dump.

Since kernel time and user time is stored in KTHREAD,
which is part of kernel memory, it should still be possible
to display thread list sorted by CPU usage, right?

L.

4

Yes, you could write an extension to do it for kernel mode if you can’t get a user dump. (!runaway only knows how to operate on user mode targets.)

There are some handy wrappers in engextcpp that might be helpful if you wanted to write something to do that (ExtNtOsInformation::GetKernelProcessList, ExtNtOsInformation::GetKernelProcessThreadList).

Or, if you particularly like writing scriptlets in the debugger interpreter language, see !for_each_thread.

  • S (Msft)

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, November 20, 2013 12:00 AM
To: Kernel Debugging Interest List
Subject: RE: [windbg] Extension for CPU time consumed?

Have you tried !runaway (after snapping a user mode dump of the
process in
question) ?

Nope, because I didn’t know it exists :slight_smile: I have kernel dump from the machine (it’s been reinstalled already), so !runaway does not work. !running only shows the thread for “bang.sys”
that was used to create the dump.

Since kernel time and user time is stored in KTHREAD, which is part of kernel memory, it should still be possible to display thread list sorted by CPU usage, right?

L.

WINDBG is sponsored by OSR

OSR is hiring!! Info at http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

5

I have source to a debugger extension here that walks each thread, captures
the call stack, and then only reports the unique ones at the end:

http://www.osronline.com/OsrDown.cfm/apexts.zip?name=apexts.zip&id=559

Only one way to do it (it leverages !for_each_thread), but it should be easy
enough to modify to capture/sort CPU time instead.

-scott
OSR

“Skywing” wrote in message news:xxxxx@windbg…

Yes, you could write an extension to do it for kernel mode if you can’t get
a user dump. (!runaway only knows how to operate on user mode targets.)

There are some handy wrappers in engextcpp that might be helpful if you
wanted to write something to do that
(ExtNtOsInformation::GetKernelProcessList,
ExtNtOsInformation::GetKernelProcessThreadList).

Or, if you particularly like writing scriptlets in the debugger interpreter
language, see !for_each_thread.

  • S (Msft)

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, November 20, 2013 12:00 AM
To: Kernel Debugging Interest List
Subject: RE: [windbg] Extension for CPU time consumed?

Have you tried !runaway (after snapping a user mode dump of the
process in
question) ?

Nope, because I didn’t know it exists :slight_smile: I have kernel dump from the
machine (it’s been reinstalled already), so !runaway does not work. !running
only shows the thread for “bang.sys”
that was used to create the dump.

Since kernel time and user time is stored in KTHREAD, which is part of
kernel memory, it should still be possible to display thread list sorted by
CPU usage, right?

L.

WINDBG is sponsored by OSR

OSR is hiring!! Info at http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

6

this one liner will print totaltime kerneltime usertime and thread #
and a count of threads for sanity check in an excellable or gnuwin32
sortable format see if this is any good for your use

r $t4= 0; !for_each_thread "r $t4= @$t4+1; r? $t0 = ((nt!_ETHREAD *)
@#Thread )->Tcb.UserTime; r? $t1 = ((nt!_ETHREAD *) @#Thread
)->Tcb.KernelTime; r $t2 = @$t0+@$t1; .printf "%04x %04x %04x
@#Thread \n" ,@$t2,@$t1,@$t0 "; ? @$t4

windbg says 445 threads running

0000 0000 0000 0xffffffff8622f6a0
0005 0002 0003 0xffffffff8673fda8
0007 0006 0001 0xffffffff8639c020
Evaluate expression: 445 = 000001bd <---------------

a sample ascending sorted output using gnuwin sort (windbg output copy
pasted to a .txt file named times.txt )

wc -l times.txt
445 times.txt

sort /REVERSE times.txt > sortedtimes.txt

wc -l sortedtimes.txt
445 sortedtimes.txt

head -n 3 times.txt sortedtimes.txt
==> times.txt <==
029c 029c 0000 0xffffffff86fc65b8
0049 0049 0000 0xffffffff86fc5da8
0056 0056 0000 0xffffffff86fc5b30

==> sortedtimes.txt <==
c8e9 a16b 277e 0xffffffff861c9be8
c864 a08c 27d8 0xffffffff867c8688
c7ee a0c0 272e 0xffffffff86814530

On 11/20/13, xxxxx@volny.cz wrote:
> Hi,
>
> I would like to solve a problem of a PC whose CPU is utilized to 100%. I
> tried procmon and procexp, and it looks like one of the svchosts is the
> culprit. Now I would like to know which thread utilizes most CPU time.
>
> I can do that using !process XXXXXXXX 0, but it would be nice to have a tool
> that dumps all threads sorted by CPU time used
> (KTHREAD::UserTime+KTHREAD::kernelTime). I plan to write such an extension,
> but to prevent reinventing the wheel, isn’t there an existing one already?
>
> Thanks in advance,
>
> L.
>
> —
> WINDBG is sponsored by OSR
>
> OSR is hiring!! Info at http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

7

> Only one way to do it (it leverages !for_each_thread), but it

should be easy enough to modify to capture/sort CPU time instead.

Thanks Scott. Nice trick to make WinDbg call your own extension
callback from within for_each_thread command :slight_smile:

I might consider implementing it the same way how WinDbg
implements !for_each_thread, which is looking up for nt!PsActiveProcessHead,
then parsing threads starting with KPROCESS::ThreadListHead.

L.