Share via

Crash Dump Analysis Patterns (Part 13b)

Crash Dump Analysis Patterns (Part 13b)

번역: Heejune Kim (2007-12-10, drost@naver.com, https://insidekernel.net)

 

만약 OS가 할당된 구조체들을 가리키고 있는 핸들이 있다면 때로는 핸들 누수로 인해 메모리 부족을 불러 일으키기도 합니다. 다음은 서버를 멈춰버린 핸들 릭의 전형적인 예입니다. 전체 메모리 덤프는 non-paged pool이 고갈돼버린 상황을 보여줍니다:

0: kd> !vm

*** Virtual Memory Usage ***
Physical Memory:     1048352 (   4193408 Kb)
Page File: \??\C:\pagefile.sys
   Current:   4190208 Kb  Free Space:   3749732 Kb
   Minimum:   4190208 Kb  Maximum:      4190208 Kb
Available Pages:      697734 (   2790936 Kb)
ResAvail Pages:       958085 (   3832340 Kb)
Locked IO Pages:          95 (       380 Kb)
Free System PTEs:     199971 (    799884 Kb)
Free NP PTEs:            105 (       420 Kb)
Free Special NP:           0 (         0 Kb)
Modified Pages:          195 (       780 Kb)
Modified PF Pages:       195 (       780 Kb)
NonPagedPool Usage:    65244 (    260976 Kb)
NonPagedPool Max:      65503 (    262012 Kb)
********** Excessive NonPaged Pool Usage *****
PagedPool 0 Usage:      6576 (     26304 Kb)
PagedPool 1 Usage:       629 (      2516 Kb)
PagedPool 2 Usage:       624 (      2496 Kb)
PagedPool 3 Usage:       608 (      2432 Kb)
PagedPool 4 Usage:       625 (      2500 Kb)
PagedPool Usage:        9062 (     36248 Kb)
PagedPool Maximum:     66560 (    266240 Kb)

********** 184 pool allocations have failed **********

Shared Commit:          7711 (     30844 Kb)
Special Pool:              0 (         0 Kb)
Shared Process:        10625 (     42500 Kb)
PagedPool Commit:       9102 (     36408 Kb)
Driver Commit:          1759 (      7036 Kb)
Committed pages:      425816 (   1703264 Kb)
Commit limit:        2052560 (   8210240 Kb)

 

Non-paged pool 소비량을 보면 지나게 많은 스레드 오브젝트의 수를 볼 수 있습니다:

 

0: kd> !poolused 3
   Sorting by  NonPaged Pool Consumed

  Pool Used:
            NonPaged
Tag    Allocs    Frees     Diff     Used
Thre   772672   463590   309082 192867168  Thread objects , Binary: nt!ps
MmCm       42        9       33 12153104   Calls made to MmAllocateContiguousMemory , Binary: nt!mm
…
…
…

 

다음 순서는 프로세스를 리스트 해보고 그것들의 핸들 사용량을 찾아보는 일입니다:

 

0: kd> !process 0 0
…
…
…
PROCESS 88b75020  SessionId: 7  Cid: 172e4    Peb: 7ffdf000  ParentCid: 17238
    DirBase: c7fb6bc0  ObjectTable: e17f50a0  HandleCount: 143428.
    Image: iexplore.exe
…
…
…

 

저 프로세스를 현재 프로세스 컨텍스트로 바꾸고 그것의 핸들을 리스트하여 스레드 오브젝트에 연속적으로 할당된 핸들을 보여줍니다:

 

0: kd> .process 88b75020
Implicit process is now 88b75020
0: kd> .reload /user

0: kd> !handle
…
…
…
0d94: Object: 88a6b020  GrantedAccess: 001f03ff Entry: e35e1b28
Object: 88a6b020  Type: (8b780c68) Thread
    ObjectHeader: 88a6b008
        HandleCount: 1  PointerCount: 1

0d98: Object: 88a97320  GrantedAccess: 001f03ff Entry: e35e1b30
Object: 88a97320  Type: (8b780c68) Thread
    ObjectHeader: 88a97308
        HandleCount: 1  PointerCount: 1

0d9c: Object: 88b2b020  GrantedAccess: 001f03ff Entry: e35e1b38
Object: 88b2b020  Type: (8b780c68) Thread
    ObjectHeader: 88b2b008
        HandleCount: 1  PointerCount: 1

0da0: Object: 88b2a730  GrantedAccess: 001f03ff Entry: e35e1b40
Object: 88b2a730  Type: (8b780c68) Thread
    ObjectHeader: 88b2a718
        HandleCount: 1  PointerCount: 1

0da4: Object: 88b929a0  GrantedAccess: 001f03ff Entry: e35e1b48
Object: 88b929a0  Type: (8b780c68) Thread
    ObjectHeader: 88b92988
        HandleCount: 1  PointerCount: 1

0da8: Object: 88a57db0  GrantedAccess: 001f03ff Entry: e35e1b50
Object: 88a57db0  Type: (8b780c68) Thread
    ObjectHeader: 88a57d98
        HandleCount: 1  PointerCount: 1

0dac: Object: 88b92db0  GrantedAccess: 001f03ff Entry: e35e1b58
Object: 88b92db0  Type: (8b780c68) Thread
    ObjectHeader: 88b92d98
        HandleCount: 1  PointerCount: 1

0db0: Object: 88b4a730  GrantedAccess: 001f03ff Entry: e35e1b60
Object: 88b4a730  Type: (8b780c68) Thread
    ObjectHeader: 88b4a718
        HandleCount: 1  PointerCount: 1

0db4: Object: 88a7e730  GrantedAccess: 001f03ff Entry: e35e1b68
Object: 88a7e730  Type: (8b780c68) Thread
    ObjectHeader: 88a7e718
        HandleCount: 1  PointerCount: 1

0db8: Object: 88a349a0  GrantedAccess: 001f03ff Entry: e35e1b70
Object: 88a349a0  Type: (8b780c68) Thread
    ObjectHeader: 88a34988
        HandleCount: 1  PointerCount: 1

0dbc: Object: 88a554c0  GrantedAccess: 001f03ff Entry: e35e1b78
Object: 88a554c0  Type: (8b780c68) Thread
    ObjectHeader: 88a554a8
        HandleCount: 1  PointerCount: 1
…
…
…

 

이들 스레드를 조사하면 스택 트레이스와 시작 주소를 보여줍니다:

 

0: kd> !thread 88b4a730
THREAD 88b4a730  Cid 0004.1885c  Teb: 00000000 Win32Thread: 00000000 TERMINATED
Not impersonating
DeviceMap                 e1000930
Owning Process            8b7807a8       Image:         System
Wait Start TickCount      975361         Ticks: 980987 (0:04:15:27.921)
Context Switch Count      1
UserTime                  00:00:00.0000
KernelTime                00:00:00.0000
Start Address mydriver!StatusWaitThread (0xf5c5d128)
Stack Init 0 Current f3c4cc98 Base f3c4d000 Limit f3c4a000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0
ChildEBP RetAddr  Args to Child
f3c4ccac 8083129e ffdff5f0 8697ba00 a674c913 hal!KfLowerIrql+0×62
f3c4ccc8 00000000 808ae498 8697ba00 00000000 nt!KiExitDispatcher+0×130

0: kd> !thread 88a554c0
THREAD 88a554c0  Cid 0004.1888c  Teb: 00000000 Win32Thread: 00000000 TERMINATED
Not impersonating
DeviceMap                 e1000930
Owning Process            8b7807a8       Image:         System
Wait Start TickCount      975380         Ticks: 980968 (0:04:15:27.625)
Context Switch Count      1
UserTime                  00:00:00.0000
KernelTime                00:00:00.0000
Start Address mydriver!StatusWaitThread (0xf5c5d128)
Stack Init 0 Current f3c4cc98 Base f3c4d000 Limit f3c4a000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0
ChildEBP RetAddr  Args to Child
f3c4ccac 8083129e ffdff5f0 8697ba00 a674c913 hal!KfLowerIrql+0×62
f3c4ccc8 00000000 808ae498 8697ba00 00000000 nt!KiExitDispatcher+0×130

 

스레드가 종료되었고 스레드의 시작 주소는 mydriver.sys에 속한다는 것을 볼 수 있습니다. 따라서 우리는 핸들 릭의 원인을 찾기 위해서는 mydriver 를 조사해야 한다고 말할 수 있습니다.

- Dmitry Vostokov @ DumpAnalysis.org -

Comments