Windows 10: WinDBG - The Basics for Debugging Crash Dumps in Windows 10

How to: WinDBG - The Basics for Debugging Crash Dumps in Windows 10

WinDBG - The Basics for Debugging Crash Dumps in Windows 10


Information WinDBG (Windows DeBuGger) is an analytic tool used for analysing and debugging Windows crash dumps, also known as BSODs (Blue Screens of Death). It is part of the Windows Developer Kit which is a free download from Microsoft and is used by the vast majority of debuggers, including here on Ten Forums.



Note The tutorial below outlines the basic instructions when analysing dump files, it is by no means a complete and definitive guide. This tutorial will show you how to look for problematic drivers and highlights a few other common basic issues that arise when debugging. There are many commands to master within WinDBG and many different methods to use for different STOP codes which aren't covered in this tutorial. Mastering WinDBG and becoming proficient in debugging crash dumps take a lot of time and dedication, it is certainly not a quick process but can be highly rewarding once dumps start to make more sense.




INTRODUCTION [/i] Table of Contents
Part One: WinDBG Installation and Configuration
Part Two: Opening BSOD Logs
Part Three: The Dump File and Basic Analysis
Part Four: Common Bugchecks and Plans of Attack
Part Five: SCSI Pass-Thru Direct (SPTD)
Part Six: Recommended Research Materials




PART ONE [/i] WinDBG Installation and Configuration
To install and configure WinDBG follow WinDBG - Install Configure - Windows 10 Forums for full instructions.



PART TWO [/i] Opening BSOD Logs
Step One: 7Zip and Opening Dump Files
Step Two: The WinDBG Interface



STEP ONE [/i] 7Zip and Opening Dump Files

Information Dump files are saved with the .dmp file extension and are identified by the icon below.



Small memory dump files (most commonly used for analysing BSODs) are saved locally to %SystemRoot%\Minidump.

For information on configuring which dump files Windows creates see Dump Files - Configure Windows 10 to Create on BSOD.
A) When a log file is downloaded from Ten Forums it will be in the form of a .zip file. The easiest way to access the contents is using 7Zip, a free open source file compression tool. Once 7Zip is installed download an uploaded log file from a thread in BSOD Crashes and Debugging - Windows 10 Forums and open the saved destination folder. Logs can appear as ZIP Files or WinRAR Archives (see image below).
EXAMPLE:


B) Right click on the log you wish to examine, hover over 7Zip in the context menu and click/tap on Open archive



C) Once the archive is opened you'll see the dump files located at the top of the archive. The number of dump files present will vary from archive to archive and is completely dependent on how many BSODs the user has received.





Tip Always open dumps from the most recent date backwards, opening older dumps first can give you a false picture of what the current issue actually is. Note the date of the dump files too, dumps that are many months old may no longer be relevant.
D) On occasion you will encounter an archive that contains no dump files. This usually means that the user's Windows configuration is not set to create dump files on BSOD, refer the user to Dump Files - Configure Windows 10 to Create on BSOD to ensure the machine creates dump files on subsequent BSODs. The user will have to upload new logs once new BSODs have been encountered to capture the new dump files.
EXAMPLE:




E) Double click/tap on the most recent dump which will open in WinDBG provided it has been configured correctly.



STEP TWO [/i] The WinDBG Interface
When you open your first dump file you will be greeted with a window like the one below.



It's worth noting that the dump readout window (highlighted in the screenshot above) is separate to the WinDBG window. It can be moved around and/or resized (by dragging the corner of the window) within the WinDBG window. Manipulate the dump readout window to the size that suits your personal preference, giving yourself a bigger workspace can make for easier reading.
EXAMPLE: Dump window manually resized for larger viewing area:




The dump file will take a few seconds to load as it connects to the internet and downloads the required symbols to display in the readout. You will see the message Debugee not connected in the command bar while the symbols are downloaded. You can not work on the dump file until all of the relevant symbols are downloaded.



Once the symbols have been downloaded and the dump is ready to analyse you will see the message Followup: MachineOwner at the bottom of the dump text.



At this point you're ready to start analysing the dump file. All commands are entered into the command bar at the bottom of the dump window.






PART THREE [/i] The Dump File and Basic Analysis
Step One: Bugchecks At a Glance
Step Two: !thread and Driver Analysis
Step Three: !analyze-v and !irp




STEP ONE [/i] Bugchecks At a Glance
Bugchecks are always presented in the form of a STOP followed by a hexadecimal number, the hexadecimal value always being 0xXXXXXXXX. The hexadecimal code defines the type of bugcheck. Bugchecks can be referred to in the following ways:

EXAMPLE:

STOP: 0x0000000A
0x0000000A
0xA
A
All of these methods equate to the same bugcheck. For the remainder of this tutorial bugchecks will be referred to in the shorthand format (0xA).

The first thing you need to note when opening a dump file is the Bugcheck code, this will largely define the underlying issue that caused the crash. To identify a Bugcheck code refer to the Windows Dev Center Bug Check Code Reference. In the example below we'll be dealing with a 0x9F.



Having consulted the Bug Check Code Reference you will learn that a 0x9F is a later in this tutorial.



STEP TWO [/i] !thread and Driver Analysis
One common cause of BSODs is third party device drivers. To see if a device driver has been named in the crash dump you will need to dump the stack, follow the steps below to perform the procedure and see which drivers, if any, have been named in the stack.

1: In the command bar type in !thread and hit enter, this will display the information from the active thread at the time of the crash.

2: Scan through the thread until you find the Base and Limit texts and their hexadecimal outputs.
Code:

In the command bar type dps followed by the Limit value first and the Base value second, it is important to input the values in the correct order. Hit enter to dump the stack.
Code:

Tip You cannot right click and copy within the WinDBG window. Select the text required by highlighting with the mouse and using CTRL+C, then paste the text into the command bar using CTRL+V.

3: When the stack has been dumped you will see a wall of hexadecimal text, a small example is outlined in the code below:
Code:

4: To see whether a driver or drivers are named in the stack, scroll through and scan for outputs like the two examples below:







Here you can see two offending drivers, atikmdag.sys and lvuvc64.sys. These drivers have been named in the crash dump so you can assume they're contributing to the problem. First you must identify the drivers, to do this refer to the archive opened in Part One C) and open the drivers.txt file.


Note If your default program association for .txt files isn't already set to Notepad, change it prior to opening.
5: In the drivers.txt file, looking in the Path column, scroll down until you find the name of the driver listed in the dump file. Reference the Display Name and Description columns for the identity of the driver.
atikmdag.sys:



In this case the Display Name and Description read the same as the driver name, in such an instance a quick Google search for atikmdag.sys will reveal it's the ATI Graphics driver.
lvuvc64.sys




In this case the driver is named as a LogitechHD Webcam C61 driver.

6: Now you have identified the drivers, check the age of each one by typing lmvm followed by the driver name (minus the .sys) into the command bar of WinDBG and hit enter.
Code:

Gives the readout:

Code:

The date of the driver is highlighted giving an indication of whether an update to a newer version or rollback to an earlier version should be applied to test.
Code:

Gives the readout:

Code:

In this case the date of the driver would indicate an update is required as the installed version is relatively old.

This method can be applied to all drivers that appear in stacks within dump files. It is worth noting however that not all dump files will show drivers as the problem, in some instances it may be hardware that is causing issues but if the STOP code is one that normally indicates a driver as being the problem it may be necessary to run Driver Verifier to weed out the offending driver. These calls come with experience, and identifying drivers as causes of BSODs is just the tip of the proverbial iceberg.



STEP THREE [/i] !analyze-v
The !analyze command displays information about the current exception (or Bugcheck), adding the -v extension displays the verbose output as pictured below. Two points of information to take from the !analyze-v output in this instance is Arg4 (the blocked irp) and the FAILURE_BUCKET_ID and BUCKET_ID (pictured below).



Type !irp into the command box followed by the value of Arg4 and hit enter.
Code:

The output from !irp shows that the third stack is active and blames athrx.sys for the DRIVER_POWER_STATE_FAILURE.
Code:

As alluded to in the previous !analyze-v output screenshot, athrx.sys has been named in the current stack. Referencing that driver name in the drivers.txt file you can see that the driver is Qualcomm Atheros network adapter driver, this is the driver responsible for the DRIVER_POWER_STATE_FAILURE at system shutdown. Running the lmvm athrx command shows that the driver is old and needs updating.
Code:

PART FOUR [/i] Common Bugchecks and Plans of Attack
There are several bugchecks that appear on a regular basis which should nearly always require the same course of action, two of them are covered below.

• 0x116 (VIDEO_TDR_ERROR)
  
• 0x124 (WHEA_UNCORRECTABLE_ERROR)

0x116 [/i] VIDEO_TDR_ERROR
A VIDEO_TDR_ERROR always relates to the system's graphics card (TDR = Timeout Detection Recovery). This could be down to a number of reasons including drivers, heat, power to the card, a faulty PCI-e slot or simply a defective GPU. It's in your best interests to employ common sense trouble shooting techniques in these situations.

• Try later and/or earlier drivers
  
• Measure the GPU's temperatures at idle and under load
  
• Ensure the GPU is free from dust build up and that the fan is working correctly
  
• Reseat the GPU and check all connections are securely made
  
• Check PSU voltages in BIOS
  
• Try a different card in the system

0x124 [/i] WHEA_UNCORRECTABLE_ERROR
A 0x124 is one of the worst STOP codes to encounter for the sole reason that the dump files usually give nothing away as to the cause of the problem. WHEA (Windows Hardware Error Architecture) errors signify a problem with hardware but very rarely pinpoint the culprit. In these scenarios it is advised to run a series of hardware stress and diagnostic tests to try and pinpoint the problem. A template is offered below which covers the four main components (GPU, CPU, RAM, HDD) and gives you a fighting chance of narrowing down the problematic device.



Information While 0x124s are usually caused by defective hardware it has been proven on occasion for software to be the culprit. See 0x124 software section for information.

To attempt dig deeper into a 0x124 use the !analyze-v command, in the example below you'll see the PROCESS_NAME: audiodg.exe.



In this particular case there were also 0x1000007E (SYSTEM_THREAD_EXCEPTION_NOT_HANDLED_M) dumps which identified a Logitech Headset driver (dated 2013) as an issue. This tied in with the audio reference highlighted in the 0x124 so the recommended action would be to update the driver for the headset, negating the need to stress test components.

If there is no clear indicator within the verbose output run !errrec on Arg2 the Address of the WHEA_ERROR_RECORD structure.



Again this will often result in a dead end, the !errrec output from the above dump reveals nothing of use. This is a very common output from this particular command within 0x124s.
Code:

SOFTWARE RELATED 0x124s
On more than one instance software has been identified as being the cause of STOP 0x124s, namely Gigabyte motherboard utilities. Follow the links below for more detailed information and analysis.

• BSOD While Watching Videos Online
  
• BSOD When Computer is Idle
  
• BSOD After New Motherboard and CPU

0x124 TROUBLESHOOTING TEMPLATE


Tip Any of the stress or diagnostic tests below can be used individually to diagnose hardware faults outside 0x124 situations.



STOP 0x124 Troubleshooting [/i] Read carefully before proceeding.


Warning

If you're overclocking your system, revert back to stock clocks now.





Note Test the system once the overclock is removed before continuing with the steps outlined below.



ONE [/i] CPU TEST


Run Prime95 to stress test your CPU. Prime95 Tutorial



Warning Your CPU temperatures will rise quickly while under this stress test. Keep a keen eye on them and abort the test if temperatures rise too high.



TWO [/i] RAM TEST


Run MemTest86+ to analyse your RAM. MemTest86+ Tutorial







Note MemTest86+ needs to be run for at least 8 passes for conclusive results. Set it running before you go to bed and leave it overnight. We're looking for zero errors here. Even a single error will indicate RAM failure.



THREE [/i] HDD TEST


Run SeaTools to check the integrity of your HDD. SeaTools Tutorial



Run chkdsk Chkdsk




FOUR [/i] GPU TEST


Run Furmark to stress test your GPU. Furmark Tutorial


Note Run Furmark for around 30 minutes.

Warning Your GPU temperatures will rise quickly while Furmark is running. Keep a keen eye on them and abort the test if temperatures rise too high.



PART FIVE [/i] SCSI Pass-Thru Direct (SPTD)


Information SCSI Pass-Thru Direct (SPTD) is a proprietary API and device driver which uses an alternative method of accessing SCSI devices. SPTD is bundled into DAEMON Tools and Alcohol 120% and is a proven cause of countless BSODs. Uninstalling DAEMON Tools and/or Alcohol 120% does not remove SPTD from the system, it has to be uninstalled using an additional process outlined below.
To check if DAEMON Tools and/or Alcohol 120% is installed on a system:

1: Open the selected archive as per Step 1 C)

2: Locate the msinfo32.nfo file and double click to open the file. This will enable you to see the details of the system experiencing the BSODs.



3: With the msinfo32.nfo file opened click on the + sign next to Software Environment in the left hand navigation pane to expand the section then click on Startup Programs in the expanded menu. Browse the program list to see if DAEMON Tools or Alcohol 120% are installed.


4: If either program is installed it is best practice to recommend the user uninstall them. Use the template below to instruct the user on how to completely uninstall SPTD. Delete whichever program is not applicable.
TEMPLATE:

1. Uninstall Daemon Tools/Alcohol 120%.2. Download the SPTD standalone installer and perform the following steps.3. Double click the executable to open it.4. Click the button shown below.





Information If the button it is grayed out as shown in the image, there is no more SPTD installation on your system and you can just close the window.



PART SIX [/i] Recommended Research Materials
Learning to debug BSOD crash dumps is no small feat and a knowledge of the internal of Windows can be hugely advantageous. The three books listed below will offer a vast insight into the workings of Windows and furnish you with a greater knowledge of what's actually happening within dump files.

• Advanced Windows Debugging by Mario Hewardt and Daniel Pravat
• Windows Internals 6th Edition Part 1 by Mark Russinovich, David Solomon and Alex Ionescu
• Windows Internals 6th Edition Part 2 by Mark Russinovich, David Solomon and Alex Ionescu

This concludes this beginner's guide to debugging BSODs in Windows 10, if you have any questions please feel free to ask in the thread.

Gav.



Related Tutorials

• How to Install and Configure WinDBG for BSOD Analysis
• BSOD Minidump - Configure to Create in Windows 10 - Windows 10 Forums

:)

 

BAD_POOL_CALLER x000000c2 bsod in WIndows 10

I use WinDbg (Windows Debugger) to open and analyze dump files.

Here is a good link on the basics for debugging in Windows 10:

WinDBG - The Basics for Debugging Crash Dumps in Windows 10

 

When the Laptop wakes up from hibernation it restarts with DPC WATCHDOG VIOLATION stopcode

You're welcome.

I use WinDbg (Windows Debugger).

Here is a good link on the basics for debugging in Windows 10:

WinDBG - The Basics for Debugging Crash Dumps in Windows 10

 

Updated to include information on SCSI Pass-Thru Direct.

 

Very good tutorial Gav!

 

Wow Gav mate that is some piece of work can I take it (without having thoroughly read it yet) as the same form of dealing with BSOD's in 7?

 

It is the same for Windows 7 to 10 *Smile

 

Thanks Martjin it just looked a little more detailed which is really what I was after perhaps I won't make too many mistakes this time*Wink

 

I hope someone can help me with this:
When I type in dps Limit Base
I should get the name of the offending driver/drivers within the wall of text,but it isn't the case when I use it.
Can somebody throw a light on this?
Example dump attachedMinidump.zip

 

It would be better if you show us what you get, I have no idea which one of those dumps you are reading, let's stick to one dump and see what might be the problem.

It could also be that no driver gets flagged after doing that, then you have to resort to other methods.

Looking at this dump real quick,

052816-34953-01.zip

There is no driver getting flagged so you wouldn't see it.

 

Thanks.I was examining the newest dump. Nevermind,I understand it isn't necessary that driver gets flagged in BSOD.What are the other methods?

 

This is from the bottom of that section,

 

Sorry to bother you experts again, but I want to know if there is a way from the minidumps only to get the drivers list loaded during the crash with date of the driver?
In MS community forums we don't use DM log collector so we don't have a driver.txt file.

 

The drivers list can be found without command, in the menu go to debug and choose modules.

 

Debugging a crash and i need help. Don't want you to do all the work, just give me a way to follow.