Memory Dumps Done Right

Memory Dumps Done RightSoftware has bugs, it’s just a fact of life. When troubleshooting a problem with an application, one of the most valuable bits of information you can have is the state of the entire application at the time of a problem. This is especially true for an application that is currently running and cannot be interrupted (i.e. restarted). But how do you get more details on the application’s current state?

The common solution is to add verbose logging to the application itself. For .NET apps on Windows this commonly entails using logging frameworks like log4net that provide ways for operation teams to dynamically increase log message verbosity at run time via configuration. Unfortunately this depends on the developer(s) making sure they write the application code to log every relevant detail and in all but the most basic applications it can be difficult to ensure the application is logging enough detail. So what else can we do to get a complete view of an application’s current state while it’s running?

Well, fortunately Windows provides ways to capture memory dumps. A memory dump is just that: a copy (dump) of an application’s state exactly as it was while it was running in at the time the dump was captured. In Windows, there are two kinds of memory dumps for typical user-mode applications :

  1. Full dump: this includes the entire memory space of the process
  2. Mini dump: this includes only selected parts of the memory associated with the process

More information can be found in Microsoft’s Documentation.

Memory dumps sound great, but how do we create one? One of the most obvious ways is via the Task Manager. Simply fire up Task Manager, find the process your interested in on the Processes tab, right click the process and select “Create Dump File”.

Memory Dump in Task Manager

The problem with this approach is that Task manager doesn’t make it obvious whether you’re getting a memory dump from a 32-bit process or a 64-bit process. This is very important because if you take a memory dump of a 64-bit process from a 32-bit Task Manager or vice versa, all sorts of weirdness will arise when later analyzing the memory dump.

So the moral of the story is you need to know ahead of time if the process for which you are trying to get a memory dump was built for 64-bit (x64) or 32-bit (x86). Your development team would be happy to answer that, or there are several utilities you can use as well.

Once you know the bitness for which the application was built, you can then launch the appropriate Task Manager:

  • For capturing 64-bit (x64) memory dumps: C:\Windows\system32\taskmgr.exe
  • For capturing 32-bit (x86) memory dumps: C:\Windows\SysWOW64\taskmgr.exe

This already seems like a lot of rigmarole to remember and involves a lot of point and click.

While there are many ways to capture a memory dump on Windows, we recommend using the ProcDump utility from the Windows SysInternals package. As mentioned before, we recommend putting the entire PsTools suite on every one of your Windows workstations and servers in a known location like C:\PsTools.

ProcDump gives you a single command line interface to capture memory dumps for 64-bit or 32-bit process. It has many other options that let you choose between a full dump or a mini-dump, whether to capture the dump in “hang mode” (attach to the process, copy its memory space, and detach without interruption) or crash mode (attach to the process, and if it throws an unhandled exception, copy its memory space before letting it shut down).

Here’s a a typical set of ProcDump command lines to remember to capture a full “hang mode” memory dump that should cover most use cases:

  • For 32-bit (x86) processes: procdump.exe -ma <PID> C:\MyApp.dmp
  • For 64-bit (x64) processes: procdump.exe -64 -ma <PID> C:\MyApp.dmp

Replace <PID> with the process ID of the process you want to capture, and replace the path “C:\MyApp.dmp” with the path and filename to which you want the memory dump saved. These command lines will cause procdump to attach to the process, write out a memory dump, and detach without interrupting the process. How long the dump process takes depends on the application. It can be just a few seconds or 5 minutes or more.

ProcDump can also capture memory dumps by the name of the executable. Run “procdump -?” for an explanation of all the options.

We’ll talk more about analyzing the contents of a memory dump in a future article.

Again there are many different ways to capture a memory dump on Windows, just remember to be aware of the application’s target platform: 64-bit (x64) or 32-bit (x86).

 
Comments

No comments yet.

Leave a Reply