Everything you ever wanted to ask about DBGHELP.DLL, IMAGEHLP.DLL and SFP
Plus a little Python thrown in.
This document will give you some background information how you can retrieve C/C++ debug information in Win32 systems (that is: Windows NT/2000/XP/2003, but not 16-bit systems like 98 and ME). In the course of doing so, I will present to you a minimal debugger, written in python, that has some benefits over using "fullblown" debugging systems - for example, it is faster and you can script it to watch specific function calls. Neat, right?
Note: Python is used here for writing a debugger for C/C++ or, more generally, any binary Windows application. We will not discuss debugging Python programms - we will discuss writting a Debugger in python. That is different.
What are those DLLs?
DBGHELP.DLLanalyzes debug information in a file / in a running process. For example, you can use it to do a StackWalk, find symbols for an address and similar stuff.
IMAGEHLP.DLLanalyzes the PE structure of a file. PE is short for "Portable Executable" and is the file format of a lot (but of course not all) Microsoft files. For example, you can use it to get to the export sections in a DLL.
PSAPI.DLLanalyzes running processes and gives you status information for them. For example, you can use it to enumerate all processes in the system, find out loaded modules and so on.
Which DLL is the most important? There is no answer to this question, because all of these are important for a debugger.
Where is the problem?
These DLLs - or rather, some of these DLLs - have been distributed as part of the operating system since (at least) Windows NT 4.0. Some Microsoft applications (such as Developer Studio) have also shipped these. Of course, every OS and every app has a different version - does the term "DLL hell" ring a bell? Basically, there are two problems:
- Some machines do not have all of those DLLs
- Many systems have old or mismatching versions of those DLLs, which results, at best, in mayhem.
Which version should I use?
Microsoft brings out new versions of
DBGHELP.DLL as part of the freely available
Debugging Tools for Windows.
If you are a C/C++ developer, you should install these tools on your machine, for the following reasons:
- They include the latest
DBGHELP.DLLwhich, trust me, you want.
- They include the great Kernel Debugger, which you can use to debug a remote machine. Its fun!
- They include the symbol server / remote symbol handler, which I will discuss further down below.
Since releasing "Windows 2000", Microsoft has packaged the
DBGHELP.DLL (as well as the other two) with the operating system,
and protected them using SFP (System File Protection). This, bascially, sucks.
It means that you cannot replace the DLL - even though the version is completely outdated,
broken, and missing a lot of features. The version that you really want is the version that is part of the
Debugging Tools for Windows.
The problem is not so big regarding
PSAPI.DLL - these versions are fairly OK, but missing on NT4 systems.
How do I install a new DBGHELP.DLL?
These are your options:
- Copy your binary to the
- Copy the
DBGHELP.DLLto your applications' directory
DBGHELP.DLLto something else, but unique, such as
TPKHELP.DLL. Next, patch your binary to link to
- Boot 2000/XP in Safe Mode. For reasons only Microsoft knows, "safe mode" means disabled "system file protection". Go
ahead and figure it out. In safe mode, you can replace the DLL in system32 (and, necessarily, in
dllcache, see below).
- Find all
DBGHELP.DLLfiles (there are likely to be several, in
ServicePackFiles. Kill em all. You'll get a Windows warning to insert the original CD to restore these. Cancel this. Copy your new version. You'll get a Windows warning to insert the original CD to restore these. Cancel this. Curse SFP.
If you think that all of these options are more or less crap, then - welcome to the club!
Setting up a symbol server
Problem: Windows has no debug symbols. If you installed the ones you can find on your Install CDs (MSDN version), you will very soon be out of luck: because every time you visit WindowsUpdate to fix another "remote arbitrary code execution" bug, you get new versions and they don't match the symbols in your system.
Solution: Well, Microsoft has had a pretty good idea, actually: installing a public symbol server. You can connect to that machine (provided you are on the network) and get the correct dbg information for all components on your system (on demand), cached locally in a directory of your choice. Nice!
To install, create a system wide environment variable:
Logoff / Logon, and you should be fine. Note that
c:\cache will be the cache directory, you can change that name.
Note: For this to work you need the
symsrv.dll, which is part of the
Debugging Tools for Windows.
I have mentioned this URL now three times, and I'm pretty sure that by now you have them installed on your system, right?
Well, add the directory to your path and you're fine.
Using Python to debug stuff
I have written a Python23 extension DLL
pydebug.pyd, that you can use to write debuggerlike applications
in Python. You can use it to dump symbol tables, find symbols in memory etc. I have also written a
mini Debugger that you can use as a starting point,
procman.py. Before you can see what
procman.py can do for you, you need to install it. Duh.
- You need Python 2.3 and the Win32 extensions installed.
- You need to install the a refreshed
DBGHELP.DLLas described above. The easiest way of doing so is downloading the Debugging Tools for Windows and using
sfcrestore.py(see code above!).
- Download procman.zip. It is freeware and includes the full source.
- You might want to setup the MS symbol server as described above.
To test it: try this:
python procman.py notepad
After you have closed Notepad, open
pyman32.log and enjoy.
procman.py do for you?
First off, it is (considerably) faster than a normal debugger. So, if you have something
that is a timing issue, you can try it with
Second, you can use it to set breakpoints at function calls to DLLs. E.g. to
Check the source, you'll find these lines:
self.patches = [ PatchThis( "KERNEL32!LoadLibraryA", args = [("LPSTR","lpszLibrary")], result = "WINAPI" ) ]
As you can see, this is a list so feel free to roll your own.
Third, its free, and by toying around with the source you can learn stuff about