Interview with Stephen Teilhet, author of "Subclassing and Hooking With Visual Basic"

vbRad: Let's clear up one thing for our readers. This book is NOT about prostitution, right?
steilhet: Yes, you are one of the few that made that connection. Actually, I used to joke about it when the book was in its infancy and I got a lot of strange looks.
vbRad: Seriously speaking, did the publisher give you any grief about the book title? After all, "hooking" means something else entirely outside the scope of Windows programming.
steilhet: No, actually, my original idea was just a book about using hooks within the VB realm and he (Ron Petrusha) prompted me to add subclassing to the book. Which worked out great, since subclassing and hooking are very similar in ways.
vbRad: Speaking of the title, could you please give our readers (who haven't been exposed to it) a quick definition of what subclassing/hooking is all about?
steilhet: In simplest terms, both techniques intercept windows messages that are sent back and forth in the system. If you've ever used a tool such as Spy++ you can see the amount and types of messages that are passed between objects in the Windows OS. This is where the similarities end, though. Subclassing and each of the 15 different hooks intercept different messages at different points in the message stream. For this reason it is important to do your homework before choosing a particuar hook or type of subclassing in which to use for an application.
vbRad: What real-world experience do you have with subclassing/hooking that will make your readers feel confident that you are the right source for this type of information? (i.e. how did you come to work with hooking/subclassing)
steilhet: There are two projects that I have worked on that come to mind. On each project the requirement was to use VB, but I also needed more control over the application windows that was allowed through VB. Some examples are controling the size and position of the form. Once I progressed through these projects, I became more and more interested in these techniques and started to study them on my own. In doing so I was also able to help numerous others in debugging and completing projects in which subclassing and hooking were being used. In fact, these techniques are used throughout MFC projects in VC++ and I was able to help several people debug code and also help them get a better understanding of what MFC was doing. I also have at least one personal project that I take on by myself every so often, they always involve learning how something works. In this instance it was how automated test applications operated. I learned that by using several hooks WH_CBT, WH_JOURNALRECORD, and WH_JOURNALPLAYBACK to name a few, I could create a fully functional test application. At that time I was planning on writing an article on the technique, but John Robbins beat me to the punch with two articles that he wrote in the MSDN magazine.
vbRad: Speaking of hooks like WH_JOURNALRECORD, is it possible to extend these or other hooks to intercept other system activities (like disk writing activity), or are Windows pretty much limited to the ones you describe in the book (i.e. the 15 hooks).
steilhet: Unfortunately, I know of no way to extend these hooks. They are built-in to the operating system to intercept specific messages at specific points. There is a web site (www.sysinternals.com) that specialize in very low level techniques which employ techniques used in device driver building to watch hardware events such as writing to the disk, but this is much more involved that using the Windows hooks. There is a documented 16th hook (WH_HARDWARE) that was never incorporated in any version of Windows. To my knowledge, it is not slated to be incorporated into Windows either. This hook may have been intended for system activities such as watching for disk or other hardware activity to occur.
vbRad: That's too bad, it would have been useful :-) What are the downsides to subclassing/hooking for an average VB programmer working for a standard corporation? Would you classify subclassing/hooking to be "safe code"?
steilhet: I would definately classify subclassing/hooking to be "safe code", but only if you really understand what you are doing. I remember my first foray into subclassing was to copy some subclassing code out of a magazine and use it. Had it not worked correctly with my other code, I could have encountered stack overflow conditions, frozen the application (or all applications, possibly), or some other fault may have occured. This is not a technique to be taken lightly. If you do not know the implications of using subclassing/hooking it may seem that this type of code is unsafe. In writing the code for the book, I ran into many problems, but most of them were easily corrected. In some cases I may have simply not discarded a message and let it pass on to its destination window. I remember one particularly bad debugging instance when my app would infinately recurse whenever I started it. By using Spy++ I could see that it was actually setting up an infinite loop whenever I would subclass a text box and print out every message that was being written to it. As it happened, whenever text is written to a text box, many messages are sent to that text box in order to write the text in its client area. Each of those message were being picked up and written to the text box, and on and on, so that it never stopped writing text to the text box. This brings up a good point, get to know and love tools such as Spy++ when debugging subclassing/hooking code.
vbRad: Other than Spy++, which can be a bit scary for someone who takes an initial foray into the world of windows messages, what other tools would you recommend Also, how often does something come up in your daily work that requires use of subclassing or hooking and necessitates use of tools like Spy++?
steilhet: Here are some of the tools that I use, note that most of them are not as useful as Spy++ for subclassing/hooking: Process Explorer, DebugView, Depends, File Monitor, Registry Monitor, Process Viewer, WinDiff, Fusion Log Viewer (.NET only), ILDasm (.NET only), and of course Spy++. There are others, but I do not use them nearly as often. For subclassing/hooking there really is not substitute for Spy++. Subclassing and hooking rely almost entirely on the messages sent between objects in the Windows OS, therefore, Spy++, which watches messages, is the best tool to see what is going on inside of your applications (or others for that matter). The Depends and Process Viewer tools are great for looking at the internal makeup of a DLL, EXE, or an entire process. This helps a lot with system hook, which are set up inside of Win32 DLL's and pulled in to other running processes. I know Spy++ may spit out a lot of information, but learing how to use it can make a world of difference when debugging an app that uses subclassing and/or hooking. As for how often subclassing/hooking projects come up in my daily work, I would have to say not often. Although, I have had to do some investigative/debugging work with a lot of legacy code that uses subclassing, in these instances I have frequently resorted to Spy++ to see what is happening. It is often hard to get new projects going that employ subclassing/hooking because of the lack of knowlege about it (especially in the VB community) or thinking that it is a secret art available only to the most proficient C++ developers. It is my hope that this book will eradicate that lack of knowlege and allow others to use these techniques safely and efficiently.
vbRad: That's true. Subclassing does have a bit of that stigma, also associated in part that it is difficult to get good information about it. Hopefully this book will dispel this myth and help programmers see what it is all about. It has certainly helped me clear up some things that were fuzzy in the past. You have a chapter in the book on .NET. It seems that subclassing won't be used as much, because it is incorporated into the framework, while hooks are not. Does this apply to VB.NET only or to C# as well? Does C# provide an easier way to handle hooks? Would a hook then be an "unmanaged code" in the .NET framework?
steilhet: Actually, it is a very small effort to port either the subclassing or hooking code from VB.NET to C#. Hooks are actually a little more simple to use in C# due to the way that delegates are declared and used in C# versus VB.NET. The filter function that you write for the hook in any .NET language will be managed, but you must call out to unamanaged code via P/Invoke to call the SetWindowHookEx and UnHookWindowsHookEx API functions when setting and removing the hooks. The syntax for calling out to unmanaged API's via P/Invoke is slightly different in VB.NET and C#.
vbRad: You state in the book that it is impossible to subclass a window in another process. There is a product out there from Desaware that claims to do just that. What method, in your opinion, do they use to achieve this functionality? Do they simply hook in to the entire Windows message stream and filter out all the unnecessary messages?
steilhet: Yes, I remember reading that too. To do this you would need to create a system wide hook using C++. You would then determine if you need to use the WH_CALLWNDPROC hook (intercepts only sent messages) or the WH_GETMESSAGE hook (intercepts only posted messages) or both. This system wide hook would then be created inside of a Win32 DLL and injected into every running process. Your filter function for the hook is contained inside the DLL and is injected into each of the processes as well. This filter function then needs to look for messages directed towards one or more windows in which you want to "subclass". This is obviously very simplified. There is much more to it. There is a book "Advanced Windows" by Jeffrey Richter that describes this process in some better detail although it is entirely in C++. I did mention in the book that using these hooks one could in essence subclass a window outside of its process. I did not go into this since it mainly involves using C++ and the book was focused more towards the VB and VB.NET developer. Actually, it is kind of misleading, you are really using hooks to simulate subclassing a window in another process. Using hooks outside of your applications process (system-wide hooks) are very interesting, but beware, they are much more difficult to debug and they pose a serious drain on system wide performance if not used correctly (sometimes even when used correctly, they can put a drain on system performance). So, use this technique with caution.
vbRad: And the last question. Having never written a book before, what compelled you to start writing?
steilhet: It might sound a little silly, but at my previous job I was getting very bored with what I was doing. I was working on building another database access interface (a VB form with a bunch of text boxes on it). There were a few times where I had some down time while I was waiting for components to be developed by other groups. During these times my mind never stayed at rest, I was always coming up with ideas. One of them manifested itself through an article I wrote for VBPJ (Feb 2000) entitled "Making your boolean logic error free". That was my very first foray into professional writing. I was planning more articles on using the various hooks from within VB, but as the ideas grew I toyed with the idea of writing a book. At first it was almost a joke, but as the ideas grew in size and number I took a chance with a reputable publisher (O'Reilly and Associates) and sent in a 12 page proposal. The editor called me up and interviewed me to make sure I would be willing to put in the time and that I was serious about the book. Afterwards, he decided to take a chance on me and the rest is history. Writing this book has been very fulfilling for me, it has been especially fulfilling for me intellectually. I have grown to love writing and have plans in the works for a new .NET book. You will be pleased to know that it is not just another how to write VB.NET or C# code it is much more involved. It will basically begin where those books leave off.
vbRad: That's great, because I am kind of bored with those types of books already - they rehash what should be (will be?) in the online help anyway. Thanks for taking the time to do the interview, I wish you continued luck with this and the next book. We
steilhet: One more thing, I should mention that the code on the O'Reilly web site uses the Beta 1 version of VB.NET for the examples in chapters 22 & 23. I need to update that code to reflect the changes in Beta 2
vbRad: We'll look forward to the updated code.
steilhet: Thanks, for having me.

Want more reviews and sample chapters? Click here.